Re: [PATCH v3 4/9] kunit: Add ability to filter attributes

From: Guenter Roeck
Date: Tue Aug 29 2023 - 18:47:21 EST

Hi,

On Tue, Jul 25, 2023 at 09:25:15PM +0000, Rae Moar wrote:
> Add filtering of test attributes. Users can filter tests using the
> module_param called "filter".
>
> Filters are imputed in the format: <attribute_name><operation><value>
>
> Example: kunit.filter="speed>slow"
>
> Operations include: >, <, >=, <=, !=, and =. These operations will act the
> same for attributes of the same type but may not between types.
>
> Note multiple filters can be inputted by separating them with a comma.
> Example: kunit.filter="speed=slow, module!=example"
>
> Since both suites and test cases can have attributes, there may be
> conflicts. The process of filtering follows these rules:
> - Filtering always operates at a per-test level.
> - If a test has an attribute set, then the test's value is filtered on.
> - Otherwise, the value falls back to the suite's value.
> - If neither are set, the attribute has a global "default" value, which
> is used.
>
> Filtered tests will not be run or show in output. The tests can instead be
> skipped using the configurable option "kunit.filter_action=skip".
>
> Note the default settings for running tests remains unfiltered.
>
> Finally, add "filter" methods for the speed and module attributes to parse
> and compare attribute values.
>
> Note this filtering functionality will be added to kunit.py in the next
> patch.
>
> Reviewed-by: David Gow <davidgow@xxxxxxxxxx>
> Signed-off-by: Rae Moar <rmoar@xxxxxxxxxx>

This patch causes all my s390 boot tests to crash.

[ 17.009188] # module: kunit
[ 17.009239] 1..8
[ 17.014967] ok 1 parse_filter_test
[ 17.020755] ok 2 filter_suites_test
[ 17.026350] ok 3 filter_suites_test_glob_test
[ 17.033102] ok 4 filter_suites_to_empty_test
[ 17.035422] Unable to handle kernel pointer dereference in virtual kernel address space
[ 17.035546] Failing address: 0000000001501000 TEID: 0000000001501407
[ 17.035617] Fault in home space mode while using kernel ASCE.
[ 17.035833] AS:0000000002164007 R3:000000001f71c007 S:000000001f711000 P:000000000150131f
[ 17.036313] Oops: 0004 ilc:2 [#1] SMP
[ 17.036851] Modules linked in:
[ 17.037026] CPU: 0 PID: 193 Comm: kunit_try_catch Tainted: G N 6.5.0-01207-g1c59d383390f #1
[ 17.037106] Hardware name: QEMU 8561 QEMU (KVM/Linux)
[ 17.037199] Krnl PSW : 0704e00180000000 0000000000950e18 (kunit_next_attr_filter+0xc8/0x1c8)
[ 17.037451] R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 RI:0 EA:3
[ 17.037531] Krnl GPRS: 0000037f00000000 000000000150120a 000000000000003e 000000000000003e
[ 17.037583] 0000038000197e04 0000038000197c04 0000038000197df0 000003800000000a
[ 17.037629] 0000038000000005 0000000001501203 00000000015011fe 0000037f0000000c
[ 17.037675] 0000000005404100 0000038000197e08 0000000000950dda 0000038000197cd8
[ 17.038176] Krnl Code: 0000000000950e0a: b9180098 agfr %r9,%r8
[ 17.038176] 0000000000950e0e: a7080000 lhi %r0,0
[ 17.038176] #0000000000950e12: e32090000090 llgc %r2,0(%r9)
[ 17.038176] >0000000000950e18: 92009000 mvi 0(%r9),0
[ 17.038176] 0000000000950e1c: b904003a lgr %r3,%r10
[ 17.038176] 0000000000950e20: c418007bd73c lgrl %r1,00000000018cbc98
[ 17.038176] 0000000000950e26: b25d0013 clst %r1,%r3
[ 17.038176] 0000000000950e2a: a714fffe brc 1,0000000000950e26
[ 17.038625] Call Trace:
[ 17.038679] [<0000000000950e18>] kunit_next_attr_filter+0xc8/0x1c8
[ 17.038749] [<000000000094e5f2>] parse_filter_attr_test+0x92/0x360
[ 17.038796] [<000000000094e39a>] kunit_generic_run_threadfn_adapter+0x32/0x48
[ 17.038840] [<0000000000186702>] kthread+0x142/0x148
[ 17.038884] [<0000000000103138>] __ret_from_fork+0x40/0x58
[ 17.038927] [<000000000102df9a>] ret_from_fork+0xa/0x30
[ 17.038984] INFO: lockdep is turned off.
[ 17.039027] Last Breaking-Event-Address:
[ 17.039062] [<0000000000950da4>] kunit_next_attr_filter+0x54/0x1c8
[ 17.039303] Kernel panic - not syncing: Fatal exception: panic_on_oops

Bisect log is attached.

Guenter

---
# bad: [1c59d383390f970b891b503b7f79b63a02db2ec5] Merge tag 'linux-kselftest-nolibc-6.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
# good: [2dde18cd1d8fac735875f2e4987f11817cc0bc2c] Linux 6.5
git bisect start '1c59d383390f' 'v6.5'
# good: [42a7f6e3ffe06308c1ec43a7dac39a27de101574] Merge tag 'x86_microcode_for_v6.6_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
git bisect good 42a7f6e3ffe06308c1ec43a7dac39a27de101574
# good: [330235e87410349042468b52baff02af7cb7d331] Merge tag 'acpi-6.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm
git bisect good 330235e87410349042468b52baff02af7cb7d331
# good: [5a31cc7297072a7266a910ca5266b640d27803b4] Merge tag 'linux-kselftest-next-6.6-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
git bisect good 5a31cc7297072a7266a910ca5266b640d27803b4
# good: [eea70cdac61087fdb7a1c1e6f306c0bf38ec39d4] tools/nolibc: riscv: shrink _start with _start_c
git bisect good eea70cdac61087fdb7a1c1e6f306c0bf38ec39d4
# good: [faeb4e09fe77262f9a6b2f9f874eec0b6850721e] selftests/nolibc: add test support for ppc64
git bisect good faeb4e09fe77262f9a6b2f9f874eec0b6850721e
# bad: [76066f93f1df27657eb937b7c9c091e3a6abf4db] kunit: add tests for filtering attributes
git bisect bad 76066f93f1df27657eb937b7c9c091e3a6abf4db
# good: [64bd4641310c41a1ecf07c13c67bc0ed61045dfd] MAINTAINERS: add Rust KUnit files to the KUnit entry
git bisect good 64bd4641310c41a1ecf07c13c67bc0ed61045dfd
# bad: [529534e8cba3e60f843a682e2a3149612b30d608] kunit: Add ability to filter attributes
git bisect bad 529534e8cba3e60f843a682e2a3149612b30d608
# good: [02c2d0c2a84172c3c7ec0229c61db55d23dd4730] kunit: Add speed attribute
git bisect good 02c2d0c2a84172c3c7ec0229c61db55d23dd4730
# good: [a00a72709175a4d53096301a8792b8171d1223e5] kunit: Add module attribute
git bisect good a00a72709175a4d53096301a8792b8171d1223e5
# first bad commit: [529534e8cba3e60f843a682e2a3149612b30d608] kunit: Add ability to filter attributes