kunit: Mark filter* params as rw

By allowing the filter_glob parameter to be written to, it's possible to
tweak the testsuites that will be executed on new module loads. This
makes it easier to run specific tests without having to reload kunit and
provides a way to filter tests on real HW even if kunit is builtin.
Example for xe driver:

1) Run just 1 test
# echo -n xe_bo > /sys/module/kunit/parameters/filter_glob
# modprobe -r xe_live_test
# modprobe xe_live_test
# ls /sys/kernel/debug/kunit/
xe_bo

2) Run all tests
# echo \* > /sys/module/kunit/parameters/filter_glob
# modprobe -r xe_live_test
# modprobe xe_live_test
# ls /sys/kernel/debug/kunit/
xe_bo xe_dma_buf xe_migrate xe_mocs

For completeness and to cover other use cases, also change filter and
filter_action to rw.

Link: https://lore.kernel.org/intel-xe/dzacvbdditbneiu3e3fmstjmttcbne44yspumpkd6sjn56jqpk@vxu7sksbqrp6/
Reviewed-by: Rae Moar <rmoar@google.com>
Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: run test suites only after module initialization completes

Commit 2810c1e99867 ("kunit: Fix wild-memory-access bug in
kunit_free_suite_set()") fixed a wild-memory-access bug that could have
happened during the loading phase of test suites built and executed as
loadable modules. However, it also introduced a problematic side effect
that causes test suites modules to crash when they attempt to register
fake devices.

When a module is loaded, it traverses the MODULE_STATE_UNFORMED and
MODULE_STATE_COMING states before reaching the normal operating state
MODULE_STATE_LIVE. Finally, when the module is removed, it moves to
MODULE_STATE_GOING before being released. However, if the loading
function load_module() fails between complete_formation() and
do_init_module(), the module goes directly from MODULE_STATE_COMING to
MODULE_STATE_GOING without passing through MODULE_STATE_LIVE.

This behavior was causing kunit_module_exit() to be called without
having first executed kunit_module_init(). Since kunit_module_exit() is
responsible for freeing the memory allocated by kunit_module_init()
through kunit_filter_suites(), this behavior was resulting in a
wild-memory-access bug.

Commit 2810c1e99867 ("kunit: Fix wild-memory-access bug in
kunit_free_suite_set()") fixed this issue by running the tests when the
module is still in MODULE_STATE_COMING. However, modules in that state
are not fully initialized, lacking sysfs kobjects. Therefore, if a test
module attempts to register a fake device, it will inevitably crash.

This patch proposes a different approach to fix the original
wild-memory-access bug while restoring the normal module execution flow
by making kunit_module_exit() able to detect if kunit_module_init() has
previously initialized the tests suite set. In this way, test modules
can once again register fake devices without crashing.

This behavior is achieved by checking whether mod->kunit_suites is a
virtual or direct mapping address. If it is a virtual address, then
kunit_module_init() has allocated the suite_set in kunit_filter_suites()
using kmalloc_array(). On the contrary, if mod->kunit_suites is still
pointing to the original address that was set when looking up the
.kunit_test_suites section of the module, then the loading phase has
failed and there's no memory to be freed.

v4:
- rebased on 6.8
- noted that kunit_filter_suites() must return a virtual address
v3:
- add a comment to clarify why the start address is checked
v2:
- add include <linux/mm.h>

Fixes: 2810c1e99867 ("kunit: Fix wild-memory-access bug in kunit_free_suite_set()")
Reviewed-by: David Gow <davidgow@google.com>
Tested-by: Rae Moar <rmoar@google.com>
Tested-by: Richard Fitzgerald <rf@opensource.cirrus.com>
Reviewed-by: Javier Martinez Canillas <javierm@redhat.com>
Signed-off-by: Marco Pagani <marpagan@redhat.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: add is_init test attribute

Add is_init test attribute of type bool. Add to_string, get, and filter
methods to lib/kunit/attributes.c.

Mark each of the tests in the init section with the is_init=true attribute.

Add is_init to the attributes documentation.

Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Rae Moar <rmoar@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: add KUNIT_INIT_TABLE to init linker section

Add KUNIT_INIT_TABLE to the INIT_DATA linker section.

Alter the KUnit macros to create init tests:
kunit_test_init_section_suites

Update lib/kunit/executor.c to run both the suites in KUNIT_TABLE and
KUNIT_INIT_TABLE.

Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Rae Moar <rmoar@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: Fix possible memory leak in kunit_filter_suites()

If the outer layer for loop is iterated more than once and it fails not
in the first iteration, the filtered_suite and filtered_suite->test_cases
allocated in the last kunit_filter_attr_tests() in last inner for loop
is leaked.

So add a new free_filtered_suite err label and free the filtered_suite
and filtered_suite->test_cases so far. And change kmalloc_array of copy
to kcalloc to Clear the copy to make the kfree safe.

Fixes: 529534e8cba3 ("kunit: Add ability to filter attributes")
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
Reviewed-by: Rae Moar <rmoar@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: Fix the wrong kfree of copy for kunit_filter_suites()

If the outer layer for loop is iterated more than once and it fails not
in the first iteration, the copy pointer has been moved. So it should free
the original copy's backup copy_start.

Fixes: abbf73816b6f ("kunit: fix possible memory leak in kunit_filter_suites()")
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
Reviewed-by: Rae Moar <rmoar@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: Fix missed memory release in kunit_free_suite_set()

modprobe cpumask_kunit and rmmod cpumask_kunit, kmemleak detect
a suspected memory leak as below.

If kunit_filter_suites() in kunit_module_init() succeeds, the
suite_set.start will not be NULL and the kunit_free_suite_set() in
kunit_module_exit() should free all the memory which has not
been freed. However the test_cases in suites is left out.

unreferenced object 0xffff54ac47e83200 (size 512):
comm "modprobe", pid 592, jiffies 4294913238 (age 1367.612s)
hex dump (first 32 bytes):
84 13 1a f0 d3 b6 ff ff 30 68 1a f0 d3 b6 ff ff ........0h......
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<000000008dec63a2>] slab_post_alloc_hook+0xb8/0x368
[<00000000ec280d8e>] __kmem_cache_alloc_node+0x174/0x290
[<00000000896c7740>] __kmalloc+0x60/0x2c0
[<000000007a50fa06>] kunit_filter_suites+0x254/0x5b8
[<0000000078cc98e2>] kunit_module_notify+0xf4/0x240
[<0000000033cea952>] notifier_call_chain+0x98/0x17c
[<00000000973d05cc>] notifier_call_chain_robust+0x4c/0xa4
[<000000005f95895f>] blocking_notifier_call_chain_robust+0x4c/0x74
[<0000000048e36fa7>] load_module+0x1a2c/0x1c40
[<0000000004eb8a91>] init_module_from_file+0x94/0xcc
[<0000000037dbba28>] idempotent_init_module+0x184/0x278
[<00000000161b75cb>] __arm64_sys_finit_module+0x68/0xa8
[<000000006dc1669b>] invoke_syscall+0x44/0x100
[<00000000fa87e304>] el0_svc_common.constprop.1+0x68/0xe0
[<000000009d8ad866>] do_el0_svc+0x1c/0x28
[<000000005b83c607>] el0_svc+0x3c/0xc4

Fixes: a127b154a8f2 ("kunit: tool: allow filtering test cases via glob")
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
Reviewed-by: Rae Moar <rmoar@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: Fix possible memory leak in kunit_filter_suites()

If both filter_glob and filters are not NULL, and kunit_parse_glob_filter()
succeed, but kcalloc parsed_filters fails, the suite_glob and test_glob of
parsed kzalloc in kunit_parse_glob_filter() will be leaked.

As Rae suggested, assign -ENOMEM to *err to correctly free copy and goto
free_parsed_glob to free the suite/test_glob of parsed.

Fixes: 1c9fd080dffe ("kunit: fix uninitialized variables bug in attributes filtering")
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
Suggested-by: Rae Moar <rmoar@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: Fix possible null-ptr-deref in kunit_parse_glob_filter()

Inject fault while probing kunit-example-test.ko, if kzalloc fails
in kunit_parse_glob_filter(), strcpy() or strncpy() to NULL will
cause below null-ptr-deref bug. So check NULL for kzalloc() and
return int instead of void for kunit_parse_glob_filter().

Unable to handle kernel paging request at virtual address dfff800000000000
KASAN: null-ptr-deref in range [0x0000000000000000-0x0000000000000007]
Mem abort info:
ESR = 0x0000000096000005
EC = 0x25: DABT (current EL), IL = 32 bits
SET = 0, FnV = 0
EA = 0, S1PTW = 0
FSC = 0x05: level 1 translation fault
Data abort info:
ISV = 0, ISS = 0x00000005, ISS2 = 0x00000000
CM = 0, WnR = 0, TnD = 0, TagAccess = 0
GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
[dfff800000000000] address between user and kernel address ranges
Internal error: Oops: 0000000096000005 [#1] PREEMPT SMP
Modules linked in: kunit_example_test cfg80211 rfkill 8021q garp mrp stp llc ipv6 [last unloaded: kunit_example_test]
CPU: 4 PID: 6047 Comm: modprobe Tainted: G W N 6.5.0-next-20230829+ #141
Hardware name: linux,dummy-virt (DT)
pstate: 80000005 (Nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : strncpy+0x58/0xc0
lr : kunit_filter_suites+0x15c/0xa84
sp : ffff800082a17420
x29: ffff800082a17420 x28: 0000000000000000 x27: 0000000000000004
x26: 0000000000000000 x25: ffffa847e40a5320 x24: 0000000000000001
x23: 0000000000000000 x22: 0000000000000001 x21: dfff800000000000
x20: 000000000000002a x19: 0000000000000000 x18: 00000000750b3b54
x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000
x14: 0000000000000000 x13: 34393178302f3039 x12: ffff7508fcea4ec1
x11: 1ffff508fcea4ec0 x10: ffff7508fcea4ec0 x9 : dfff800000000000
x8 : ffff6051b1a7f86a x7 : ffff800082a17270 x6 : 0000000000000002
x5 : 0000000000000098 x4 : ffff028d9817b250 x3 : 0000000000000000
x2 : 0000000000000000 x1 : ffffa847e40a5320 x0 : 0000000000000000
Call trace:
strncpy+0x58/0xc0
kunit_filter_suites+0x15c/0xa84
kunit_module_notify+0x1b0/0x3ac
blocking_notifier_call_chain+0xc4/0x128
do_init_module+0x250/0x594
load_module+0x37b0/0x44b4
init_module_from_file+0xd4/0x128
idempotent_init_module+0x2c8/0x524
__arm64_sys_finit_module+0xac/0x100
invoke_syscall+0x6c/0x258
el0_svc_common.constprop.0+0x160/0x22c
do_el0_svc+0x44/0x5c
el0_svc+0x38/0x78
el0t_64_sync_handler+0x13c/0x158
el0t_64_sync+0x190/0x194
Code: 5400028a d343fe63 12000a62 39400034 (38f56863)
---[ end trace 0000000000000000 ]---
Kernel panic - not syncing: Oops: Fatal exception
SMP: stopping secondary CPUs
Kernel Offset: 0x284761400000 from 0xffff800080000000
PHYS_OFFSET: 0xfffffd7380000000
CPU features: 0x88000203,3c020000,1000421b
Memory Limit: none
Rebooting in 1 seconds..

Fixes: a127b154a8f2 ("kunit: tool: allow filtering test cases via glob")
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
Reviewed-by: Rae Moar <rmoar@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: Fix the wrong err path and add goto labels in kunit_filter_suites()

Take the last kfree(parsed_filters) and add it to be the first. Take
the first kfree(copy) and add it to be the last. The Best practice is to
return these errors reversely.

And as David suggested, add several labels which target only the things
which actually have been allocated so far.

Fixes: 529534e8cba3 ("kunit: Add ability to filter attributes")
Fixes: abbf73816b6f ("kunit: fix possible memory leak in kunit_filter_suites()")
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
Reviewed-by: Rae Moar <rmoar@google.com>
Suggested-by: David Gow <davidgow@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: Allow kunit test modules to use test filtering

External tools, e.g., Intel GPU tools (IGT), support execution of
individual selftests provided by kernel modules. That could be also
applicable to kunit test modules if they provided test filtering. But
test filtering is now possible only when kunit code is built into the
kernel. Moreover, a filter can be specified only at boot time, then
reboot is required each time a different filter is needed.

Build the test filtering code also when kunit is configured as a module,
expose test filtering functions to other kunit source files, and use them
in kunit module notifier callback functions. Userspace can then reload
the kunit module with a value of the filter_glob parameter tuned to a
specific kunit test module every time it wants to limit the scope of tests
executed on that module load. Make the kunit.filter* parameters visible
in sysfs for user convenience.

v5: Refresh on tpp of attributes filtering fix
v4: Refresh on top of newly applied attributes patches and changes
introdced by new versions of other patches submitted in series with
this one.
v3: Fix CONFIG_GLOB, required by filtering functions, not selected when
building as a module (lkp@intel.com).
v2: Fix new name of a structure moved to kunit namespace not updated
across all uses (lkp@intel.com).

Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: Make 'list' action available to kunit test modules

Results from kunit tests reported via dmesg may be interleaved with other
kernel messages. When parsing dmesg for modular kunit results in real
time, external tools, e.g., Intel GPU tools (IGT), may want to insert
their own test name markers into dmesg at the start of each test, before
any kernel message related to that test appears there, so existing upper
level test result parsers have no doubt which test to blame for a specific
kernel message. Unfortunately, kunit reports names of tests only at their
completion (with the exeption of a not standarized "# Subtest: <name>"
header above a test plan of each test suite or parametrized test).

External tools could be able to insert their own "start of the test"
markers with test names included if they new those names in advance.
Test names could be learned from a list if provided by a kunit test
module.

There exists a feature of listing kunit tests without actually executing
them, but it is now limited to configurations with the kunit module built
in and covers only built-in tests, already available at boot time.
Moreover, switching from list to normal mode requires reboot. If that
feature was also available when kunit is built as a module, userspace
could load the module with action=list parameter, load some kunit test
modules they are interested in and learn about the list of tests provided
by those modules, then unload them, reload the kunit module in normal mode
and execute the tests with their lists already known.

Extend kunit module notifier initialization callback with a processing
path for only listing the tests provided by a module if the kunit action
parameter is set to "list" or "list_attr". For user convenience, make the
kunit.action parameter visible in sysfs.

v2: Don't use a different format, use kunit_exec_list_tests() (Rae),
- refresh on top of new attributes patches, handle newly introduced
kunit.action=list_attr case (Rae).

Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
Cc: Rae Moar <rmoar@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: Report the count of test suites in a module

According to KTAP specification[1], results should always start from a
header that provides a TAP protocol version, followed by a test plan with
a count of items to be executed. That pattern should be followed at each
nesting level. In the current implementation of the top-most, i.e., test
suite level, those rules apply only for test suites built into the kernel,
executed and reported on boot. Results submitted to dmesg from kunit test
modules loaded later are missing those top-level headers.

As a consequence, if a kunit test module provides more than one test suite
then, without the top level test plan, external tools that are parsing
dmesg for kunit test output are not able to tell how many test suites
should be expected and whether to continue parsing after complete output
from the first test suite is collected.

Submit the top-level headers also from the kunit test module notifier
initialization callback.

v3: Fix new name of a structure moved to kunit namespace not updated in
executor_test functions (lkp@intel.com).
v2: Use kunit_exec_run_tests() (Mauro, Rae), but prevent it from
emitting the headers when called on load of non-test modules.

[1] https://docs.kernel.org/dev-tools/ktap.html#

Signed-off-by: Janusz Krzysztofik <janusz.krzysztofik@linux.intel.com>
Cc: Mauro Carvalho Chehab <mchehab@kernel.org>
Cc: Rae Moar <rmoar@google.com>
Reviewed-by: Rae Moar <rmoar@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: fix uninitialized variables bug in attributes filtering

Fix smatch warnings regarding uninitialized variables in the filtering
patch of the new KUnit Attributes feature.

Fixes: 529534e8cba3 ("kunit: Add ability to filter attributes")

Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@linaro.org>
Closes: https://lore.kernel.org/r/202307270610.s0w4NKEn-lkp@intel.com/

Signed-off-by: Rae Moar <rmoar@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: fix possible memory leak in kunit_filter_suites()

Inject fault while probing drm_kunit_helpers.ko, if one of
kunit_next_attr_filter(), kunit_filter_glob_tests() and
kunit_filter_attr_tests() fails, parsed_filters,
parsed_glob.suite_glob/test_glob alloced in
kunit_parse_glob_filter() is leaked.
And the filtered_suite->test_cases alloced in kunit_filter_glob_tests()
or kunit_filter_attr_tests() may also be leaked.

unreferenced object 0xff110001067e4800 (size 1024):
comm "kunit_try_catch", pid 96, jiffies 4294671796 (age 763.547s)
hex dump (first 32 bytes):
73 75 69 74 65 32 00 00 00 00 00 00 00 00 00 00 suite2..........
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<00000000116e8eba>] __kmalloc_node_track_caller+0x4e/0x140
[<00000000e2f9cce9>] kmemdup+0x2c/0x60
[<000000002a36710b>] kunit_filter_suites+0x3e4/0xa50
[<0000000045779fb9>] filter_suites_test+0x1b7/0x440
[<00000000cd1104a7>] kunit_try_run_case+0x119/0x270
[<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0
[<00000000d195ac13>] kthread+0x2c7/0x3c0
[<00000000b79c1ee9>] ret_from_fork+0x2c/0x70
[<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30
unreferenced object 0xff11000105d79b00 (size 192):
comm "kunit_try_catch", pid 96, jiffies 4294671796 (age 763.547s)
hex dump (first 32 bytes):
f0 e1 5a 88 ff ff ff ff 60 59 bb 8a ff ff ff ff ..Z.....`Y......
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<000000000d6e4891>] __kmalloc+0x4d/0x140
[<000000006afe50bd>] kunit_filter_suites+0x424/0xa50
[<0000000045779fb9>] filter_suites_test+0x1b7/0x440
[<00000000cd1104a7>] kunit_try_run_case+0x119/0x270
[<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0
[<00000000d195ac13>] kthread+0x2c7/0x3c0
[<00000000b79c1ee9>] ret_from_fork+0x2c/0x70
[<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30
unreferenced object 0xff110001067e6000 (size 1024):
comm "kunit_try_catch", pid 98, jiffies 4294671798 (age 763.545s)
hex dump (first 32 bytes):
73 75 69 74 65 32 00 00 00 00 00 00 00 00 00 00 suite2..........
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<00000000116e8eba>] __kmalloc_node_track_caller+0x4e/0x140
[<00000000e2f9cce9>] kmemdup+0x2c/0x60
[<000000002a36710b>] kunit_filter_suites+0x3e4/0xa50
[<00000000f452f130>] filter_suites_test_glob_test+0x1b7/0x660
[<00000000cd1104a7>] kunit_try_run_case+0x119/0x270
[<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0
[<00000000d195ac13>] kthread+0x2c7/0x3c0
[<00000000b79c1ee9>] ret_from_fork+0x2c/0x70
[<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30
unreferenced object 0xff11000103f3a800 (size 96):
comm "kunit_try_catch", pid 98, jiffies 4294671798 (age 763.545s)
hex dump (first 32 bytes):
f0 e1 5a 88 ff ff ff ff 40 39 bb 8a ff ff ff ff ..Z.....@9......
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<000000000d6e4891>] __kmalloc+0x4d/0x140
[<000000006afe50bd>] kunit_filter_suites+0x424/0xa50
[<00000000f452f130>] filter_suites_test_glob_test+0x1b7/0x660
[<00000000cd1104a7>] kunit_try_run_case+0x119/0x270
[<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0
[<00000000d195ac13>] kthread+0x2c7/0x3c0
[<00000000b79c1ee9>] ret_from_fork+0x2c/0x70
[<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30
unreferenced object 0xff11000101a72ac0 (size 16):
comm "kunit_try_catch", pid 104, jiffies 4294671814 (age 763.529s)
hex dump (first 16 bytes):
00 00 00 00 00 00 00 00 e0 2a a7 01 01 00 11 ff .........*......
backtrace:
[<000000000d6e4891>] __kmalloc+0x4d/0x140
[<00000000c7b724e7>] kunit_filter_suites+0x108/0xa50
[<00000000bad5427d>] filter_attr_test+0x1e9/0x6a0
[<00000000cd1104a7>] kunit_try_run_case+0x119/0x270
[<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0
[<00000000d195ac13>] kthread+0x2c7/0x3c0
[<00000000b79c1ee9>] ret_from_fork+0x2c/0x70
[<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30
unreferenced object 0xff11000103caf880 (size 32):
comm "kunit_try_catch", pid 104, jiffies 4294671814 (age 763.547s)
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<000000000d6e4891>] __kmalloc+0x4d/0x140
[<00000000c47b0f75>] kunit_filter_suites+0x189/0xa50
[<00000000bad5427d>] filter_attr_test+0x1e9/0x6a0
[<00000000cd1104a7>] kunit_try_run_case+0x119/0x270
[<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0
[<00000000d195ac13>] kthread+0x2c7/0x3c0
[<00000000b79c1ee9>] ret_from_fork+0x2c/0x70
[<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30
unreferenced object 0xff11000101a72ae0 (size 16):
comm "kunit_try_catch", pid 106, jiffies 4294671823 (age 763.538s)
hex dump (first 16 bytes):
00 00 00 00 00 00 00 00 00 2b a7 01 01 00 11 ff .........+......
backtrace:
[<000000000d6e4891>] __kmalloc+0x4d/0x140
[<00000000c7b724e7>] kunit_filter_suites+0x108/0xa50
[<0000000096255c51>] filter_attr_empty_test+0x1b0/0x310
[<00000000cd1104a7>] kunit_try_run_case+0x119/0x270
[<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0
[<00000000d195ac13>] kthread+0x2c7/0x3c0
[<00000000b79c1ee9>] ret_from_fork+0x2c/0x70
[<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30
unreferenced object 0xff11000103caf9c0 (size 32):
comm "kunit_try_catch", pid 106, jiffies 4294671823 (age 763.538s)
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<000000000d6e4891>] __kmalloc+0x4d/0x140
[<00000000c47b0f75>] kunit_filter_suites+0x189/0xa50
[<0000000096255c51>] filter_attr_empty_test+0x1b0/0x310
[<00000000cd1104a7>] kunit_try_run_case+0x119/0x270
[<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0
[<00000000d195ac13>] kthread+0x2c7/0x3c0
[<00000000b79c1ee9>] ret_from_fork+0x2c/0x70
[<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30
unreferenced object 0xff11000101a72b00 (size 16):
comm "kunit_try_catch", pid 108, jiffies 4294671832 (age 763.529s)
hex dump (first 16 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<000000000d6e4891>] __kmalloc+0x4d/0x140
[<00000000c47b0f75>] kunit_filter_suites+0x189/0xa50
[<00000000881258cc>] filter_attr_skip_test+0x148/0x770
[<00000000cd1104a7>] kunit_try_run_case+0x119/0x270
[<00000000c654c917>] kunit_generic_run_threadfn_adapter+0x4e/0xa0
[<00000000d195ac13>] kthread+0x2c7/0x3c0
[<00000000b79c1ee9>] ret_from_fork+0x2c/0x70
[<000000001167f7e6>] ret_from_fork_asm+0x1b/0x30

Fixes: 5d31f71efcb6 ("kunit: add kunit.filter_glob cmdline option to filter suites")
Fixes: 529534e8cba3 ("kunit: Add ability to filter attributes")
Signed-off-by: Ruan Jinjie <ruanjinjie@huawei.com>
Reviewed-by: Rae Moar <rmoar@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: fix wild-memory-access bug in kunit_filter_suites()

As for kunit_filter_suites(), When the filters arg = NULL, such as
the call of kunit_filter_suites(&suite_set, "suite2", NULL, NULL, &err)
in filter_suites_test() tese case in kunit, both filter_count and
parsed_filters will not be initialized.

So it's possible to enter kunit_filter_attr_tests(), and the use of
uninitialized parsed_filters will cause below wild-memory-access.

RIP: 0010:kunit_filter_suites+0x780/0xa40
Code: fe ff ff e8 42 87 4d ff 41 83 c6 01 49 83 c5 10 49 89 dc 44 39 74 24 50 0f 8e 81 fe ff ff e8 27 87 4d ff 4c 89 e8 48 c1 e8 03 <66> 42 83 3c 38 00 0f 85 af 01 00 00 49 8b 75 00 49 8b 55 08 4c 89
RSP: 0000:ff1100010743fc38 EFLAGS: 00010203
RAX: 03fc4400041d0ff1 RBX: ff1100010389a900 RCX: ffffffff9f940ad9
RDX: ff11000107429740 RSI: 0000000000000000 RDI: ff110001037ec920
RBP: ff1100010743fd50 R08: 0000000000000000 R09: ffe21c0020e87f1e
R10: 0000000000000003 R11: 0000000000032001 R12: ff110001037ec800
R13: 1fe2200020e87f8c R14: 0000000000000000 R15: dffffc0000000000
FS: 0000000000000000(0000) GS:ff1100011b000000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: ff11000115201000 CR3: 0000000113066001 CR4: 0000000000771ef0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
<TASK>
? die_addr+0x3c/0xa0
? exc_general_protection+0x148/0x220
? asm_exc_general_protection+0x26/0x30
? kunit_filter_suites+0x779/0xa40
? kunit_filter_suites+0x780/0xa40
? kunit_filter_suites+0x779/0xa40
? __pfx_kunit_filter_suites+0x10/0x10
? __pfx_kfree+0x10/0x10
? kunit_add_action_or_reset+0x3d/0x50
filter_suites_test+0x1b7/0x440
? __pfx_filter_suites_test+0x10/0x10
? __pfx___schedule+0x10/0x10
? try_to_wake_up+0xa8e/0x1210
? _raw_spin_lock_irqsave+0x86/0xe0
? __pfx__raw_spin_lock_irqsave+0x10/0x10
? set_cpus_allowed_ptr+0x7c/0xb0
kunit_try_run_case+0x119/0x270
? __kthread_parkme+0xdc/0x160
? __pfx_kunit_try_run_case+0x10/0x10
kunit_generic_run_threadfn_adapter+0x4e/0xa0
? __pfx_kunit_generic_run_threadfn_adapter+0x10/0x10
kthread+0x2c7/0x3c0
? __pfx_kthread+0x10/0x10
ret_from_fork+0x2c/0x70
? __pfx_kthread+0x10/0x10
ret_from_fork_asm+0x1b/0x30
</TASK>
Modules linked in:
Dumping ftrace buffer:
(ftrace buffer empty)
---[ end trace 0000000000000000 ]---
RIP: 0010:kunit_filter_suites+0x780/0xa40
Code: fe ff ff e8 42 87 4d ff 41 83 c6 01 49 83 c5 10 49 89 dc 44 39 74 24 50 0f 8e 81 fe ff ff e8 27 87 4d ff 4c 89 e8 48 c1 e8 03 <66> 42 83 3c 38 00 0f 85 af 01 00 00 49 8b 75 00 49 8b 55 08 4c 89
RSP: 0000:ff1100010743fc38 EFLAGS: 00010203
RAX: 03fc4400041d0ff1 RBX: ff1100010389a900 RCX: ffffffff9f940ad9
RDX: ff11000107429740 RSI: 0000000000000000 RDI: ff110001037ec920
RBP: ff1100010743fd50 R08: 0000000000000000 R09: ffe21c0020e87f1e
R10: 0000000000000003 R11: 0000000000032001 R12: ff110001037ec800
R13: 1fe2200020e87f8c R14: 0000000000000000 R15: dffffc0000000000
FS: 0000000000000000(0000) GS:ff1100011b000000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: ff11000115201000 CR3: 0000000113066001 CR4: 0000000000771ef0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
PKRU: 55555554
Kernel panic - not syncing: Fatal exception
Dumping ftrace buffer:
(ftrace buffer empty)
Kernel Offset: 0x1da00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
Rebooting in 1 seconds..

Fixes: 529534e8cba3 ("kunit: Add ability to filter attributes")
Signed-off-by: Ruan Jinjie <ruanjinjie@huawei.com>
Reviewed-by: David Gow <davidgow@google.com>
Tested-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: Add ability to filter attributes

Add filtering of test attributes. Users can filter tests using the
module_param called "filter".

Filters are imputed in the format: <attribute_name><operation><value>

Example: kunit.filter="speed>slow"

Operations include: >, <, >=, <=, !=, and =. These operations will act the
same for attributes of the same type but may not between types.

Note multiple filters can be inputted by separating them with a comma.
Example: kunit.filter="speed=slow, module!=example"

Since both suites and test cases can have attributes, there may be
conflicts. The process of filtering follows these rules:
- Filtering always operates at a per-test level.
- If a test has an attribute set, then the test's value is filtered on.
- Otherwise, the value falls back to the suite's value.
- If neither are set, the attribute has a global "default" value, which
is used.

Filtered tests will not be run or show in output. The tests can instead be
skipped using the configurable option "kunit.filter_action=skip".

Note the default settings for running tests remains unfiltered.

Finally, add "filter" methods for the speed and module attributes to parse
and compare attribute values.

Note this filtering functionality will be added to kunit.py in the next
patch.

Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Rae Moar <rmoar@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: Add test attributes API structure

Add the basic structure of the test attribute API to KUnit, which can be
used to save and access test associated data.

Add attributes.c and attributes.h to hold associated structs and functions
for the API.

Create a struct that holds a variety of associated helper functions for
each test attribute. These helper functions will be used to get the
attribute value, convert the value to a string, and filter based on the
value. This struct is flexible by design to allow for attributes of
numerous types and contexts.

Add a method to print test attributes in the format of "# [<test_name if
not suite>.]<attribute_name>: <attribute_value>".

Example for a suite: "# speed: slow"

Example for a test case: "# test_case.speed: very_slow"

Use this method to report attributes in the KTAP output (KTAP spec:
https://docs.kernel.org/dev-tools/ktap.html) and _list_tests output when
kernel's new kunit.action=list_attr option is used. Note this is derivative
of the kunit.action=list option.

In test.h, add fields and associated helper functions to test cases and
suites to hold user-inputted test attributes.

Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Rae Moar <rmoar@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: improve KTAP compliance of KUnit test output

Change KUnit test output to better comply with KTAP v1 specifications
found here: https://kernel.org/doc/html/latest/dev-tools/ktap.html.
1) Use "KTAP version 1" instead of "TAP version 14" as test output header
2) Remove '-' between test number and test name on test result lines
2) Add KTAP version lines to each subtest header as well

Note that the new KUnit output still includes the “# Subtest” line now
located after the KTAP version line. This does not completely match the
KTAP v1 spec but since it is classified as a diagnostic line, it is not
expected to be disruptive or break any existing parsers. This
“# Subtest” line comes from the TAP 14 spec
(https://testanything.org/tap-version-14-specification.html) and it is
used to define the test name before the results.

Original output:

TAP version 14
1..1
# Subtest: kunit-test-suite
1..3
ok 1 - kunit_test_1
ok 2 - kunit_test_2
ok 3 - kunit_test_3
# kunit-test-suite: pass:3 fail:0 skip:0 total:3
# Totals: pass:3 fail:0 skip:0 total:3
ok 1 - kunit-test-suite

New output:

KTAP version 1
1..1
KTAP version 1
# Subtest: kunit-test-suite
1..3
ok 1 kunit_test_1
ok 2 kunit_test_2
ok 3 kunit_test_3
# kunit-test-suite: pass:3 fail:0 skip:0 total:3
# Totals: pass:3 fail:0 skip:0 total:3
ok 1 kunit-test-suite

Signed-off-by: Rae Moar <rmoar@google.com>
Reviewed-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Tested-by: Anders Roxell <anders.roxell@linaro.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: add kunit.enable to enable/disable KUnit test

This patch adds the kunit.enable module parameter that will need to be
set to true in addition to KUNIT being enabled for KUnit tests to run.
The default value is true giving backwards compatibility. However, for
the production+testing use case the new config option
KUNIT_DEFAULT_ENABLED can be set to N requiring the tester to opt-in
by passing kunit.enable=1 to the kernel.

Signed-off-by: Joe Fradley <joefradley@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: executor: Fix a memory leak on failure in kunit_filter_tests

It's possible that memory allocation for 'filtered' will fail, but for the
copy of the suite to succeed. In this case, the copy could be leaked.

Properly free 'copy' in the error case for the allocation of 'filtered'
failing.

Note that there may also have been a similar issue in
kunit_filter_subsuites, before it was removed in "kunit: flatten
kunit_suite*** to kunit_suite** in .kunit_test_suites".

This was reported by clang-analyzer via the kernel test robot, here:
https://lore.kernel.org/all/c8073b8e-7b9e-0830-4177-87c12f16349c@intel.com/

And by smatch via Dan Carpenter and the kernel test robot:
https://lore.kernel.org/all/202207101328.ASjx88yj-lkp@intel.com/

Fixes: a02353f49162 ("kunit: bail out of test filtering logic quicker if OOM")
Reported-by: kernel test robot <yujie.liu@intel.com>
Reported-by: kernel test robot <lkp@intel.com>
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Reviewed-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: flatten kunit_suite*** to kunit_suite** in .kunit_test_suites

We currently store kunit suites in the .kunit_test_suites ELF section as
a `struct kunit_suite***` (modulo some `const`s).
For every test file, we store a struct kunit_suite** NULL-terminated array.

This adds quite a bit of complexity to the test filtering code in the
executor.

Instead, let's just make the .kunit_test_suites section contain a single
giant array of struct kunit_suite pointers, which can then be directly
manipulated. This array is not NULL-terminated, and so none of the test
filtering code needs to NULL-terminate anything.

Tested-by: Maíra Canal <maira.canal@usp.br>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Co-developed-by: David Gow <davidgow@google.com>
Signed-off-by: David Gow <davidgow@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: use kmemdup in kunit_filter_tests(), take suite as const

kmemdup() is easier than kmalloc() + memcpy(), per lkp bot.

Also make the input `suite` as const since we're now always making
copies after commit a127b154a8f2 ("kunit: tool: allow filtering test
cases via glob").

Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: fix executor OOM error handling logic on non-UML

The existing logic happens to work fine on UML, but is not correct when
running on other arches.

1. We didn't initialize `int err`, and kunit_filter_suites() doesn't
explicitly set it to 0 on success. So we had false "failures".
Note: it doesn't happen on UML, causing this to get overlooked.
2. If we error out, we do not call kunit_handle_shutdown().
This makes kunit.py timeout when using a non-UML arch, since the QEMU
process doesn't ever exit.

Fixes: a02353f49162 ("kunit: bail out of test filtering logic quicker if OOM")
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: bail out of test filtering logic quicker if OOM

When filtering what tests to run (suites and/or cases) via
kunit.filter_glob (e.g. kunit.py run <glob>), we allocate copies of
suites.

These allocations can fail, and we largely don't handle that.
Note: realistically, this probably doesn't matter much.
We're not allocating much memory and this happens early in boot, so if
we can't do that, then there's likely far bigger problems.

This patch makes us immediately bail out from the top-level function
(kunit_filter_suites) with -ENOMEM if any of the underlying kmalloc()
calls return NULL.

Implementation note: we used to return NULL pointers from some functions
to indicate either that all suites/tests were filtered out or there was
an error allocating the new array.

We'll log a short error in this case and not run any tests or print a
TAP header. From a kunit.py user's perspective, they'll get a message
about missing/invalid TAP output and have to dig into the test.log to
see it. Since hitting this error seems so unlikely, it's probably fine
to not invent a way to plumb this error message more visibly.

See also: https://lore.kernel.org/linux-kselftest/20220329103919.2376818-1-lv.ruyi@zte.com.cn/

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reported-by: Zeal Robot <zealci@zte.com.cn>
Reported-by: Lv Ruyi <lv.ruyi@zte.com.cn>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: add 'kunit.action' param to allow listing out tests

Context:
It's difficult to map a given .kunitconfig => set of enabled tests.
Letting kunit.py figure that out would be useful.

This patch:
* is intended to be an implementation detail used only by kunit.py
* adds a kunit.action module param with one valid non-null value, "list"
* for the "list" action, it simply prints out "<suite>.<test>"
* leaves the kunit.py changes to make use of this for another patch.

Note: kunit.filter_glob is respected for this and all future actions.

Hack: we print a TAP header (but no test plan) to allow kunit.py to
use the same code to pick up KUnit output that it does for normal tests.
Since this is intended to be an implementation detail, it seems fine for
now. Maybe in the future we output each test as SKIPPED or the like.

Go with a more generic "action" param, since it seems like we might
eventually have more modes besides just running or listing tests, e.g.
* perhaps a benchmark mode that reruns test cases and reports timing
* perhaps a deflake mode that reruns test cases that failed
* perhaps a mode where we randomize test order to try and catch
hermeticity bugs like "test a only passes if run after test b"

Tested:
$ ./tools/testing/kunit/kunit.py run --kernel_arg=kunit.action=list --raw_output=kunit
...
TAP version 14
1..1
example.example_simple_test
example.example_skip_test
example.example_mark_skipped_test
reboot: System halted

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: fix too small allocation when using suite-only kunit.filter_glob

When a user filters by a suite and not a test, e.g.
$ ./tools/testing/kunit/kunit.py run 'suite_name'

it hits this code
const int len = strlen(filter_glob);
...
parsed->suite_glob = kmalloc(len, GFP_KERNEL);
which fails to allocate space for the terminating NULL.

Somehow, it seems like we can't easily reproduce this under UML, so the
existing `parse_filter_test()` didn't catch this.

Fix this by allocating `len + 1` and switch to kzalloc() just to be a
bit more defensive. We're only going to run this code once per kernel
boot, and it should never be very long.

Also update the unit tests to be a bit more cautious.
This bug showed up as a NULL pointer dereference here:
> KUNIT_EXPECT_STREQ(test, (const char *)filtered.start[0][0]->name, "suite0");
`filtered.start[0][0]` was NULL, and `name` is at offset 0 in the struct,
so `...->name` was also NULL.

Fixes: 3b29021ddd10 ("kunit: tool: allow filtering test cases via glob")
Reported-by: kernel test robot <oliver.sang@intel.com>
Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Acked-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: tool: allow filtering test cases via glob

Commit 1d71307a6f94 ("kunit: add unit test for filtering suites by
names") introduced the ability to filter which suites we run via glob.

This change extends it so we can also filter individual test cases
inside of suites as well.

This is quite useful when, e.g.
* trying to run just the tests cases you've just added or are working on
* trying to debug issues with test hermeticity

Examples:
$ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit '*exec*.parse*'
...
============================================================
======== [PASSED] kunit_executor_test ========
[PASSED] parse_filter_test
============================================================
Testing complete. 1 tests run. 0 failed. 0 crashed.

$ ./tools/testing/kunit/kunit.py run --kunitconfig=lib/kunit '*.no_matching_tests'
...
[ERROR] no tests run!

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: add unit test for filtering suites by names

This adds unit tests for kunit_filter_subsuite() and
kunit_filter_suites().

Note: what the executor means by "subsuite" is the array of suites
corresponding to each test file.

This patch lightly refactors executor.c to avoid the use of global
variables to make it testable.
It also includes a clever `kfree_at_end()` helper that makes this test
easier to write than it otherwise would have been.

Tested by running just the new tests using itself
$ ./tools/testing/kunit/kunit.py run '*exec*'

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: David Gow <davidgow@google.com>
Acked-by: Brendan Higgins <brendanhiggins@google.com>
Tested-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: Add 'kunit_shutdown' option

Add a new kernel command-line option, 'kunit_shutdown', which allows the
user to specify that the kernel poweroff, halt, or reboot after
completing all KUnit tests; this is very handy for running KUnit tests
on UML or a VM so that the UML/VM process exits cleanly immediately
after running all tests without needing a special initramfs.

Signed-off-by: David Gow <davidgow@google.com>
Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
Reviewed-by: Stephen Boyd <sboyd@kernel.org>
Tested-By: Daniel Latypov <dlatypov@google.com>
Reviewed-by: Daniel Latypov <dlatypov@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: add kunit.filter_glob cmdline option to filter suites

E.g. specifying this would run suites with "list" in their name.
kunit.filter_glob=list*

Note: the executor prints out a TAP header that includes the number of
suites we intend to run.
So unless we want to report empty results for filtered-out suites, we
need to do the filtering here in the executor.
It's also probably better in the executor since we most likely don't
want any filtering to apply to tests built as modules.

This code does add a CONFIG_GLOB=y dependency for CONFIG_KUNIT=y.
But the code seems light enough that it shouldn't be an issue.

For now, we only filter on suite names so we don't have to create copies
of the suites themselves, just the array (of arrays) holding them.

The name is rather generic since in the future, we could consider
extending it to a syntax like:
kunit.filter_glob=<suite_glob>.<test_glob>
E.g. to run all the del list tests
kunit.filter_glob=list-kunit-test.*del*

But at the moment, it's far easier to manually comment out test cases in
test files as opposed to messing with sets of Kconfig entries to select
specific suites.
So even just doing this makes using kunit far less annoying.

Signed-off-by: Daniel Latypov <dlatypov@google.com>
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: test: add test plan to KUnit TAP format

TAP 14 allows an optional test plan to be emitted before the start of
the start of testing[1]; this is valuable because it makes it possible
for a test harness to detect whether the number of tests run matches the
number of tests expected to be run, ensuring that no tests silently
failed.

Link[1]: https://github.com/isaacs/testanything.github.io/blob/tap14/tap-version-14-specification.md#the-plan
Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
Reviewed-by: Stephen Boyd <sboyd@kernel.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

init: main: add KUnit to kernel init

Although we have not seen any actual examples where KUnit doesn't work
because it runs in the late init phase of the kernel, it has been a
concern for some time that this could potentially be an issue in the
future. So, remove KUnit from init calls entirely, instead call directly
from kernel_init() so that KUnit runs after late init.

Co-developed-by: Alan Maguire <alan.maguire@oracle.com>
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
Reviewed-by: Stephen Boyd <sboyd@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Reviewed-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

kunit: test: create a single centralized executor for all tests

Add a centralized executor to dispatch tests rather than relying on
late_initcall to schedule each test suite separately. Centralized
execution is for built-in tests only; modules will execute tests when
loaded.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Co-developed-by: Iurii Zaikin <yzaikin@google.com>
Signed-off-by: Iurii Zaikin <yzaikin@google.com>
Co-developed-by: Brendan Higgins <brendanhiggins@google.com>
Signed-off-by: Brendan Higgins <brendanhiggins@google.com>
Reviewed-by: Stephen Boyd <sboyd@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>