Re: [PATCH] selftests/lib.mk: run_tests= print summary at the end

[Hangbin Liu]

On Thu, Jun 25, 2026 at 06:22:19PM +0200, Matthieu Baerts (NGI0) wrote:
> When there are many tests and/or many log lines, it is difficult to
> quickly find out how the different tests went when running:
> 
>     make -C tools/testing/selftests run_tests
> 
> This run_tests target executes 'run_many' from runner.sh, which relies
> on ktap_helpers.sh. At the end of the execution, the KTAP_CNT_* counters
> have been populated, and the total can then be simply printed with the
> ktap_print_totals helper.
> 
> Now a summary is printed at the end, e.g. here with the two last lines:
> 
>     $ make -sC tools/testing/selftests TARGETS=(...) run_tests
>     # timeout set to (...)
>     # selftests: (...)
>     (...)
>     ok 1 selftests: (...)
>     # timeout set to (...)
>     # selftests: (...)
>     (...)
>     ok 2 selftests: (...) # SKIP
>     # timeout set to (...)
>     # selftests: (...)
>     (...)
>     not ok 3 selftests: (...)
>     # 1 skipped test(s) detected.  Consider enabling relevant config options to improve coverage.
>     # Totals: pass:1 fail:1 xfail:0 xpass:0 skip:1 error:0
> 
> This summary is printed, using '#' as prefix to be treated as a comment
> in the TAP format. The summary is also printed when summary=1 is used to
> redirect log lines in different files because it still looks useful when
> there are many tests, not to have to check each line.
> 
> Note that this 'make' command will continue to exit without errors
> (returned code is 0) when the tests can be executed. I guess it is
> better not to change this behaviour to be able to continue detecting
> kselftests failures.
> 
> Signed-off-by: Matthieu Baerts (NGI0) <matttbe@xxxxxxxxxx>
> ---
>  tools/testing/selftests/lib.mk | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
> index f02cc8a2e4ae..de15e63bc6b4 100644
> --- a/tools/testing/selftests/lib.mk
> +++ b/tools/testing/selftests/lib.mk
> @@ -106,7 +106,8 @@ define RUN_TESTS
>  	if [ "X$(summary)" != "X" ]; then       \
>  		per_test_logging=1;		\
>  	fi;                                     \
> -	run_many $(1)
> +	run_many $(1);				\
> +	ktap_print_totals
>  endef
>  
>  define INSTALL_INCLUDES

This helps get a clear summary for make run_tests.

Reviewed-by: Hangbin Liu <liuhangbin@xxxxxxxxxx>