Re: [PATCH] selftest: add a script to perform selftest compile tests

[shuah]

On 4/13/20 4:15 PM, Tim Bird wrote:
> From: Tim Bird <tim.bird@xxxxxxxx>
>
> Add ksft-compile-test.sh.  This is a program used to test
> cross-compilation and installation of selftest tests.
>
> See the test usage for help
>
> This program currently tests 3 scenarios out of a larger matrix
> of possibly interesting scenarios.  For each scenario, it conducts
> multiple tests for correctness.  This version tests:
>   1) does the test compile

Is it necessary to write this long a script? Could we just parse
the "kselftest
>   2) is the kernel source directory clean after the compile

Can you use make mrproper and see if anything needs cleaning?

>   3) does the test install operation succeed
>   4) does the test run script reference the test

I like the idea of being able to test, however I am not convinced
you would need a whole new script for it.


> Signed-off-by: Tim Bird <tim.bird@xxxxxxxx>
> ---
>   MAINTAINERS                                  |   6 +
>   tools/testing/selftests/ksft-compile-test.sh | 567 +++++++++++++++++++++++++++
>   2 files changed, 573 insertions(+)
>   create mode 100755 tools/testing/selftests/ksft-compile-test.sh
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index cc1d18c..a6289c7 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -9127,6 +9127,12 @@ S:	Maintained
>   F:	tools/testing/selftests/
>   F:	Documentation/dev-tools/kselftest*
>   
> +KERNEL SELFTEST SELFTEST
> +M:	Tim Bird <tim.bird@xxxxxxxx>
> +L:	linux-kselftest@xxxxxxxxxxxxxxx
> +S:	Maintained
> +F:	tools/testing/selftests/ksft-compile-test.sh
> +

Please don't add another entry to the MAINTAINERS file just
for a shell script under tools/testing/selftests

This doesn't make sense.

>   KERNEL UNIT TESTING FRAMEWORK (KUnit)
>   M:	Brendan Higgins <brendanhiggins@xxxxxxxxxx>
>   L:	linux-kselftest@xxxxxxxxxxxxxxx
> diff --git a/tools/testing/selftests/ksft-compile-test.sh b/tools/testing/selftests/ksft-compile-test.sh
> new file mode 100755
> index 0000000..e36e858
> --- /dev/null
> +++ b/tools/testing/selftests/ksft-compile-test.sh
> @@ -0,0 +1,567 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0-only OR MIT
> +#
> +# ksft-compile-test.sh - test compiling Linux kernel selftests under lots of
> +#   different configurations.  This is used to check that cross-compilation
> +#   and install works properly for a newly submitted test target, and
> +#   also that changes to existing test Makefiles don't regress with regard to
> +#   this functionality.
> +#
> +# Copyright 2020 Sony Corporation
> +#
> +# Here are the things that Shuah Kahn asked for on 3/6/2020
> +# 1. Cross-compilation & relocatable build support
> +# 2. Generates objects in objdir/kselftest without cluttering main objdir
> +# 3. Leave source directory clean
> +# 4. Installs correctly in objdir/kselftest/kselftest_install and adds
> +#        itself to run_kselftest.sh script generated during install.
> +#

I was asking for fixes to individual tests.

> +# Would be nice to make sure other features also work:
> +# 5. can use absolute, relative, or current directory for output directory
> +# 6. can use ~ in output directory path
> +#

I do think this can be achieved with a simpler script wrapper around
existing commands and kselftest_install.sh instead of writing a whole
new shell script.

thanks,
-- Shuah