Re: [PATCH] selftests: Add a taint selftest

[Andrew Delgadillo]

On Tue, Aug 23, 2022 at 10:06 PM Greg KH <gregkh@xxxxxxxxxxxxxxxxxxx> wrote:
>
> On Tue, Aug 23, 2022 at 09:19:58PM +0000, Andrew Delgadilo wrote:
> > From: Andrew Delgadillo <adelg@xxxxxxxxxx>
> >
> > When testing a kernel, one of the earliest signals one can get is if a
> > kernel has become tainted. For example, an organization might be
> > interested in mass testing commits on their hardware. An obvious first
> > step would be to make sure every commit boots, and a next step would be
> > to make sure there are no warnings/crashes/lockups, hence the utility of
> > a taint test.
>
> What's wrong with the tools/debugging/kernel-chktaint script?
>
> Why do we need another "get what the taint status is" program?

The main functionality that this selftests has that kernel-chktaint
does not is that it exits with a non-zero status code if the kernel is
tainted. kernel-chktaint outputs information to stdout based on the
taint status, but will always exit 0.

The issue with this is that it cannot be plugged into a test runner
that checks the exit code of a test script. In other words, if I
wanted to plug it into git bisect, I would have to wrap
kernel-chktaint in a command that transformed the output to an exit
code. Sure that is doable, but it is not as simple as it could be.

More concretely, I am setting kselftest runs against kernel commits
(with a harness that logs kselftest runs into some other
infrastructure), and such a test that is missing is a kselftest that
checks the kernel's taint status. One could argue that one should just
create a kselftest target that calls into kernel-chktaint and parse
the output there to determine what the exit status is, but that seems
fragile as a change in the underlying script could break it. For
example, if I want to test for taint #18, and I am grepping for the
string " * an in-kernel test has been run (#18)", I will actually get
a false positive because the underlying script does not check for
taint #18. Contrived example yes, but I think it shows that textual
grepping for errors is error prone (as an aside, I'll send a patch to
update the script to check for the new taint bit).

> thanks,
>
> greg k-h