Re: RFC - kernel selftest result documentation (KTAP)

[Kees Cook]

On Tue, Jun 16, 2020 at 09:16:01PM +0000, Bird, Tim wrote:
> So far, most of the CI systems don't parse out diagnostic data, so it doesn't
> really matter what the format is.  If it's useful for humans, it's valuable as is.
> However, it would be nice if that could change.  But without some formalization
> of the format of the diagnostic data, it's an intractable problem for CI systems
> to parse it.  So it's really a chicken and egg problem.  To solve it, we would have
> to determine what exactly needs to be provided on a consistent basis for diagnostic
> data across many tests.  I think that it's too big a problem to handle right now.
> I'm not opposed to migrating to some structure with yaml in the future, but free
> form text output seems OK for now.

For a CI system, if I see a test has failed, I expect to be able to
click a link to get the log of that test, which includes the diagnostic
lines. The other reason to have them there is to show progress during a
manual run.

> > Yeah, I think it would be nice if all test frameworks/libraries for the
> > kernel output tests in the same language.
> Agreed.

$ git grep "TAP version"
exec/binfmt_script:print("TAP version 1.3")
kselftest.h:            printf("TAP version 13\n");
kselftest/runner.sh:    echo "TAP version 13"
resctrl/resctrl_tests.c:        printf("TAP version 13\n");
size/get_size.c:        print("TAP version 13\n");

Looks like there are 2 tests to convert to kselftest.h, and then we can
just change the version to 14 in the header and the runner. ;)

-- 
Kees Cook