Re: [PATCH] selftests/nolibc: add 7 tests for memcmp()

[Willy Tarreau]

On Fri, Oct 21, 2022 at 08:56:45AM -0700, Paul E. McKenney wrote:
> On Fri, Oct 21, 2022 at 08:03:40AM +0200, Willy Tarreau wrote:
> > This adds 7 combinations of input values for memcmp() using signed and
> > unsigned bytes, which will trigger on the original code before Rasmus'
> > fix. This is mostly aimed at helping backporters verify their work, and
> > showing how tests for corner cases can be added to the selftests suite.
> > 
> > Before the fix it reports:
> >   12 memcmp_20_20 = 0                      [OK]
> >   13 memcmp_20_60 = -64                    [OK]
> >   14 memcmp_60_20 = 64                     [OK]
> >   15 memcmp_20_e0 = 64                    [FAIL]
> >   16 memcmp_e0_20 = -64                   [FAIL]
> >   17 memcmp_80_e0 = -96                    [OK]
> >   18 memcmp_e0_80 = 96                     [OK]
> > 
> > And after:
> >   12 memcmp_20_20 = 0                      [OK]
> >   13 memcmp_20_60 = -64                    [OK]
> >   14 memcmp_60_20 = 64                     [OK]
> >   15 memcmp_20_e0 = -192                   [OK]
> >   16 memcmp_e0_20 = 192                    [OK]
> >   17 memcmp_80_e0 = -96                    [OK]
> >   18 memcmp_e0_80 = 96                     [OK]
> > 
> > Cc: Rasmus Villemoes <linux@xxxxxxxxxxxxxxxxxx>
> > Signed-off-by: Willy Tarreau <w@xxxxxx>
> 
> I have pulled both of these in, thank you!
 
Thanks!

> One thing, though...  I had to do "make clean" in both tools/include/nolibc
> and tools/testing/selftests/nolibc to make those two "[FAIL]" indications
> go away.  Does this mean that I am doing something wrong?

No you didn't do anything wrong, it was the same for me and initially it
was intentional, but probably it wasn't that good an idea. What happens
is that we first prepare a pseudo-sysroot with kernel headers and nolibc
headers, then we build the test based on this sysroot. Thus if any uapi
header or nolibc header changes, nothing is detected. And I'm not much
willing to always reinstall everything for every single test, nor to
detect long dependency chains. Maybe I should think about adding another
target to clean+test at the same time, or maybe make the current
"nolibc-test" target do that and have a "retest" to only rebuild. But
that needs to be thought about with the QEMU test as well (because most
of the time for a quick test I don't build the kernel nor start QEMU, I
just call the executable directly).

Any ideas or suggestions are welcome, of course. We could consider that
if we build a kernel and start QEMU, it's long enough to justify a
systematic clean maybe ?

> It would be good to know before I send the pull request containing these,
> so that we can let Linus know of anything special he needs to do to
> ensure a valid test result.

I see. In the worst case, a preliminary "make clean" will do it. We just
need to decide what's the best solution for everyone (i.e. not waste too
much time between tests while not getting misleading results by accident).

Thanks!
Willy