Re: [PATCH] selftests/nolibc: add 7 tests for memcmp()

[Paul E. McKenney]

On Fri, Oct 21, 2022 at 07:01:34PM +0200, Willy Tarreau wrote:
> On Fri, Oct 21, 2022 at 08:56:45AM -0700, Paul E. McKenney wrote:
> > On Fri, Oct 21, 2022 at 08:03:40AM +0200, Willy Tarreau wrote:
> > > This adds 7 combinations of input values for memcmp() using signed and
> > > unsigned bytes, which will trigger on the original code before Rasmus'
> > > fix. This is mostly aimed at helping backporters verify their work, and
> > > showing how tests for corner cases can be added to the selftests suite.
> > > 
> > > Before the fix it reports:
> > >   12 memcmp_20_20 = 0                      [OK]
> > >   13 memcmp_20_60 = -64                    [OK]
> > >   14 memcmp_60_20 = 64                     [OK]
> > >   15 memcmp_20_e0 = 64                    [FAIL]
> > >   16 memcmp_e0_20 = -64                   [FAIL]
> > >   17 memcmp_80_e0 = -96                    [OK]
> > >   18 memcmp_e0_80 = 96                     [OK]
> > > 
> > > And after:
> > >   12 memcmp_20_20 = 0                      [OK]
> > >   13 memcmp_20_60 = -64                    [OK]
> > >   14 memcmp_60_20 = 64                     [OK]
> > >   15 memcmp_20_e0 = -192                   [OK]
> > >   16 memcmp_e0_20 = 192                    [OK]
> > >   17 memcmp_80_e0 = -96                    [OK]
> > >   18 memcmp_e0_80 = 96                     [OK]
> > > 
> > > Cc: Rasmus Villemoes <linux@xxxxxxxxxxxxxxxxxx>
> > > Signed-off-by: Willy Tarreau <w@xxxxxx>
> > 
> > I have pulled both of these in, thank you!
>  
> Thanks!
> 
> > One thing, though...  I had to do "make clean" in both tools/include/nolibc
> > and tools/testing/selftests/nolibc to make those two "[FAIL]" indications
> > go away.  Does this mean that I am doing something wrong?
> 
> No you didn't do anything wrong, it was the same for me and initially it
> was intentional, but probably it wasn't that good an idea. What happens
> is that we first prepare a pseudo-sysroot with kernel headers and nolibc
> headers, then we build the test based on this sysroot. Thus if any uapi
> header or nolibc header changes, nothing is detected. And I'm not much
> willing to always reinstall everything for every single test, nor to
> detect long dependency chains. Maybe I should think about adding another
> target to clean+test at the same time, or maybe make the current
> "nolibc-test" target do that and have a "retest" to only rebuild. But
> that needs to be thought about with the QEMU test as well (because most
> of the time for a quick test I don't build the kernel nor start QEMU, I
> just call the executable directly).
> 
> Any ideas or suggestions are welcome, of course. We could consider that
> if we build a kernel and start QEMU, it's long enough to justify a
> systematic clean maybe ?
> 
> > It would be good to know before I send the pull request containing these,
> > so that we can let Linus know of anything special he needs to do to
> > ensure a valid test result.
> 
> I see. In the worst case, a preliminary "make clean" will do it. We just
> need to decide what's the best solution for everyone (i.e. not waste too
> much time between tests while not getting misleading results by accident).

Maybe just document the careful/slow way, then people doing it more
frequently can do it the clever/fast way.

My guess is that the careful/slow is this:

	pushd tools/include/nolibc
	make clean
	make
	popd
	pushd tools/testing/selftests/nolibc
	make clean
	make -j32 run

Or did I miss a turn in there somewhere?

							Thanx, Paul