Re: [PATCH 00/42] nolibc: update to resync with out-of-tree project

[Paul E. McKenney]

On Tue, Feb 08, 2022 at 05:41:50AM +0100, Willy Tarreau wrote:
> On Mon, Feb 07, 2022 at 04:00:28PM -0800, Paul E. McKenney wrote:
> > I queued these from email to get some time on them.  If an update is
> > needed (for example, based on discussion with David Laight), please feel
> > free to send me an update and I can drop the current set in favor of a
> > new version.
> 
> OK thanks!
> 
> > Good stuff, by the way, thank you!!!
> 
> thanks ;-)
> 
> > Hmmm...  I should make rcutorture able to smoke-test this.  My thought
> > would be to add some code to rcutorture's init.c that tried out the
> > system calls.
> 
> I don't think we should abuse rcutorture to test nolibc, however you
> could decide that rcutorture could benefit from some improvements
> (take arguments, parse env or /proc/cmdline, report some info such
> as syscall errors etc).
> 
> > But I bet that you already have some test code.  ;-)
> 
> Actually my testing remains limited in terms of reproducibility. For
> most syscalls I used to just modify a simple test file (hello.c) to
> update the syscall and generally run it under strace; in addition,
> my preinit code builds with it and all my kernels rely on it, which
> gives me extra confidence I didn't break common stuff.
> 
> I've thought about starting to create a real test suite for all these
> calls, either just one syscall or function per file (for easier testing),
> or one test file per test unit (e.g. one for stdlib, one for sys, etc),
> or maybe something in between.

I am mainly looking for a regression test I can run, with or without
rcutorture.  ;-)

> > One approach would be to place the test code in tools/nolibc, and
> > have rcutorture's init.c either #include that or link to it.
> 
> There could be an interesting idea to develop here. While I don't find
> it logical to abuse rcutorture for this, we could instead consider a
> more general init-based test suite that could cover various aspects
> including syscalls and rcutorture. We could let the user choose at
> build time what to implement (so that we don't waste testing time on
> the libc aspect when rcutorture is needed for example), and/or force
> tests on the cmdline (convenient under qemu for example). It could
> also be used to verify if some syscalls were dropped due to some
> config options being turned off.
> 
> I was thinking about having just a numbered suite that iterates over
> numerous tests and provides a test number, a function/syscall name,
> a variant and a result. Something that could easily be copy-pasted
> to ease reporting of breakage.
> 
> This could be especially helpful to port to other architectures. For
> example we're still missing PPC and I don't have much experience there
> (the only one I have access to runs and old version of AIX). And a
> test suite could easily tell if something is broken such as a
> non-implemented syscall.

All excellent points!

> I was hesitant about where to place such a test suite, because I didn't
> want to pollute the include files with tests. In the out-of-tree code
> there's already a "hello.c" file and I started to think about moving the
> rest to include/. But your proposal of tools/nolibc makes sense in that
> it would keep the suite separate from the include files. Actually I'd
> rather put that under tools/testing/selftests/nolibc. With this done,
> I think it would remove some maintenance burden and I could probably
> just kill the out-of-tree project and consider that the up-to-date one
> is in the kernel.

Agreed, tools/testing/selftests/nolibc makes a lot of sense.  As does
having things in-tree, especially given that it seems to have several
contributors now, which presumably means at least that many users.

> > Thoughts?  Especially thoughts about better testing approaches?
> 
> Hehe, each time I think I'm done for a while on this project, you
> manage to fuel me with many interesting ideas for improvements ;-)

"It is a service I provide."  ;-)

							Thanx, Paul