Re: [Announce] LPC 2018: Testing and Fuzzing Microconference

[Dmitry Vyukov]

On Sat, Sep 22, 2018 at 2:52 PM, Matthew Wilcox <willy@xxxxxxxxxxxxx> wrote:
> On Wed, Sep 19, 2018 at 10:13:15AM -0700, Dhaval Giani wrote:
>> Sasha and I are pleased to announce the Testing and Fuzzing track at
>> LPC [ 1 ]. We are planning to continue the discussions from last
>> year's microconference [2]. Many discussions from the Automated
>> Testing Summit [3] will also continue, and a final agenda will come up
>> only soon after that.
>>
>> Suggested Topics
>>
>> - Syzbot/syzkaller
>> - ATS
>> - Distro/stable testing
>> - kernelci
>> - kernelci auto bisection
>> - Unit testing framework
>>
>> We look forward to other interesting topics for this microconference
>> as a reply to this email.
>
> I would like to talk about the IDA test suite that was recently merged.
> See lib/test_ida.c.  It can be built as a module (CONFIG_TEST_IDA=m),
> built-in (=y) or built in userspace (as part of the radix tree test
> suite for historical reasons) along with the current kernel code.
>
> Being able to build the test suite in userspace allows for much more
> rapid development.  Building it in kernel space offers testing across
> a wide range of configurations that I don't have access to and can't
> necessarily simulate well in userspace.
>
> My userspace implementation of kmalloc() simulates failures (in a rather
> heavy-handed way; every non-GFP_KERNEL allocation fails).  That's rather
> harder to simulate in kernel space.  I'd like there to be a way for a
> kernel space test suite to ask for kmalloc failures so that the failure
> paths can be tested.

Hi Matthew,

kmalloc fault injection is already implemented with
CONFIG_FAULT_INJECTION. For original fault injection, you more-or-less
ask to fail X% of allocations at random. It's great for testing
servers for stability, but not so well suited for testing. Recently
I've added so-called "systematic" fault injection
(/proc/thread-self/fail-nth) which is perfect for unit testing. It
allows to ask to fail N-th allocation request in the current task. So
a unit test for a syscall can do:

for (i = 0;; i++) {
  write(/proc/thread-self/fail-nth, i);
  syscall_under_test();
  if (read(/proc/thread-self/fail-nth) != 0)
    break;
}

which allows to examine each failure site one-by-one systematically
(without making random processes on the machine fail too).


> I think the idea of building parts of the core kernel libraries in
> userspace and testing them there has greater generality than just the
> IDA, IDR, XArray & radix tree and might profitably be adopted by other
> parts of lib/.  The userspace simulation of other parts of the kernel
> may well need to be extended.

This would be great. Besides providing faster turn-around time, this
allow us to finally have something like:

make test [subsystem]

which will run all tests for the subsystem locally (in parallel,
giving final OK/FAIL). This is not possible to in-kernel tests,
because they require a machine and an image, and it's not possible to
support everything that people use.