Re: LKML admins (syzbot emails are not delivered)

[Fengguang Wu]

On Tue, Jan 16, 2018 at 08:59:36AM +0100, Dmitry Vyukov wrote:
> On Tue, Jan 16, 2018 at 8:12 AM, Theodore Ts'o <tytso@xxxxxxx> wrote:
> > On Mon, Jan 15, 2018 at 10:38:42AM -0600, Eric W. Biederman wrote:
> > > Sometimes the branches on linux-next are experimental crap.  If someone
> > > adds an experimental memory allocator to linux-next before discovering
> > > it causes all kinds of problems I don't want bug reports about my code
> > > not being able to allocate memory because the memory allocator was bad.
> > >
> > > If you don't have the resources to test the individual branches of
> > > linux-next please just test Linus's tree.   That will be much more
> > > meaningful and productive.
> >
> > I have to agree with Eric here, the reason why Fengguang Wu's 0-day
> > testing robot is much better received by developers is that he does
> > not test linux-net, but rather individual subsystem git trees and
> > branches.  His test automation also does an automatic bisection
> > search, and can point at a specific commit --- at which point e-mail
> > goes out to owner of the subsystem git tree, and to the people who
> > authored and/or reviewed the guilty commit.
> >
> > Dmitry, perhaps you could collaborate with Intel's 0-day testing
> > folks?  They have code which does all of this, and perhaps it can be
> > leveraged.
>
> +Fengguang
>
> Please note that in most cases 0-day solves an order of magnitude
> simpler problem. Build/sparse errors are much faster to find, always
> possible to precisely bisect and attribute. Yes, for that you just
> test every commit, bisect and send targeted emails. syzbot only finds
> runtime bugs, lots of them are related to races and can't be reliably
> reproduced, bisected, etc. Lots of them are old (e.g. predate KASAN
> that detects them). But they still can be fixed. In ~half of cases
> developers fix them looking only at the oops report.
>
> The last time I checked 0-day infrastructure was closed source.
>
> Fengguang, what do you do with trinity crashes that happen
> episodically, but you can't reliably reproduce, bisect and attribute?

0-day runs most trinity tests in QEMU machines, which can run
massively in parallel. Which means we can afford to bisect them
by running up to 1000 boots in each bisect step. Ditto for KASAN
errors.

Since Xiaolong (CCed) has enabled syzkaller in 0-day, it could in
theory utilize the same auto bisect infrastructure. If we can make
sure syzkaller runs effectively in the 0-day test farm.

Thanks,
Fengguang