Re: [syzbot] [bcachefs?] general protection fault in proc_sys_compare

[Theodore Ts'o]

On Fri, Mar 07, 2025 at 09:33:11AM -0500, Kent Overstreet wrote:
>
> > Maybe this is something Syzbot could implement?
> 
> Wouldn't it be better to have it in 'git bisect'?

"Git bisect" is the wrong layer of abstraction.  It doesn't know
anything about (a) how build the software package (which might not be
the kernel, remember), nor how to run a test, nor how to tell whether
a test run was successful or a failure.

It's expected that those tools need to be built on top of "git
bisect."  For example Steve Rostedt has contributed ktest.pl, which is
in the kernel sources in tools/teseting/ktest/.  This assumes that the
system under test is a bare metal machine where is accessible via
ssh/scp, and that builds are done on the local machine where ktest.pl
is run.

ktest.pl is not something I've used myself, since I do pretty much all
of my testing using VM's, and in the case of gce-xfstests, we spin up
a fast compile VM which does the kernel compilation and uploads the
freshly compiled kernel to Google Cloud Storage (GCS), and then kicks
off one or more test VM's which fetches the kernel from GCS, with the
command of which tests to run encoded in the test VM metadata.  I also
have a monitoring VM that detects if a test VM hangs, so it can
automatically restart the test VM.  All of this is logic which is't
supported by ktest.pl, and obviously *far* beyond the scope of "git
bisect".

And while Syzbot can use Google Cloud in addition to qemu, its
infrastruture for how it compiles kernels and runs its test VM's is
sufficiently different that it's not clear that software written for
one infrastructure would be easily applicable to another.

> If only we had interns and grad students for this sort of thing :)

The lightweight test manager (ltm) for gce-xfstests was implemented by
an intern.  And the kernel compilation service (kcs), git branch
watcher, and git bisection engine for gce-xfstests was done by a group
of undergraduates at a unversity in Boston as part of a software
engineering class project.

Mentoring interns and undergraduates was incredibly fulfilling, and I
very much enjoyed the experience.  I'd like to think I helped them to
become better software engineers.  However, mentoring students takes a
significant amount of time, and on net, it's not clear it was a win
from a personal time ROI perspective.

We did manage to recruit the intern to become a SWE at Google after he
graduated, so that was definitely considered a win from my company's
perspective.  :-)

						- Ted