Re: AUTOSEL process

[Willy Tarreau]

On Sat, Mar 11, 2023 at 09:21:58PM -0800, Eric Biggers wrote:
> I mean, "patches welcome" is a bit pointless when there is nothing to patch, is
> it not?

Maybe it's because they're used to regularly receive complaints suggesting
to improve the process without knowing where to start from.

> Even Sasha's stable-tools, which he finally gave a link to, does not
> include anything related to AUTOSEL.  It seems AUTOSEL is still closed source.

I don't know.

> BTW, I already did something similar "off to the side" a few years ago when I
> wrote a script to keep track of and prioritize syzbot reports from
> https://syzkaller.appspot.com/, and generate per-subsystem reminder emails.
> 
> I eventually ended up abandoning that, because doing something off to the side
> is not very effective and is hard to keep up with.  The right approach is to
> make improvements to the "upstream" process (which was syzbot in that case), not
> to bolt something on to the side to try to fix it after the fact.

I think that improving the upstreaming process does have some value,
of course, especially when it's done from tools that can reliably and
durably be improved. But it's not rocket science when input comes from
humans. We still occasionally see patches missing an s-o-b. Humans can't
be fixed, and the more the constraints that are added on them, the higher
the failure rate they will show. However, regardless of the detailed
knowledge of how to format this or that tag, it should still be possible
for any patch author to answer the question "what do you want us to do
with that patch". If most of the patches at least contain such info, it
already becomes possible to figure after it gets merged whether the intent
was to get it backported or not. It's not trivial but NLP and code analysis
definitely help on this. And there will always be some patches whose need
for backporting will be detected after the merge. That's why I'm seeing a
lot of value in the post-processing part, because for me trying to fix
the input will have a very limited effect.

And let's face it, if a patch series gets merged, it means that at some
point someone understood what to do with it, so all the needed information
was already there, given a certain context. The cost is thousands of
brains needed to decode that. But with the improvements in language
processing, we should at some point be able to release some brains with
more-or-less similar results and try to lower the barrier to contribution
by relaxing patch submission rules instead of adding more. My feeling is
that it's what we should aim for given that the number of maintainers and
contributors doesn't seem to grow as fast as the code size and complexity.

That's where I'm seeing a lot of value in the AUTOSEL work. If it can show
the direction to something better so that in 3 or 4 years we can think
back and say "remember the garbage it was compared to what we have now",
I think it will have been a fantastic success.

> So I hope people can understand where I'm coming from, with hoping that what the
> stable maintainers are doing can just be improved directly, without first
> building something from scratch off to the side as that is just not a good way
> to do things.

It's generally not good but here there's little to start from, and
experimenting with your ideas on LKML threads or commit series can be
a nice way to explore completely different approaches without being
limited by what currently exists. Maybe you'll end up with something
completely orthogonal and the combination of both of your solutions
will already be a nice improvement. who knows.

> But sure, if that's the only option to get anything nontrivial
> changed, I'll try to do it.

Thanks!
Willy