Re: AUTOSEL process

[Willy Tarreau]

On Sat, Mar 11, 2023 at 11:46:05AM -0800, Eric Biggers wrote:
> (And please note, the key word here is *confidence*.  We all agree that it's
> never possible to be absolutely 100% sure whether a commit is appropriate for
> stable or not.  That's a red herring.

In fact even developers themselves sometimes don't know, and even when they
know, sometimes they know after committing it. Many times we've found that
a bug was accidently resolved by a small change. Just for this it's important
to support a post-merge analysis.

> And I would assume, or at least hope, that the neural network thing being used
> for AUTOSEL outputs a confidence rating and not just a yes/no answer.  If it
> actually just outputs yes/no, well how is anyone supposed to know that and fix
> that, given that it does not seem to be an open source project?)

Honestly I don't know. I ran a few experiments with natural language
processors such as GPT-3 on commit messages which contained human-readable
instructions, and asking "what am I expected to do with these patches", and
seeing the bot respond "you should backport them to this version, change
this and that in that version, and preliminary take that patch". It
summarized extremely well the instructions delivered by the developer,
which is awesome, but was not able to provide any form of confidence
level. I don't know what Sasha uses but wouldn't be surprised it shares
some such mechanisms and that it might not always be easy to get such a
confidence level. But I could be wrong.

Willy