Re: [REGRESSION] Linux kernel 6.12.75 fails to compile with -Werror=implicit-function-declaration

[Peter Schneider]

Am 05.03.2026 um 21:21 schrieb Sasha Levin:
> On Thu, Mar 05, 2026 at 05:40:09PM +0000, Brett A C Sheffield wrote:
> > On 2026-03-04 20:00, Peter Schneider wrote:
> > > I already found and reported this in the RC cycle [1], and Sasha dropped it in -rc2 [2], and now in the release, it
> > > obviously has, somewhat mysteriously, reappeared [3], affecting all of today's 6.x stable branch releases.
> >
> > Greg, Sasha et al.
> >
> > Can we make a small adjustment to the stable kernel testing process please,
> > whereby we release a kernel that we have actually tested, instead of adding and
> > dropping patches at the last moment and releasing a kernel that no one has
> > tested?
> >
> > We are only a small pool of testers. If we find a bug, can we fix it, release a
> > new RC and test again please?  We can have an RC3. Even an RC4.  Perhaps if we
> > bogoselect fewer patches in the first place we might have less work to do. It's
> > better to miss a backport for a bug no one has reported than to pull stuff in
> > without proper review.
>
> Could you suggest which fixes from v6.19..v6.19.6 could have been left outside
> the tree?
>
> > The current stable process is introducing bugs. Bugs that never existed in
> > mainline.
>
> Releasing yesterday's tree was (my) human error: I don't have as much
> automation and scripting as Greg does, so many of the steps I've taken were
> manual and prone to errors. I'm working on improving this workflow on my end.
>
> This, however, wasn't an issue with our process, which is why I'm curious which
> bugs you're referring to?
>
> > The 3 kernels released today were tested by no one before release.
>
> Right - the 3 kernels released today simply dropped a commit that caused a
> built failure. We sometimes do that to address simple build or functionality
> breakages (this happened with v6.19.2 and v6.19.5 too).

I agree with that, and in this case I don't see a risk here. It's probably fine.

But I have a major headache regarding this one, because Brett is right here:

> > The seven kernels yesterday were similarly tested by no one before release.
> > We weren't given the opportunity.
>
> Could you explain this point please? There were quite a few folks who provided
> their Tested-by...

The people who tacked their Tested-by on yesterday's 6.1.165, 6.6.128 and 6.12.75 RC2 did so after testing without the 
patch "x86/kexec: add a sanity check on previous kernel's ima kexec buffer", because after my initial report you dropped 
it, but in the final releases it was present again (essentially invalidating the Tested-By), causing the same build 
failure, but only on X86, and only with CONFIG_WERROR=Y.

Now in todays three releases which fix this, you wrote in the release announcements "Only upgrade if you've observed a 
build failure with 6.1.165." / 6.6.128 / 6.12.75.

But this wording is, IMHO, inaccurate and inadquate, because people who built yesterdays releases with CONFIG_WERROR=N 
will not have seen a build failure, and if they now, because of your release announcement, DO NOT upgrade to one of 
todays releases, they now have a kernel with an incomplete patchset, i.e. with only the patch ""x86/kexec: add a sanity 
check on previous kernel's ima kexec buffer" and not the three missing prerequisite patches from Harshit's patch series 
which he said he will properly backport later, and this could be built only by lucky coincedence, and THIS is the 
combination of code nobody actually tested which Brett talked about.

What does it do? Does is cause harm? I don't know. Do you know? Maybe Harshit could tell us if it's a serious omission 
or if it's not critical. This, IMHO, should have been avoided. The better wording in the release announcements of today 
would have been: "All users on X86 must upgrade", so that nobody stays, unaware, on a kernel with that incomplete patch set.

This gives me some vibes of "It compiles, so let's ship it", which is a questionable level of QA you might expect from 
some (shady) commercial companies, but I think the Linux community should (and can!) do better. But I feel the process 
was kinda broken this time, not because of your fault, but because lack of consistency in tooling, as you said yourself. 
And also the unlucky timing of finding that bash/dash behaviour differnence in your tooling vs Greg's, that caused RC2 
in the first place.

Sorry if that sounds too harsh. I don't mean it in a harsh way, but only as constructive criticism. I know Greg and you 
have an extremely complex job with maintaining the stable branches, and normally all works very well and smooth, because 
both of you are doing a hell of an excellent job! But on occasion when a release was bumpy like this one, we should look 
back and ask why, and what could have done better, and how we can improve in the future.


Beste Grüße,
Peter Schneider

--
Climb the mountain not to plant your flag, but to embrace the challenge,
enjoy the air and behold the view. Climb it so you can see the world,
not so the world can see you.                    -- David McCullough Jr.

OpenPGP:  0xA3828BD796CCE11A8CADE8866E3A92C92C3FF244
Download: https://www.peters-netzplatz.de/download/pschneider1968_pub.asc
https://keys.mailvelope.com/pks/lookup?op=get&search=pschneider1968@xxxxxxxxxxxxxx
https://keys.mailvelope.com/pks/lookup?op=get&search=pschneider1968@xxxxxxxxx