regressions with available fixes that take their time towards mainline as of post 7.4-rc2
From: Thorsten Leemhuis
Date: Thu Nov 12 2026 - 10:53:10 EST
Hi Linus!
How much do you care about the following you wrote earlier this year in
https://lore.kernel.org/all/CAHk-%3Dwi86AosXs66-yi54%2BmpQjPu0upxB8ZAfG%2BLsMyJmcuMSA@xxxxxxxxxxxxxx/
""[...] if it's a regression with a known commit that caused it, I think
the rule of thumb should generally be "[fix] within a week", preferably
before the next rc.""
I'm asking, as I see regressions left and right taking much longer --
even when they easily could be fixed within that timeframe or maybe a
little more. And my efforts to motivate people to get at least somewhat
closer to that rule of thumb seem less and less fruitful, as people
ignore me partially or completely -- just like the "Expectations and
best practices for fixing regressions" in
Documentation/process/handling-regressions.rst, which even give a bit
more time than your rule of thumb.
Of course I could ask you every week to pull some fixes directly (like I
did the past cycle) or even collect them in a tree and submit them as PR
to you (the idea came up there). But well, bypassing subsystem
maintainers' knowledge and their CIs should be an exception, as it will
lead to problems or disaster sooner or later for obvious reasons (and I
guess won't scale either).
I also fully understand that the AI patch influx is a reason why some
regressions take longer to get fixed, as my inquiries and prodding put
even more load on already overloaded people. At the same time the
current patch volume seems to be the new normal for at least the near
future. And overall the core problems that lead to slowly fixed
regressions from what I see are the same as before, they just occur more
often.
The core problem is basically: the "no regression" rule is well known,
but people interpret vastly differently how to realize it in practice --
and often it's reviewing, committing, and mainlining fixes, as outlined
a few months ago here in an analysis for the maintainers summit:
https://lore.kernel.org/ksummit/75dc0e2d-2deb-4776-977c-1de7c33465e1@xxxxxxxxxxxxx/
Instead of approaching this from a theoretical and retrospective side,
let me this time point out regressions that are unfixed right now
despite available fixes:
######
# drm/msm/a8xx: failure during boot
Caused by 93b6c8d10fad (v7.4-rc1). Known before the merge window.
Proposed fix is a oneliner submitted to a different subsys on
2026-10-21. It got two Tested-by:, but no reply from the maintainer yet.
A prod from me about one week after submission didn't have any effect
afaics:
https://lore.kernel.org/all/20261021080517.2161-1-okerixx@xxxxxxxxx/
Report:
https://lore.kernel.org/lkml/ar_AuRr1eaZzOfw9@baldur/
Side note: here the msm maintainer explains why he mainlined the culprit
despite the known regression:
https://lore.kernel.org/lkml/CACSVV00x091i7BKW+bj0RUnkd4rOrHXbhZ4zF6SuTU1NAqsFTA@xxxxxxxxxxxxxx/
On a wider note:
There is no way for maintainers to automatically filter patch
submissions that fix user-reported recent regressions, which would allow
maintainers to prioritize such fixes (and me to better keep an eye on
those, too). This could have helped here, as the maintainer recently
applied fixes to his fixes branch, so it not like he is on vacation of
something like that. So he might not even be aware the patch fixes a
regression (that, admittedly, is strictly speacing a problem of another
subsystem, as it exposed it) -- or was unavailable for some time and
when coming back simply deleted old mails (which I can relate to, but
well, it's bad when it comes to regression fixes).
######
# phy: qcom: qmp-pcie: link never retrains and the root port comes up
with no endpoint
Caused by 910b828b22b7 (v7.3-rc1). A revert was posted about a month ago:
https://lore.kernel.org/lkml/20261013-qmp-pcie-skip-reset-revert-v1@xxxxxxxxxxxxxxxx/
I prodded the thread about two weeks ago; three days later it got
"Reviewed-by" from a subsystem reviewer. Today the maintainer finally
accepted the patch, so maybe it will hit mainline soon now.
On a wider note:
Might be a variant of the problem mentioned above: maintainers have no
way to automatically filter and prioritize patch submissions that fix
user-reported and recent regressions.
Could also be confusion on how to handle user-reported regressions from
recent development cycles. Lorenzo voiced those recently:
https://mastodonapp.uk/@ljs/117191681502501017 But he is not the only
one from what I see; that has always been the case, but it seems your
recent statement "Non-critical fixes to long-standing issues are simply
not appropriate for this late in the release cycle." in the Linux
7.1-rc5 announcement made this worse. Or maybe I'm confused, as I assume
the follow statement of yours is still true:
"I do consider "regression in an earlier release" to be a regression
that needs fixing. There's obviously a time limit: [...]"
https://lore.kernel.org/all/CAHk-=wis_qQy4oDNynNKi5b7Qhosmxtoj1jxo5wmB6SRUwQUBQ@xxxxxxxxxxxxxx/
######
# arm64: dts: imx8qm-ss-conn: USB 3.0 broken on Apalis iMX8QM
Caused by 73c3b40e536d (v7.4-rc1). Known for two weeks. Author and
committer is also the maintainer for the area in question. Posted a
revert nearly a week ago, but it is no heading towards mainline so far
afaics:
https://lore.kernel.org/lkml/20261106161809.3512378-1-Frank.Li@xxxxxxxxxxx/
######
# drm/msm/dp: SoC to reset
Caused by 1bda647ce61b (v7.4-rc1). Know for a week. The affected user
reported this by way of posting a patch for it, but it didn't get a
reply yet.
https://lore.kernel.org/lkml/20261104072048.3486286-1-burke.cates@xxxxxxxxx/
On a wider note: I see more users that are affected by a regression are
directly posting a patch for it. Those fixes thus lack a "Reported-by:"
tag, which makes it even harder for reviewers and maintainers to see
when a patch is a regression fix and prioritize those.
######
# drm: amdgpu: Black screen after boot with newer kernels in multi-GPU
system
Caused by 11c141672045 (v7.3-rc1). Known for 10 weeks, bisected about a
month ago. Reported at least six times by now afaics:
https://gitlab.freedesktop.org/drm/amd/-/work_items/5762
https://gitlab.freedesktop.org/drm/amd/-/work_items/5979
https://gitlab.freedesktop.org/drm/amd/-/work_items/5806
https://gitlab.freedesktop.org/drm/amd/-/work_items/6120
https://lore.kernel.org/amd-gfx/20261014080523.186533-1-rezanikzadch@xxxxxxxxx/
https://lore.kernel.org/amd-gfx/CAGxFSwnJYc03fN+XWU8oLW9EbXhmetQuj1s3-m8UdZHoBdGJ8Q@xxxxxxxxxxxxxx/
Fixes are being prepared for more than three weeks now, but not yet ready:
https://lore.kernel.org/lkml/20261019-vfct-regression-v1-0-5a63ddd378dd@xxxxxxx/
https://lore.kernel.org/lkml/20261111-vfct-regression-v6-0-393797bbb6e3@xxxxxxx/
A revert was considered and even submitted a month ago, but not
performed because it would "trade one regression for annother":
https://lore.kernel.org/dri-devel/20261019194106.2796150-1-alexander.deucher@xxxxxxx/
Might be the case, albeit the changes that would have been reverted do
not have a Fixes: tag. So this is not a clear case and might be a misfit
in this mail, still wanted to mention it due to the relatively huge
number of reports and the time it takes so far to get this resolved.
######
# ntfs3: SEEK_DATA reports WOF compressed files as holes, cp creates
zero-filled copies
Caused by c61326967728 (v7.0-rc1). The affected user reported this by
way of posting a patch for it more than a month ago now, but it didn't
get a reply yet.
https://lore.kernel.org/lkml/20261005223318.19303-1-marcin@xxxxxxxxxxxxx/
######
# wifi: iwlwifi: BE200: Wi-Fi interface becomes unusable due to firmware
faults
Known since the end of June. Two reports, both reporters saw the issue
starting to happen around the 7.1 timeframe -- but the cause was later
bisected to the 6.5 change e9b63341dc15, so it looks like something
(another kernel change, firmware, ...) exposed the problem earlier this
year:
https://bugzilla.kernel.org/show_bug.cgi?id=221675
https://bugzilla.kernel.org/show_bug.cgi?id=221898
A fix was attached to bugzilla and confirmed to be working at the end of
August -- and even merged to iwlwifi tree back then. It was then posted
to the lists end of September:
https://lore.kernel.org/all/20260930122439.08094e4c344f.Ic0a4bf544fd6ebaf1fc62b40bf463bd289e58cf0@changeid/
Since then it's also in -next, these days as fdd83f3ad06a26 ("wifi:
iwlwifi: honor the per-channel 320 MHz regulatory flag"). Near the end
of the latest merge window I asked why it's taking so long, but in reply
I only got "You really need to stop pressuring people. Everyone's
overloaded. Heaping more on doesn't help.". The fix from what I see is
still not on its way towards mainline. That being said: it feels a bit
like I'm missing something here -- but OTOH from both reports (one of
them linked in the fix) it looks like it's a recent regression that
takes quite a while to get fixed, despite an available fix.
######
Ciao, Thorsten