Re: [REGRESSION] 6.12.36+: usb: hub: post-resume delayed work triggers > uncorrected MCE / data fabric sync flood on Threadripper 7970X > (bisected to aec11e5f9c45)

[Mathieu Fluhr]

On Sun, Aug 23, 2026 at 12:37 PM Thorsten Leemhuis
<regressions@xxxxxxxxxxxxx> wrote:
>
> On 8/23/26 12:15, Mathieu Fluhr wrote:
> Thx for the report. Side note: mentioning the essence at the top before
> explaining backstory and details is usually the way better approach to
> get people interested in a bug report, especially if it's more that one
> or two paras.

Point taken :)

> Anyway, as you mentioned later and quoted below, your problems are
> caused by a 6.12.y backport of 8f5b7e2bec1c36 ("usb: hub: fix detection
> of high tier USB3 devices behind suspended hubs") [v6.16-rc5 (2025-07-06
> 23:10:26)], authored by Mathias Nyman (now CCed).
>
> There were at least three earlier reports about problems caused by that
> commit, all mentioning adb (which I assume you use, too):

Yes, looking at the reports, this looks definitely related.
What is happening when the emulator starts is that it also starts adbd
if not running on the Linux side, to communicate with the emulator
later on.

So, this could perfectly explain the persistent state after the
"Android 15 priming": Since the Android 15 repo contains a more recent
ADB, where the issue has been eventually wored around, it is started
firs, and later, when 11 starts, it does not kill it, restarting its
older adbd, but instead keep the new one running.

So, long story short, on my side, I was still looking at the "wrong
side" of the problem, trying to find what QEMU was doing wrong, where
in the end, it was more the interactions that adb was doing with the
USB bus during C2 state that was causing the issue.

>
> https://lore.kernel.org/all/qc0nhk9c6l0a08bkfeplrm3qjssgrjkvkp@xxxxxxxxx/
> https://lore.kernel.org/all/CAGQo=TJXef-QzbkFEpTcJ138j0k6WYnV4=tNY8Q7s3U7DUOY+Q@xxxxxxxxxxxxxx/
> https://bugzilla.kernel.org/show_bug.cgi?id=221103
>
> FWIW, https://bugzilla.kernel.org/show_bug.cgi?id=221103#c16 mentions
> the workaround you apparently found, too. The patch Mario (now CCed,
> just like the authors of the two mailed reports linked above; and
> Michael, too, who seems to be involved) proposed in the last comment of
> that ticket apparently didn't help. And no other fix is in sight, or am
> I missing something?
>
> Mathias reverting the culprit in mainline a option to resolve this (I
> assume it is not, but I'm asking just to be sure)?

Ok, here you lost me a little: Looking to all the links provided, yes,
this looks very similar to what I am experiencing. But (and correct me
if I am wrong)
1. reverting the "bad commit" is not really an option, since it was
originally authored for a specific purpose (other than annoying AMD
owners :)
2. the patch that Mario proposed is very recent, and would address the
issue not on the USB but on the PCI level?

Thanks again for the swift answer here and all the informative links!
I must admit that I researched a lot debugging this issue, but I never
thought about looking to the USB side of the problem, focusing more on
the CPU side.

Kind Regards,
Mathieu

>
> Ciao, Thorsten
>
> >>> As an AOSP developer I am compiling daily different versions of AOSP on it,
> >>> mostly building an Android (Automotive) emulator for quick "code-build-test"
> >>> runs. I use Ubuntu 24.04 LTS for that, updating to the latest HWE kernels that
> >>> Ubuntu provide there (6.8, 6.11, 6.14 etc. until 7.0 recently), with no manual
> >>> modifications applied.
> >>>
> >>> 3 weeks ago, I needed to analyze and issue happening inside Android 11,
> >>> building an emulator for a simple Android phone [1]. But when I started using
> >>> this emulator, first with the latest 7.0 kernel I noticed several hard crashes,
> >>> sometimes just freezing my workstation (with fans full on, but sometimes fans
> >>> full off), but also sometimes automatically rebooting it.
> >>>
> >>> After a few days of deep investigations (To be honest, I first suspected an
> >>> issue with the nivida driver), I found out that a pattern to reproduce this
> >>> quickly was to let the computer idle with the emulator running. The crash was
> >>> always occurring under 20/25 minutes, most of the time letting it idle for less
> >>> than 10 was even sufficient.
> >>>
> >>> This made me a bit curious, and looking a bit deeper (and with a little help
> >>> of AI) I was able to find a workaround. forbidding my CPU to enter C2 state,
> >>> using "processor.max_cstate=1" argument. Using this, I was not able to
> >>> reproduce the crash for more than an hour, but I did not pursue there very
> >>> much: As a developer I hate workarounds :)
> >>>
> >>> I instead started to get "back in time" on a clean and up-to-date fresh Ubuntu
> >>> 24.04 install, reverting back to "good old" kernel versions, since I could not
> >>> convince myself that first my CPU was dying and second that the issue has always
> >>> been there. I confirmed that 6.8 and 6.11 (again Ubuntu HWE packages) work
> >>> fine, but the crash was reproducible using 6.14 and above.
> >>>
> >>> I then got my hands dirty, and started to test different mainline kernel
> >>> prebuilt packages up to a point where I confirmed that 6.12.35 worked fine, but
> >>> 6.12.40 not. I then bisected both versions, ensuring a good case meant the
> >>> emulator was idling without any crash for 1 hour minimum. This lead at the end
> >>> to the following commit:
> >>>
> >>>   aec11e5f9c452ef64e2c113637ab89a67a5ceb62
> >>>   ("usb: hub: fix detection of high tier USB3 devices behind suspended hubs")
> >>>   [mainline: 8f5b7e2bec1c36578fdaa74a6951833541103e27]
> >>>
> >>> Being quite astonished that something related to C2 state was triggered by an
> >>> USB patch, I then tested the latest 7.0 kernel, this time using the
> >>> "usbcore.autosuspend=-1" argument instead. To my surprise, I could not
> >>> reproduce the crash, even with the exact same emulator idling for 2 hours.
> >>>
> >>> Also, something very astonishing, that I still cannot fully understand today:
> >>>  1. "priming" my system with a 30 seconds (!) run of a modern Android
> >>>      emulator [2] cleared the issue: After closing the 15 emulator and starting
> >>>      the 11, I could let it idle for again more than an hour. It seems even not
> >>>      be related to the 'kvm' kernel modules, since removing the module and
> >>>      re-inserting it between both emulator did not change a thing.
> >>>  2. A few times (I did not really invest debugging this TBH), the crash even
> >>>     occurred shortly (2-3 minutes) after I closed the Android 11 emulator.
> >>>
> >>> Now, the technical details of my setup...
> >>>
> >>> Hardware / software
> >>> ===================
> >>>
> >>> CPU:        AMD Ryzen Threadripper 7970X (Storm Peak, family 19h)
> >>> Microcode:  0x0a10810c
> >>> Board:      ASUS Pro WS TRX50-SAGE WIFI, BIOS 1203 and 1317 (both affected)
> >>> Memory:     128 GB DDR5 ECC RDIMM @ 4800 MT/s (JEDEC, no XMP/EXPO)
> >>> GPU:        NVIDIA RTX 3080 Ti (also reproduced with RTX 5080)
> >>> Distro:     Ubuntu 24.04 and 26.04 with 595-open NVIDIA driver
> >>> USB:        8 onboard xHCI controllers; only a USB keyboard and mouse
> >>>             attached
> >>>
> >>> Error signature
> >>> ===============
> >>>
> >>> On the boot following each crash I could always see the following lines in
> >>> the dmesg logs:
> >>> ---8<-------------------------------------------------------------------------
> >>>   x86/amd: Previous system reset reason [0x88000800]: an uncorrected
> >>>            error caused a data fabric sync flood event
> >>>   x86/amd: Previous system reset reason [0x88000800]: a software sync
> >>>            flood event occurred
> >>> ---8<-------------------------------------------------------------------------
> >>>
> >>> When I was fortunate enough and had an automatic reboot, this was also inside:
> >>> ---8<-------------------------------------------------------------------------
> >>>   [Hardware Error]: event severity: fatal
> >>>   [Hardware Error]:   section_type: IA32/X64 processor error
> >>>   [Hardware Error]:    Error Structure Type: cache error
> >>>   [Hardware Error]:    Check Information: 0x000000000602001f
> >>>   [Hardware Error]:     Transaction Type: 2, Generic
> >>>   [Hardware Error]:     Level: 0
> >>>   [Hardware Error]:     Processor Context Corrupt: true
> >>>   [Hardware Error]:     Uncorrected: true
> >>>   mce: [Hardware Error]: CPU 34: Machine Check: 0 Bank 5: aea0000000000108
> >>>   mce: [Hardware Error]: TSC 0 ADDR 1ffffff98e8ee38 MISC d0150fff00000000
> >>>        SYND 4d000000 IPID 500b020049b00
> >>> ---8<-------------------------------------------------------------------------
> >>>
> >>> The signature is bit-identical across every occurrence except for the
> >>> reporting CPU, which varies (observed on CPUs 1, 3, 7, 16, 32, 34, 35).
> >>>
> >>>
> >>> Reproducer
> >>> ==========
> >>>
> >>> 1. Boot an affected kernel with default idle settings (C2 available,
> >>>    no max_cstate restriction).
> >>> 2. Start an old and/or 32/64-bit Android emulator (QEMU/KVM guest) and leave it
> >>>    idle on the launcher screen. Nothing else running.
> >>> 3. System hard-resets within 20 minutes.
> >>>
> >>> Under sustained CPU load the fault never occurs; it requires the system to be
> >>> idle. turbostat confirms ~99% C2 residency across all cores in the crashing
> >>> condition.
> >>>
> >>>
> >>> Bisection
> >>> =========
> >>>
> >>> ---8<-------------------------------------------------------------------------
> >>> git bisect start
> >>> # status: waiting for both good and bad commits
> >>> # good: [783cd2c3dca8b6c434e955b84c20c8940588dc68] Linux 6.12.35
> >>> git bisect good 783cd2c3dca8b6c434e955b84c20c8940588dc68
> >>> # status: waiting for bad commit, 1 good commit known
> >>> # bad: [d90ecb2b1308b3e362ec4c21ff7cf0a051b445df] Linux 6.12.40
> >>> git bisect bad d90ecb2b1308b3e362ec4c21ff7cf0a051b445df
> >>> # good: [3ce57d493dd81ecc27033c7a918cc0d0ddaa8edc] ata: libata-acpi:
> >>> Do not assume 40 wire cable if no devices are enabled
> >>> git bisect good 3ce57d493dd81ecc27033c7a918cc0d0ddaa8edc
> >>> # good: [bbd385b65f9e56cab1243e753510a99a59110083] net/mlx5e: Add new
> >>> prio for promiscuous mode
> >>> git bisect good bbd385b65f9e56cab1243e753510a99a59110083
> >>> # good: [2a76bc2b24ed889a689fb1c9015307bf16aafb5b] smb: client: fix
> >>> use-after-free in crypt_message when using async crypto
> >>> git bisect good 2a76bc2b24ed889a689fb1c9015307bf16aafb5b
> >>> # good: [95a13b0a6b042ac5333b67f59af1dffe7b71137b] riscv:
> >>> traps_misaligned: properly sign extend value in misaligned load
> >>> handler
> >>> git bisect good 95a13b0a6b042ac5333b67f59af1dffe7b71137b
> >>> # good: [40b5b4ba8ed87c0bfb6268c10589777652ebde4c] drm/mediatek: Add
> >>> wait_event_timeout when disabling plane
> >>> git bisect good 40b5b4ba8ed87c0bfb6268c10589777652ebde4c
> >>> # bad: [affb46db59f908474a211f23953c3b9109f0d647] net: libwx: fix
> >>> multicast packets received count
> >>> git bisect bad affb46db59f908474a211f23953c3b9109f0d647
> >>> # good: [ee56da95f8962b86fec4ef93f866e64c8d025a58] btrfs: fix block
> >>> group refcount race in btrfs_create_pending_block_groups()
> >>> git bisect good ee56da95f8962b86fec4ef93f866e64c8d025a58
> >>> # bad: [bf71baa3cfe754c6e99ad62711b9fb3e3da33517] usb: hub: Fix
> >>> flushing of delayed work used for post resume purposes
> >>> git bisect bad bf71baa3cfe754c6e99ad62711b9fb3e3da33517
> >>> # bad: [e11359640090ab26e4153cce583f50e0b8979f0d] usb: hub: Fix
> >>> flushing and scheduling of delayed work that tunes runtime pm
> >>> git bisect bad e11359640090ab26e4153cce583f50e0b8979f0d
> >>> # bad: [aec11e5f9c452ef64e2c113637ab89a67a5ceb62] usb: hub: fix
> >>> detection of high tier USB3 devices behind suspended hubs
> >>> git bisect bad aec11e5f9c452ef64e2c113637ab89a67a5ceb62
> >>> # first bad commit: [aec11e5f9c452ef64e2c113637ab89a67a5ceb62] usb:
> >>> hub: fix detection of high tier USB3 devices behind suspended hubs
> >>> ---8<-------------------------------------------------------------------------
> >>>
> >>> (Please note that I did not perform a full "revert test", since the aec11e5f9c45
> >>> commit could not be cleanly reverted on both 6.12.40 and 7.0.0)
> >>>
> >>>
> >>> Workarounds
> >>> ===========
> >>>
> >>> Either of these prevents the crash on an affected kernel:
> >>>   usbcore.autosuspend=-1     (disables USB runtime PM)
> >>>   processor.max_cstate=1     (prevents C2 entry)
> >>>
> >>>
> >>> Ruled out
> >>> =========
> >>>
> >>> - GPU/driver: reproduced with RTX 5080 (595-open module) and RTX 3080 Ti
> >>>               (both 595-open and 595 proprietary modules)
> >>> - ASUS USB4 PCIe add-in card: reproduces in ~5 min with the card physically
> >>>                               removed
> >>> - AVIC: kvm_amd avic=N on both good and bad kernels
> >>> - TSA mitigation: tsa=off verified applied
> >>>   (/sys/devices/system/cpu/vulnerabilities/tsa = "Vulnerable"), still crashes
> >>> - Memory: stock JEDEC 4800, no XMP/EXPO, no ECC errors logged
> >>>   (ras-mc-ctl reports zero CE/UE)
> >>> - Thermal/power: ~47-50 C at time of crash, system idle; 1600W Seasonic PSU
> >>> - Hardware health: multi-hour AOSP compiles (all 64 threads, heavy cache
> >>>   load) and repeated GPU stress runs complete without error
> >>>
> >>>
> >>> I hope this gives you enough details to start looking at what could cause this
> >>> weird behavior. Just FYI, when asked about hardware damage, AI suggested more
> >>> something like "a CPU-level microcode erratum in the deep-idle path on this
> >>> platform", but being old-school, I tend to always triple-check what AI tells
> >>> before claiming it myself :)
> >>>
> >>> Thanks and Kind Regards,
> >>> Mathieu
> >>>
> >>> ------
> >>> [1] using android-11.0.0_r46 release tag and "sdk_phone_x86_64" as lunch target
> >>> [3] using android-15.0.0_r32 release tag and "sdk_phone64_x86_64" as lunch
> >>>     target this time, since Google only releases 64-bit only today
> >
>