Re: [REGRESSION?] hibernation soft lockup in migration/0 at multi_cpu_stop after image creation

[Thorsten Leemhuis]

[Sending this with a amdgpu list and maintainers in CC to ensure they
know about this. Note, looks a bit like this is another variant of
https://gitlab.freedesktop.org/drm/amd/-/work_items/5387 ]

On 7/10/26 11:07, Hlib Wondertan wrote:
> Last known actually working: unknown. I do not currently have a kernel version where hibernation was known to work reliably on this machine.
> 
> For this exact `migration/0` soft-lockup signature, however, I only remember seeing it after updating to `7.1.2-arch3-1`. I do not have pstore evidence of this soft-lockup on the earlier 7.0.x kernels and I've logged every failure while staying on 7. Therefore, there is a chance this is a regression.
> 
> What I have locally is:
> 
> 7.0.11-arch1-1: hibernate/image-not-found failure with pstore pointing at amdgpu/ttm, but not this soft-lockup
> 7.0.12/7.0.13/7.0.14: other hibernate/resume failures, not always with useful pstore, and no captured migration/0 soft-lockup
> 7.1.2-arch3-1: current pstore captures show the soft-lockup in migration/0 after image creation
> 
> 
> So yes, this may be related to the drm/amd issue you linked: https://gitlab.freedesktop.org/drm/amd/-/work_items/5387 <https://gitlab.freedesktop.org/drm/amd/-/work_items/5387>
> 
> 
> I have seen the 5387-style TTM/amdgpu issue quite often on this machine before 7.1 and also on 7.1, and I have been reporting those reproductions in that GitLab issue as well. The reason I filed this separately is that the visible pstore signature changed from the TTM/amdgpu oops path to a hibernation `migration/0` soft-lockup after image creation.
> 
> Mainline 7.2-rc: not tested yet. The only currently installed/running kernel here is `7.1.2-arch3-1`.
> 
> I can test a mainline/vanilla 7.2-rc kernel next and report whether it is still affected. If it still reproduces, I can then work on finding a real good/bad range for a bisect; right now I do not have a trustworthy last-good baseline.
> 
> 
> 
> On Fri, 10 Jul 2026 at 12:54, Hlib Wondertan <hlibwondertan@xxxxxxxxx
> <mailto:hlibwondertan@xxxxxxxxx>> wrote:
> 
> 
>     Last known actually working: unknown. I do not currently have a kernel version where hibernation was known to work reliably on this machine.
> 
>     For this exact `migration/0` soft-lockup signature, however, I only remember seeing it after updating to `7.1.2-arch3-1`. I do not have pstore evidence of this soft-lockup on the earlier 7.0.x kernels and I've logged every failure while staying on 7. Therefore, there is a chance this is a regression.
> 
>     What I have locally is:
> 
>     ```text
>     7.0.11-arch1-1: hibernate/image-not-found failure with pstore pointing at amdgpu/ttm, but not this soft-lockup
>     7.0.12/7.0.13/7.0.14: other hibernate/resume failures, not always with useful pstore, and no captured migration/0 soft-lockup
>     7.1.2-arch3-1: current pstore captures show the soft-lockup in migration/0 after image creation
>     ```
> 
>     So yes, this may be related to the drm/amd issue you linked:
> 
>     ```text
>     https://gitlab.freedesktop.org/drm/amd/-/work_items/5387 <https://gitlab.freedesktop.org/drm/amd/-/work_items/5387>
>     ```
> 
>     I have seen the 5387-style TTM/amdgpu issue quite often on this machine before 7.1 and also on 7.1, and I have been reporting those reproductions in that GitLab issue as well. The reason I filed this separately is that the visible pstore signature changed from the TTM/amdgpu oops path to a hibernation `migration/0` soft-lockup after image creation.
> 
>     Mainline 7.2-rc: not tested yet. The only currently installed/running kernel here is `7.1.2-arch3-1`.
> 
>     I can test a mainline/vanilla 7.2-rc kernel next and report whether it is still affected. If it still reproduces, I can then work on finding a real good/bad range for a bisect; right now I do not have a trustworthy last-good baseline.
> 
> 
> 
>     On Fri, 10 Jul 2026 at 11:19, Thorsten Leemhuis
>     <regressions@xxxxxxxxxxxxx <mailto:regressions@xxxxxxxxxxxxx>> wrote:
> 
>         Hi!
> 
>         On 7/9/26 20:38, Hlib Wondertan wrote:
>         >
>         > I am seeing hibernation failures on a Framework Laptop 16 AMD
>         Ryzen
>         > AI 300 Series. The best capture I have is from EFI pstore: the
>         kernel
>         > creates the hibernation image, then CPU0 soft-locks in the CPU
>         stopper
>         > thread and panics.
> 
>         Thx for the report. You CCed the regression list, which leads to a
>         questions: what was the last version this was working?
> 
>         Furthermore: Is mainline (7.2-rc) still affected?
> 
>         And FWIW, if we are lucky some developer will show up in the
>         next few
>         days with an idea what's wrong. But I doubt this will happen in this
>         case. So you most likely will have to bisect this to get this
>         resolved:
> 
>         https://docs.kernel.org/admin-guide/verify-bugs-and-bisect-
>         regressions.html <https://docs.kernel.org/admin-guide/verify-
>         bugs-and-bisect-regressions.html>
>         https://docs.kernel.org/admin-guide/bug-bisect.html <https://
>         docs.kernel.org/admin-guide/bug-bisect.html>
> 
>         Ciao, Thorsten
>         P.S.: Wild guess, but I wonder if this is related to (also
>         waiting for
>         somebody to bisect):
>         https://gitlab.freedesktop.org/drm/amd/-/work_items/5387
>         <https://gitlab.freedesktop.org/drm/amd/-/work_items/5387>
> 
>         > Impact: the machine powers/reboots instead of resuming the
>         previous
>         > session. The next boot reported no valid hibernation image.
>         >
>         > System:
>         >
>         > ```text
>         > Machine: Framework Laptop 16 (AMD Ryzen AI 300 Series)
>         > DMI board: FRANMHCP09
>         > BIOS: 03.06, 2026-04-23
>         > Kernel: 7.1.2-arch3-1
>         > Arch Linux
>         > ```
>         >
>         > Kernel config has the debug options requested for this class
>         of issue:
>         >
>         > ```text
>         > CONFIG_HIBERNATION=y
>         > CONFIG_PM_DEBUG=y
>         > CONFIG_PM_SLEEP_DEBUG=y
>         > CONFIG_SLUB_DEBUG=y
>         > CONFIG_PAGE_POISONING=y
>         > CONFIG_STACKTRACE=y
>         > ```
>         >
>         > Relevant command line, with the root UUID elided:
>         >
>         > ```text
>         > root=/dev/mapper/root rootfstype=btrfs rootflags=subvol=@ rw
>         > resume=/dev/mapper/root resume_offset=4728064
>         > rtc_cmos.use_acpi_alarm=1 drm.debug=0x1df log_buf_len=32M
>         > ignore_loglevel printk.devkmsg=on slub_debug=UFPZ
>         > ```
>         >
>         > The hibernate attempt was triggered by upowerd at low battery.
>         A systemd sleep hook ran before hibernation and captured:
>         >
>         > ```text
>         > phase=pre
>         > operation=hibernate
>         > date=2026-07-09T17:45:04+04:00
>         > uname=Linux Wonderframe 7.1.2-arch3-1 #1 SMP PREEMPT_DYNAMIC
>         Fri, 03 Jul 2026 23:25:36 +0000 x86_64 GNU/Linux
>         > pm_test=[none] core processors platform devices freezer
>         > disk=platform [shutdown] reboot suspend test_resume
>         > panic_on_oops=1
>         > hardlockup_panic=1
>         > softlockup_panic=1
>         >
>         > BAT1 status=Discharging
>         > BAT1 capacity=4
>         > MemAvailable:   18303436 kB
>         > SwapTotal:      48212748 kB
>         > SwapFree:       48212748 kB
>         > HardwareCorrupted:     0 kB
>         > ```
>         >
>         > Relevant GPUs:
>         >
>         > ```text
>         > amdgpu 0000:c4:00.0 PCI_ID=1002:7480
>         > amdgpu 0000:c5:00.0 PCI_ID=1002:150E
>         > ```
>         >
>         > pstore excerpt from the failed hibernate:
>         >
>         > ```text
>         > [17199.818113] PM: hibernation: Creating image
>         > [17200.075162] PM: hibernation: Need to copy 3578614 pages
>         > [17200.075167] PM: hibernation: Normal pages needed: 3578614 +
>         1024, available pages: 4622595
>         > [17223.501212] PM: hibernation: Image created (2866538 pages
>         copied, 712076 zero pages)
>         > [17224.160864] watchdog: BUG: soft lockup - CPU#0 stuck for
>         26s! [migration/0:20]
>         > [17224.160965] CPU: 0 UID: 0 PID: 20 Comm: migration/0 Not
>         tainted 7.1.2-arch3-1 #1 PREEMPT(full) 
>         914ec70c9fc54a99dc91d290948a1dd1bb226c49
>         > [17224.160969] Hardware name: Framework Laptop 16 (AMD Ryzen
>         AI 300 Series)/FRANMHCP09, BIOS 03.06 04/23/2026
>         > [17224.160970] Stopper: multi_cpu_stop+0x0/0x140 <-
>         __stop_cpus.constprop.0+0x6d/0xb0
>         > [17224.160980] RIP: 0010:multi_cpu_stop+0x87/0x140
>         > [17224.161015] Kernel panic - not syncing: softlockup: hung tasks
>         > [17224.161017] CPU: 0 UID: 0 PID: 20 Comm: migration/0
>         Tainted: G             L      7.1.2-arch3-1 #1 PREEMPT(full) 
>         914ec70c9fc54a99dc91d290948a1dd1bb226c49
>         > [17224.161019] Tainted: [L]=SOFTLOCKUP
>         > [17224.161020] Hardware name: Framework Laptop 16 (AMD Ryzen
>         AI 300 Series)/FRANMHCP09, BIOS 03.06 04/23/2026
>         > [17224.161020] Stopper: multi_cpu_stop+0x0/0x140 <-
>         __stop_cpus.constprop.0+0x6d/0xb0
>         > [17224.161063] RIP: 0010:multi_cpu_stop+0x87/0x140
>         > ```
>         >
>         > There were no SLUB/page-poison corruption messages in the
>         pstore chunks, despite `slub_debug=UFPZ`.
>         >
>         > An earlier hibernation pstore capture on the same kernel
>         showed the same broad failure class, also in `migration/0`, but
>         with `RIP: stop_machine_yield+0x6/0x10`.
>         >
>         > I have attached:
>         >
>         > ```text
>         > pstore-20260709-1748.tar.gz
>         > hibernate-pre-20260709-174504.log
>         > ```
>         >
>         > Please let me know what additional data would be useful. I can
>         keep the debug boot parameters enabled and retest with a
>         proposed patch or additional PM/debug options.
>         >
>         >
>