Re: Unknown NMI after resume from hibernation: restore kernel leaves the NMI watchdog counter running on CPU 0

[Rafael J. Wysocki (Intel)]

On Wed, Oct 28, 2026 at 3:23 AM Sergei <fistboom@xxxxxxxxx> wrote:
>
> Hi,
>
> I'm hitting the problem Tony reported in July 2024 [1] on an AMD laptop,
> and as far as I can tell it's still there in current mainline, so here's
> another data point.
>
> After almost every resume from hibernation I get exactly one
>
>   Uhhuh. NMI received for unknown reason 2c on CPU 0.
>   Dazed and confused, but trying to continue
>
> somewhere between 1 and ~130 seconds after "hibernation exit". It shows
> up after 25 of the last 28 hibernation cycles, always on CPU 0, and never
> after suspend-to-RAM.
>
> Machine: Huawei NBLL-WXX9, AMD Ryzen 5 4500U (Renoir, family 0x17),
> BIOS 1.07. Seen on 7.2.3 and reproduced on 7.3-rc1 (both Manjaro builds;
> the only taint is U, from ryzenadj writing to PCI config space). TLP is
> installed and sets kernel.nmi_watchdog=0 at runtime (that's its default);
> the kernel command line doesn't touch the watchdog, so it is on at boot.
>
> It's the same sequence as in [1]: the restore kernel boots with the NMI
> watchdog enabled, programs the perf counter on every CPU, takes the APs
> down and jumps into the image.

So the restore kernel shouldn't do that.  It should disable the
watchdog before jumping into the image exactly because it doesn't know
whether or not the watchdog is enabled in the image kernel.

> The image kernel has the watchdog off, so
> nothing owns the counter that is still running on CPU 0. When it
> overflows, perf_event_nmi_handler() finds no active events and the NMI
> ends up as unknown. After the wrap the counter just keeps counting from
> zero, which is why there is only one NMI per resume.
>
> Reading the PMU MSRs right after a resume on 7.3-rc1 shows it directly:
>
>   CPU0:    PERF_CTL0 = 0x530076  PERF_CTR0 = 0xfffe53fa3a48  (EN, INT)
>   CPU1..5: PERF_CTL0 = 0x130076  PERF_CTR0 = 0xfffa...-0xfffe...  (INT only)
>
> Event 0x76 with the counters sitting at about -(watchdog_thresh * cpu_khz)
> (23.7e9 here) is the watchdog's setup. CPU0 had about 7.2e9 cycles left,
> and the unknown NMI came 69 seconds after "hibernation exit".
>
> Booting with nmi_watchdog=0, so the restore kernel doesn't arm it either,
> makes it go away: after a hibernation cycle every PERF_CTLn is 0 on all
> CPUs and no NMI shows up.
>
> I don't see anything in the resume path that stops the hardlockup
> detector before swsusp_arch_resume() (resume_target_kernel() only touches
> the softlockup watchdog), and the two patches from [1] never got a reply.
> Laptops running TLP probably all see this, since turning the NMI watchdog
> off is TLP's default.
>
> Is the direction Pavel suggested back then (suspend/resume hooks in the
> watchdog code) still the preferred one, or should the restore kernel just
> stop the hardlockup detector before jumping into the image?

It should do this IMV.

> I'm happy to test patches.

Added to the todo list, but no promises on the time frame.

> [1] https://lkml.iu.edu/hypermail/linux/kernel/2407.3/01199.html