Re: [RFC PATCH v1 0/2] genirq: add migratable NMI registration and show NMI counts
From: Ashok Raj
Date: Mon Nov 16 2026 - 13:31:44 EST
Hi Vladimir
On Mon, Nov 16, 2026 at 10:38:00AM +0000, Vladimir Murzin wrote:
> Hi Ashok,
>
> On 11/14/26 23:49, Ashok Raj wrote:
> > request_nmi() currently models NMIs as per-CPU sources. This does not
> > represent a globally routed NMI source with an affinity-capable irqchip,
> > which needs to be rerouted when its target CPU is offlined.
> >
> > This RFC adds IRQF_NMI_MIGRATABLE as a request_nmi() registration mode
> > for such a source. It remains excluded from normal IRQ balancing, but the
> > generic startup and CPU-hotplug paths may select an online target. The
> > second patch fixes the related /proc/interrupts display: NMI-safe flow
> > handling updates the per-CPU counters rather than irq_desc::tot_count.
> >
> > Patch 1: Add the migratable-NMI registration mode, validate that it is
> > used only through request_nmi() with an affinity-capable non-per-CPU
> > irqchip, and add IRQ KUnit coverage for registration and CPU-offline
> > migration.
> >
> > Patch 2: Emit the per-CPU counters for all NMI descriptors, matching
> > kstat_irqs_desc().
> >
> > Testing:
> > - ARM64 changed-object builds with WERROR=1
> > - ARM64 !SMP changed-object build
> > - ARM64 IRQ KUnit run under QEMU with two CPUs, including registration
> > validation and CPU-offline migration
> > - local downstream GHES delivery and migration validation for the intended
> > consumer topology (not part of this series)
> >
> > Base and dependencies:
> >
> > This RFC was developed and tested on v7.4-rc2 plus the FEAT_NMI v2 series
> > [1]. It depends on that series' GIC NMI support and is therefore not directly
> > applicable to mainline or linux-next as posted. This RFC is for design
> > feedback; it will need rebasing onto the upstream form of those prerequisites
> > before submission.
> >
> > [1] https://lore.kernel.org/linux-arm-kernel/20261112132042.129062-1-vladimir.murzin@xxxxxxx/
> >
>
> I'm not an expert in this area, but based on what I've learned so far,
> NMI being per-CPU like makes life much easier ;)
I'm still learning the ARM side :-), so please correct me if I've got
something wrong.
The events I'm working on are platform RAS events described via:
HEST -> GHES entries.
My prototype uses an extended Type-4 GHES entry (Type-4 is
just for the RFC discussion here; I also have an alternative based on Marc's
earlier suggestion).
These are SPIs, so they can be routed to any CPU. But request_nmi() as
structured picks one CPU and pins the interrupt to it, treating it as
per-CPU. If that CPU goes offline, the interrupt is orphaned.
I see two ways to handle this:
Option 1: Retarget on CPU offline. My POC registers a hotplug callback that
picks a new target CPU. If none is available (e.g. this is the last online
CPU), it vetoes the offline. This works, but it feels like a band-aid.
Option 2: Extend the API for SPIs. Add a flag that lets the caller opt out
of the per-CPU requirement. Affinity migration and retargeting on offline
then follow the normal SPI path, which seems more natural to me.
I prefer Option 2. If it fits your series, I'll post the POC as an RFC for
feedback.
> IIUC, one of the main problems with a shared NMI (as opposed to a
> per-CPU) is handling shared state. [...] the same NMI handler could
> execute concurrently on two CPUs.
>
> IIUC, one of the main problems with a shared NMI (as opposed to a
> per-CPU) is handling shared state. Consider:
>
> CPU0 CPU1 CPU2
>
> ---------------------------------------------------------------------------
>
> <NMI>
> handling started
> set NMI affinity to CPU2
> <NMI>
> handling started
> mutate shared state
> mutate shared state
> handling finished
> handling finished
>
> IOW, the same NMI handler could execute concurrently on two CPUs,
> potentially corrupting shared state. Once again more than happy to
> be corrected if I missed something :)
There is no risk of the same normal, non-per-CPU IRQ action handler
executing concurrently on both the previous and newly selected CPUs during
an affinity change. Genirq sets IRQD_IRQ_INPROGRESS before dropping the
descriptor lock to invoke the driver's hard-IRQ handler. If another instance
arrives on the new target while the handler remains active on the previous
target, the new CPU observes that in-progress state and does not enter the
action handler.
The affinity update is not a synchronous “wait for the old CPU” operation:
the interrupt controller can route a subsequent interrupt to the new CPU
before the old handler returns. However, handler execution remains
serialized:
Previous CPU New target CPU
------------ --------------
set IRQD_IRQ_INPROGRESS
drop desc->lock
run driver's hard-IRQ handler
affinity target is reprogrammed
a subsequent IRQ arrives
observes IRQD_IRQ_INPROGRESS
does not run the action handler
handler returns
reacquire desc->lock
clear IRQD_IRQ_INPROGRESS
a later or replayed IRQ may run the
action handler on the new CPU
Any required pending/replay behavior for an event arriving during this
interval is handled by the applicable IRQ flow handler and irqchip semantics;
it does not cause simultaneous invocations of the driver's hard-IRQ handler.
Does this make sense?
Cheers,
Ashok