Re: [PATCH v2] x86/nmi: Enable panic() when NMI handler runs too long

[Paul E. McKenney]

On Tue, Oct 20, 2026 at 11:27:00AM -0700, Paul E. McKenney wrote:
> On Tue, Oct 20, 2026 at 09:45:48AM -0700, Dave Hansen wrote:
> > On 10/20/26 09:15, Paul E. McKenney wrote:
> > >>> +	toolong_nmi_panic= [X86]
> > >>> +			Cause a panic upon return from an NMI handler
> > >>> +			that took longer than the specified number
> > >>> +			of milliseconds.  The default value of zero
> > >>> +			suppresses these panics.
> > >> Cause a panic after an NMI?!
> > > When the NMI handler runs for more than ten seconds, life is hard anyway.
> > > And no, these were not bugs in the several affected NMI handlers themselves.
> > > 
> > >> Oh boy, is this trying to solve a Meta fleet problem and you're thinking, why
> > >> not upstream it too?
> > >>
> > >> Because I fail to use the usefulness of this...
> > > I might be wrong, but I doubt that Meta's fleet is the only place where
> > > such bugs appear.  😉
> > 
> > Yeah, I once upon a time had a bunch of problems with NMIs taking
> > forever to run. In my case, it was some actually wonky firmware, but
> > differentiating the broken firmware from NMI handlers just being slow is
> > what motivated the existing "took too long to run" message next to which
> > you added the nmi_panic().
> 
> I guess I am glad that it is not just us seeing this sort of thing.
> 
> The existing "took too long to run" message is a good thing, but in
> our case we needed a global timeout and we needed it to be quite long,
> as in multiple *seconds*.  I did not see a nice way to integrate what we
> needed into the existing ->max_duration checks, perhaps due to a failure
> of imagination on my part.
> 
> > That said, I'd much prefer something more generic than something
> > NMI-specific.
> > 
> > Would the systems that set toolong_nmi_panic=1 also set panic_on_warn=1?
> > Are there any other "extra" panics that these systems also find useful?
> 
> We do often also set panic_on_warn=1 on our fleet, so in theory we
> could make this be a WARN_ON() instead of a direct call to nmi_panic().
> In practice, we would need to do something like make check_panic_on_warn()
> check in_nmi() and invoke nmi_panic() if so.  Other adjustments might
> be needed, I don't claim to be familiar with this code.

And following up on this one, setting panic_on_warn=1 everywhere we care
about too-long NMI handlers is a no-go.

> Or is there a better way?

So unless I hear (or think of) a better way, I will forward-port what
I have when -rc1 comes out.

							Thanx, Paul

> > I'm kinda curious if, for instance:
> > 
> > 	hung_task_panic=1
> > or
> > 	softlockup_panic=1
> > 
> > would also panic() these systems.
> 
> They did not do so on our systems because the NMI handlers ran for "only"
> a few tens of seconds, which is not long enough to trigger these panics.
> We did get the occasional RCU CPU stall warning at 21 seconds in, but
> these proved to be unhelpful in this case because the IPIs used by this
> stall-warning code are deferred until exit from the NMI.
> 
> 							Thanx, Paul