Re: [PATCH sched/core] sched/rt: Fix RT_PUSH_IPI soft lockup loop

[Steven Rostedt]

On Thu, 14 May 2026 11:15:31 -1000
Tejun Heo <tj@xxxxxxxxxx> wrote:

> Hello, Steven.
> 
> On Thu, May 14, 2026 at 10:03:00AM -0400, Steven Rostedt wrote:
> > I was thinking about this more and does disabling the RT_PUSH_IPI cause any
> > problems for you?
> > 
> >   # echo NO_RT_PUSH_IPI > /sys/kernel/debug/sched/features  
> 
> Not at all. This is actually the mitigation that we deployed across the
> affected machines.
> 
> ...
> > -/* RT IPI pull logic requires IRQ_WORK */
> > -#if defined(CONFIG_IRQ_WORK) && defined(CONFIG_SMP)
> > +/*
> > + * RT IPI pull logic requires IRQ_WORK and doesn't make sense for uniprocessors.
> > + * If CONFIG_IRQ_FORCED_THREADING isn't set, then softirqs do not run as threads
> > + * and can cause latency larger than what RT_PUSH_IPI can save, killing the
> > + * effect of it.
> > + */
> > +#if defined(CONFIG_IRQ_WORK) && defined(CONFIG_SMP) &&	\
> > +	defined(CONFIG_IRQ_FORCED_THREADING)
> >  # define HAVE_RT_PUSH_IPI
> >  #endif  
> 
> Maybe it should trigger on force_irqthreads so that it's active only when
> irq threads are actully enabled.

Well, PREEMPT_RT doesn't need force_irqthreads for this to be enabled.
But I could keep this configured like the above, but have the feature to
be disabled on boot up if !PREEMPT_RT and force_irqthreads is not set.

> 
> Whichever way it's done tho, wouldn't this still leave machines in that
> config susceptible to IPI storms? It took a combination of factors to
> trigger - mpi3mr's threaded irq, psimon activated by systemd, and sustained
> network load - but those factors are not that exotic.

With softirqs as threads it is highly unlikely to be a problem. The
reason you saw this was because the break out to schedule happened in a
softirq that prevented scheduling from occurring right away. With irqs
as threads, so are softirqs, and they wouldn't be able to cause the
delay in scheduling that you were experiencing.

I'll write up a patch tomorrow or next week.

Thanks!

-- Steve