Re: [RFC 2/2] rcu: Remove ->dynticks_nmi_nesting from struct rcu_dynticks

From: Joel Fernandes
Date: Thu Jun 28 2018 - 17:13:24 EST


On Thu, Jun 28, 2018 at 01:02:05PM -0700, Paul E. McKenney wrote:
> > > <paulmck@xxxxxxxxxxxxxxxxxx> wrote:
[..]
> > > > > > > > So why this function-call structure? Well, you see, NMI handlers can
> > > > > > > > take what appear to RCU to be normal interrupts...
> > > > > > > >
> > > > > > > > (And I just added that fun fact to Requirements.html.)
> > > > > > >
> > > > > > > Yes, I'll definitely go through all the interrupt requirements in the doc and
> > > > > > > thanks for referring me to it.
> > > > > >
> > > > > > My concern may well be obsolete. It would be good if it was! ;-)
> > > > >
> > > > > I'd love to mandate that irq_enter() must be paired with irq_exit(). I
> > > > > don't really see any rationale for it to be otherwise. If there is a
> > > > > case, perhaps it needs to be fixed.
> > > >
> > > > Given that the usermode helpers now look to be common code using
> > > > workqueues, kthreads, and calls to do_execve(), it might well be that
> > > > the days of half-interrupts are behind us.
> > > >
> > > > But how to actually validate this? My offer of adding a WARN_ON_ONCE()
> > > > and waiting a few years still stands, but perhaps you have a better
> > > > approach.
> > >
> > > I think you should add a WARN_ON_ONCE(). Let's get the bugs fixed.
> >
> > Or the obscure features identified, as the case may be. ;-)
> >
> > Either way, will do!
>
> And here is a prototype patch.
>
> Thanx, Paul
>
> ------------------------------------------------------------------------
>
> commit ef544593a7bcad74628fa0537badc49dce1f2d95
> Author: Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx>
> Date: Thu Jun 28 12:45:23 2018 -0700
>
> rcu: Add warning to detect half-interrupts
>
> RCU's dyntick-idle code is written to tolerate half-interrupts, that it,
> either an interrupt that invokes rcu_irq_enter() but never invokes the
> corresponding rcu_irq_exit() on the one hand, or an interrupt that never
> invokes rcu_irq_enter() but does invoke the "corresponding" rcu_irq_exit()
> on the other. These things really did happen at one time, as evidenced
> by this ca-2011 LKML post:
>
> http://lkml.kernel.org/r/20111014170019.GE2428@xxxxxxxxxxxxxxxxxx
>
> The reason why RCU tolerates half-interrupts is that usermode helpers
> used exceptions to invoke a system call from within the kernel such that
> the system call did a normal return (not a return from exception) to
> the calling context. This caused rcu_irq_enter() to be invoked without
> a matching rcu_irq_exit(). However, usermode helpers have since been
> rewritten to make much more housebroken use of workqueues, kernel threads,
> and do_execve(), and therefore should no longer produce half-interrupts.
> No one knows of any other source of half-interrupts, but then again,
> no one seems insane enough to go audit the entire kernel to verify that
> half-interrupts really are a relic of the past.
>
> This commit therefore adds a pair of WARN_ON_ONCE() calls that will
> trigger in the presence of half interrupts, which the code will continue
> to handle correctly. If neither of these WARN_ON_ONCE() trigger by
> mid-2021, then perhaps RCU can stop handling half-interrupts, which
> would be a considerable simplification.
>
> Reported-by: Steven Rostedt <rostedt@xxxxxxxxxxx>
> Reported-by: Joel Fernandes <joel@xxxxxxxxxxxxxxxxx>
> Reported-by: Andy Lutomirski <luto@xxxxxxxxxx>
> Signed-off-by: Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx>

Looks good to me!

Reviewed-by: Joel Fernandes (Google) <joel@xxxxxxxxxxxxxxxxx>

thanks,

- Joel