Re: [RFC 2/2] rcu: Remove ->dynticks_nmi_nesting from struct rcu_dynticks

[Joel Fernandes]

Hi Paul,

On Wed, Jun 20, 2018 at 09:49:02AM -0700, Paul E. McKenney wrote:
> On Thu, Jun 21, 2018 at 01:05:22AM +0900, Byungchul Park wrote:
> > On Wed, Jun 20, 2018 at 11:58 PM, Paul E. McKenney
> > <paulmck@xxxxxxxxxxxxxxxxxx> wrote:
> > > On Wed, Jun 20, 2018 at 05:47:20PM +0900, Byungchul Park wrote:
> > >> Hello folks,
> > >>
> > >> I'm careful in saying that ->dynticks_nmi_nesting can be removed but I
> > >> think it's possible since the only thing we are interested in with
> > >> regard to ->dynticks_nesting or ->dynticks_nmi_nesting is whether rcu is
> > >> idle or not.
> > >
> > > Please keep in mind that NMIs cannot be masked, which means that the
> > > rcu_nmi_enter() and rcu_nmi_exit() pair can be invoked at any point in
> > > the process, between any consecutive pair of instructions.  The saving
> 
> And yes, I should have looked at this patch more closely before replying.
> But please see below.
> 
> > I believe I understand what NMI is and why you introduced
> > ->dynticks_nmi_nesting. Or am I missing something?
> 
> Perhaps the fact that there are architectures that can enter interrupt
> handlers and never leave them when the CPU is non-idle.  One example of
> this is the usermode upcalls in the comment that you removed.

I spent some time tonight and last night trying to understand this concept of
never leaving an interrupt, I hope you don't mind me asking this dumb
question... perhaps I will learn something : Could you let me know how is it
possible that an interrupt never exits?

Typically an interrupt never exiting sounds like a hard-lockup. This is how
hardlock detector works: Since regular interrupts in linux can't nest, the
hardlockup detector checks if hrtimer interrupts are being handled and if
not, then it throws a splat, panics the kernel etc. So I am a bit troubled by
this interrupt never exiting concept..

Further since an interrupt is an atomic context, it cannot sleep or schedule
into usermode so how are these upcalls handled from the interrupt?

Lastly, can you point me to an example how the rcu_nmi_enter/exit() pair can go
out sync? That is they aren't paired and nested properly? In my mind they
always should be but I may be missing the usecase. I'm happy to try and
reproduce and trace this if you can let me know how to so that I can study
it better. 

Thanks a lot Paul for your help,

 - Joel