Re: [PATCH] tracing/perf: Move rcu_irq_enter/exit_irqson() to perf trace point hook

[Peter Zijlstra]

On Tue, Feb 11, 2020 at 09:05:03AM -0500, Steven Rostedt wrote:
> On Tue, 11 Feb 2020 12:49:54 +0100
> Peter Zijlstra <peterz@xxxxxxxxxxxxx> wrote:
> 
> > On Mon, Feb 10, 2020 at 05:06:43PM -0500, Steven Rostedt wrote:
> > > +	if (!rcu_watching) {						\
> > > +		/* Can not use RCU if rcu is not watching and in NMI */	\
> > > +		if (in_nmi())						\
> > > +			return;						\
> > > +		rcu_irq_enter_irqson();					\
> > > +	}								\  
> > 
> > I saw the same weirdness in __trace_stack(), and I'm confused by it.
> > 
> > How can we ever get to: in_nmi() && !rcu_watching() ? That should be a
> > BUG.  In particular, nmi_enter() has rcu_nmi_enter().
> > 
> > Paul, can that really happen?
> 
> The stack tracer connects to the function tracer and is called at all
> the places that function tracing can be called from. As I like being
> able to trace RCU internal functions (especially as they are complex),
> I don't want to set them all to notrace. But, for callbacks that
> require RCU to be watching, we need this check, because there's some
> internal state that we can be in an NMI and RCU is not watching (as
> there's some places in nmi_enter that can be traced!).
> 
> And if we are tracing preempt_enable and preempt_disable (as Joel added
> trace events there), it may be the case for trace events too.

Bloody hell; what a trainwreck. Luckily there's comments around that
explain this!

So we haz:

| #define nmi_enter()						\
| 	do {							\
| 		arch_nmi_enter();				\

arm64 only, lets ignore for now

| 		printk_nmi_enter();				\

notrace

| 		lockdep_off();					\

notrace

| 		ftrace_nmi_enter();				\

!notrace !!!

| 		BUG_ON(in_nmi());				\
| 		preempt_count_add(NMI_OFFSET + HARDIRQ_OFFSET);\

lets make this __preempt_count_add() ASAP !

| 		rcu_nmi_enter();				\

are you _really_ sure you want to go trace that ?!?

| 		trace_hardirq_enter();				\
| 	} while (0)