Re: [RFC 0/3] Revert SRCU from tracepoint infrastructure

[Paul E. McKenney]

On Mon, Feb 10, 2020 at 10:46:16AM +0100, Peter Zijlstra wrote:
> On Sat, Feb 08, 2020 at 11:31:25AM -0500, Mathieu Desnoyers wrote:
> > ----- On Feb 7, 2020, at 3:56 PM, Joel Fernandes, Google joel@xxxxxxxxxxxxxxxxx wrote:
> > 
> > > Hi,
> > > These patches remove SRCU usage from tracepoints. The reason for proposing the
> > > reverts is because the whole point of SRCU was to avoid having to call
> > > rcu_irq_enter_irqson(). However this was added back in 865e63b04e9b2 ("tracing:
> > > Add back in rcu_irq_enter/exit_irqson() for rcuidle tracepoints") because perf
> > > was breaking..
> > 
> > I think the original patch re-enabling the rcu_irq_enter/exit_irqson() is a
> > tracepoint band-aid over what should actually been fixed within perf instead.
> > 
> > Perf should not do rcu_read_lock/unlock()/synchronize_rcu(), but rather use
> > tracepoint_synchronize_unregister() to match the read-side provided by
> > tracepoints.
> > 
> > If perf can then just rely on the underlying synchronization provided by each
> > instrumentation providers (tracepoint, kprobe, ...) and not explicitly add its own
> > unneeded synchronization on top (e.g. rcu_read_lock/unlock), then it should simplify
> > all this.
> 
> It can't. At this point it doesn't know where the event came from. Also,
> the whole perf stuff is per definition non-preemptible, as it needs to
> run from NMI context.
> 
> Furthermore, using srcu would be detrimental, because of how it has
> smp_mb() in the read side primitives.

Note that rcu_irq_enter() and rcu_irq_exit() also contain value-returning
atomics, which imply full memory barriers.

> The best we can do is move that rcu_irq_enter/exit_*() crud into the
> perf tracepoint glue I suppose.

One approach would be to define a synchronize_preempt_disable() that
waits only for pre-existing disabled-preemption regions (including
of course diabled-irq and NMI-handler regions.  Something like Steve
Rostedt's workqueue-baed schedule_on_each_cpu(ftrace_sync) implementation
might work.

There are of course some plusses and minuses:

+	Works on preempt-disable regions in idle-loop code without
	the need to invoke rcu_idle_exit() and rcu_idle_enter()..

+	Straightforward implementation.

-	Does not work on preempt-disable regions on offline CPUs.
	(I have no idea if this really matters.)

-	Schedules on idle CPUs, so usage needs to be restricted to
	avoid messing up energy-efficient systems.  (It should be
	just fine to use this for tracing.)

Thoughts?

							Thanx, Paul