Re: Should RCU_BOOST kernels use hrtimers in GP kthread?

[Sebastian Andrzej Siewior]

On 2021-02-17 07:54:47 [-0800], Paul E. McKenney wrote:
> > I though boosting is accomplished by acquiring a rt_mutex in a
> > rcu_read() section. Do you have some code to point me to, to see how a
> > timer is involved here? Or is it the timer saying that *now* boosting is
> > needed.
> 
> Yes, this last, which is in the grace-period kthread code, for example,
> in rcu_gp_fqs_loop().
>
> > If your hrtimer is a "normal" hrtimer then it will be served by
> > ksoftirqd, too. You would additionally need one of the
> > HRTIMER_MODE_*_HARD to make it work.
> 
> Good to know.  Anything I should worry about for this mode?

Well. It is always hardirq. No spinlock_t, etc. within that callback.
If you intend to wake a thread, that thread needs an elevated priority
otherwise it won't be scheduled (assuming there is a RT tasking running
which would block otherwise ksoftirqd).

Ah. One nice thing is that you can move the RCU threads to a house
keeping CPU - away from the CPU(s) running the RT tasks. Would this
scenario be still affected (if ksoftirqd would be blocked)?

Oh. One thing I forgot to mention: the timer_list timer is nice in terms
of moving forward (the timer did not fire, the condition is true and you
move the timeout forward).
A hrtimer timer on the other hand needs to be removed, forwarded and
added back to the "timer tree". This is considered more expensive
especially if the timer does not fire.

> Also, the current test expects callbacks to be invoked, which involves a
> number of additional kthreads and timers, for example, in nocb_gp_wait().
> I suppose I could instead look at grace-period sequence numbers, but I
> believe that real-life use cases needing RCU priority boosting also need
> the callbacks to be invoked reasonably quickly (as in within hundreds
> of milliseconds up through very small numbers of seconds).

A busy/overloaded kvm-host could lead to delays by not scheduling the
guest for a while.

My understanding of the need for RCU boosting is to get a task,
preempted (by a RT task) within a RCU section, back on the CPU to
at least close the RCU section. So it is possible to run RCU callbacks
and free memory.
The 10 seconds without RCU callbacks shouldn't be bad unless the OOM
killer got nervous (and if we had memory allocation failures).
Also, running thousands of accumulated callbacks isn't good either.

> Thoughts?
> 
> 							Thanx, Paul
Sebastian