Re: [PATCH RFC v2] rcu: Add a minimum time for marking boot as completed

[Joel Fernandes]

On Wed, Mar 1, 2023 at 12:11 PM Frederic Weisbecker <frederic@xxxxxxxxxx> wrote:
[...]
> > Hmmm I see what you mean, so a conservative and configurable "fail-safe"
> > timeout followed by sysctl to end the boot earlier than the timeout, should
> > do it (something like 30 seconds IMHO sounds reasonable)? In any case,
> > whatever way we go, we would not end the kernel boot before
> > rcu_end_inkernel_boot() is called at least once (which is the current
> > behavior).
> >
> > So it would be:
> >
> >   low level boot + initcalls
> >        20 sec                         30 second timeout
> > |------------------------------|--------------------------
> >                                |                         |
> >               old rcu_end_inkernel_boot()      new rcu_end_inkernel_boot()
> >
> > But it could be, if user decides:
> >   low level boot + initcalls
> >        20 sec                         10 second timeout
> > |------------------------------|--------------------------
> >                                |                         |
> >               old rcu_end_inkernel_boot()      new rcu_end_inkernel_boot()
> >                                                via /sys/ entry.
>
> The problem I have with a random default timeout is that it may break sensitive
> workloads. If the default is 30 and say the boot only takes 5 seconds and
> immediately launches a latency sensitive task, this may break things in a
> subtle way during these 25 seconds when it usually didn't. Because expedited
> RCU is a hammer interrupting all non-idle CPUs.
>
> Until now forcing expedited RCU was only performed before any user code. Now it
> crosses the boundary so better be careful. I'd personally advocate for keeping
> the current call before init is launched. Then provide an end_boot_sysctl kernel
> boot parameter that will ignore the current call before init and let the user
> signal that through sysctl.

Considering that the PREEMPT-RT system benefits from it within the 8
seconds, I will go ahead make the default 15 seconds or so and make it
tunable. Hopefully that will be an acceptable compromise, with
sufficient documentation, changelog, and so forth... If you agree I'd
appreciate your Ack on the next posting.

> > > > > So shouldn't we disable lazy callbacks by default when CONFIG_RCU_LAZY=y and then
> > > > > turn it on with "sysctl kernel.rcu.lazy=1" only whenever userspace feels ready
> > > > > about it? We can still keep the current call to rcu_end_inkernel_boot().
> > > >
> > > > Hmm IMHO that would add more knobs for not much reason honestly. We already
> > > > have CONFIG_RCU_LAZY default disabled, I really don't want to add more
> > > > dependency (like user enables the config and does not see laziness).
> > >
> > > I don't know. Like I said, different problems, different solutions. Let's
> > > identify what the issue is precisely. For example can we expect that the issues
> > > on boot can be a problem also on some temporary workloads?
> > >
> > > Besides I'm currently testing a very hacky flavour of rcu_lazy and so far it
> > > shows many idle calls that would have been delayed if callbacks weren't queued
> > > as lazy.
> >
> > Can you provide more details? What kind of hack flavor, and what is it doing?
>
> Sorry, I meant a hacky implementation of lazy to work with !NOCB.

Interesting, I'm curious which calls those are and if they should also
be call_rcu_hurry().

 - Joel