Re: [PATCH RFC v2] rcu: Add a minimum time for marking boot as completed

[Joel Fernandes]

> On Feb 27, 2023, at 1:06 PM, Uladzislau Rezki <urezki@xxxxxxxxx> wrote:
> 
> On Mon, Feb 27, 2023 at 10:16:51AM -0500, Joel Fernandes wrote:
>>> On Mon, Feb 27, 2023 at 9:55 AM Paul E. McKenney <paulmck@xxxxxxxxxx> wrote:
>>> 
>>> On Mon, Feb 27, 2023 at 08:22:06AM -0500, Joel Fernandes wrote:
>>>> 
>>>> 
>>>>> On Feb 27, 2023, at 2:53 AM, Zhuo, Qiuxu <qiuxu.zhuo@xxxxxxxxx> wrote:
>>>>> 
>>>>> 
>>>>>> 
>>>>>> From: Joel Fernandes (Google) <joel@xxxxxxxxxxxxxxxxx>
>>>>>> Sent: Saturday, February 25, 2023 11:34 AM
>>>>>> To: linux-kernel@xxxxxxxxxxxxxxx
>>>>>> Cc: Joel Fernandes (Google) <joel@xxxxxxxxxxxxxxxxx>; Frederic Weisbecker
>>>>>> <frederic@xxxxxxxxxx>; Lai Jiangshan <jiangshanlai@xxxxxxxxx>; linux-
>>>>>> doc@xxxxxxxxxxxxxxx; Paul E. McKenney <paulmck@xxxxxxxxxx>;
>>>>>> rcu@xxxxxxxxxxxxxxx
>>>>>> Subject: [PATCH RFC v2] rcu: Add a minimum time for marking boot as
>>>>>> completed
>>>>>> 
>>>>>> On many systems, a great deal of boot happens after the kernel thinks the
>>>>>> boot has completed. It is difficult to determine if the system has really
>>>>>> booted from the kernel side. Some features like lazy-RCU can risk slowing
>>>>>> down boot time if, say, a callback has been added that the boot
>>>>>> synchronously depends on.
>>>>>> 
>>>>>> Further, it is better to boot systems which pass 'rcu_normal_after_boot' to
>>>>>> stay expedited for as long as the system is still booting.
>>>>>> 
>>>>>> For these reasons, this commit adds a config option
>>>>>> 'CONFIG_RCU_BOOT_END_DELAY' and a boot parameter
>>>>>> rcupdate.boot_end_delay.
>>>>>> 
>>>>>> By default, this value is 20s. A system designer can choose to specify a value
>>>>>> here to keep RCU from marking boot completion.  The boot sequence will not
>>>>>> be marked ended until at least boot_end_delay milliseconds have passed.
>>>>> 
>>>>> Hi Joel,
>>>>> 
>>>>> Just some thoughts on the default value of 20s, correct me if I'm wrong :-).
>>>>> 
>>>>> Does the OS with CONFIG_PREEMPT_RT=y kernel concern more about the
>>>>> real-time latency than the overall OS boot time?
>>>> 
>>>> But every system has to boot, even an RT system.
>>>> 
>>>>> 
>>>>> If so, we might make rcupdate.boot_end_delay = 0 as the default value
>>>>> (NOT the default 20s) for CONFIG_PREEMPT_RT=y kernels?
>>>> 
>>>> Could you measure how much time your RT system takes to boot before the application runs?
>>>> 
>>>> I can change it to default 0 essentially NOOPing it, but I would rather have a saner default (10 seconds even), than having someone forget to tune this for their system.
>>> 
>>> Provide a /sys location that the userspace code writes to when it
>>> is ready?  Different systems with different hardware and software
>>> configurations are going to take different amounts of time to boot,
>>> correct?
>> 
>> I could add a sysfs node, but I still wanted this patch as well
>> because I am wary of systems where yet more userspace changes are
>> required. I feel the kernel should itself be able to do this. Yes, it
>> is possible the system completes "booting" at a different time than
>> what the kernel thinks. But it does that anyway (even without this
>> patch), so I am not seeing a good reason to not do this in the kernel.
>> It is also only a minimum cap, so if the in-kernel boot takes too
>> long, then the patch will have no effect.
>> 
>> Thoughts?
>> 
> Why "rcu_boot_ended" is not enough? As i see right after that an "init"
> process or shell or panic is going to be invoked by the kernel. It basically
> indicates that a kernel is fully functional.
> 
> Or an idea to wait even further? Until all kernel modules are loaded by
> user space.

I mentioned in commit message it is daemons, userspace initialization etc. There is a lot of userspace booting up as well and using the kernel while doing so.

So, It does not make sense to me to mark kernel as booted too early. And no harm in adding some builtin kernel hysteresis. What am I missing?

Thanks,

- Joel
(And Steven please dont kick my butt for the lack of wrapping, I am working on fixing it :P).


> 
> Thanks!
> 
> --
> Uladzislau Rezki