Re: [PATCH] kprobes: Fix to delay the kprobes jump optimization

[Paul E. McKenney]

On Mon, Feb 22, 2021 at 01:54:31PM +0100, Uladzislau Rezki wrote:
> On Mon, Feb 22, 2021 at 11:21:04AM +0100, Sebastian Andrzej Siewior wrote:
> > On 2021-02-19 10:33:36 [-0800], Paul E. McKenney wrote:
> > > For definiteness, here is the first part of the change, posted earlier.
> > > The commit log needs to be updated.  I will post the change that keeps
> > > the tick going as a reply to this email.
> > …
> > > diff --git a/kernel/softirq.c b/kernel/softirq.c
> > > index 9d71046..ba78e63 100644
> > > --- a/kernel/softirq.c
> > > +++ b/kernel/softirq.c
> > > @@ -209,7 +209,7 @@ static inline void invoke_softirq(void)
> > >  	if (ksoftirqd_running(local_softirq_pending()))
> > >  		return;
> > >  
> > > -	if (!force_irqthreads) {
> > > +	if (!force_irqthreads || !__this_cpu_read(ksoftirqd)) {
> > >  #ifdef CONFIG_HAVE_IRQ_EXIT_ON_IRQ_STACK
> > >  		/*
> > >  		 * We can safely execute softirq on the current stack if
> > > @@ -358,8 +358,8 @@ asmlinkage __visible void __softirq_entry __do_softirq(void)
> > >  
> > >  	pending = local_softirq_pending();
> > >  	if (pending) {
> > > -		if (time_before(jiffies, end) && !need_resched() &&
> > > -		    --max_restart)
> > > +		if (!__this_cpu_read(ksoftirqd) ||
> > > +		    (time_before(jiffies, end) && !need_resched() && --max_restart))
> > >  			goto restart;
> > 
> > This is hunk shouldn't be needed. The reason for it is probably that the
> > following wakeup_softirqd() would avoid further invoke_softirq()
> > performing the actual softirq work. It would leave early due to
> > ksoftirqd_running(). Unless I'm wrong, any raise_softirq() invocation
> > outside of an interrupt would do the same. 

And it does pass the rcutorture test without that hunk:

tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 2 --configs "TREE03" --kconfig "CONFIG_DEBUG_LOCK_ALLOC=y CONFIG_PROVE_LOCKING=y" --bootargs "threadirqs=1" --trust-make

> > I would like PeterZ / tglx to comment on this one. Basically I'm not
> > sure if it is okay to expect softirqs beeing served and waited on that
> > early in the boot.

It would be good to get other eyes on this.

I do agree that "don't wait on softirq handlers until after completion
of all early_initcall() handlers" is a nice simple rule, but debugging
violations of it is not so simple.  Adding warnings to ease debugging
of violations of this rule is quite a bit more complex than is either of
the methods of making the rule unnecessary, at least from what I can see
at this point.  The complexity of the warnings is exactly what Sebastian
pointed out earlier, that it is currently legal to raise_softirq() but
not to wait on the resulting handlers.  But even waiting is OK if that
waiting does not delay the boot sequence.  But if the boot kthread waits
on the kthread that does the waiting, it is once again not OK.

So am I missing something subtle here?

> The ksoftirqd threads get spawned during early_initcall() phase. Why not
> just spawn them one step earlier what is totally safe? I mean before
> do_pre_smp_initcalls() that calls early callbacks.
> 
> +       spawn_ksoftirqd();
>         rcu_init_tasks_generic();
>         do_pre_smp_initcalls();
> 
> With such change the spawning will not be depended on linker/compiler
> i.e. when and in which order an early_initcall(spawn_ksoftirqd) callback
> is executed.

We both posted patches similar to this, so I am not opposed.  One caveat,
though, namely that this narrows the window quite a bit but does not
entirely close it.  But it does allow the early_initcall()s to wait on
softirq handlers.

							Thanx, Paul