Re: [PATCH] kprobes: Fix to delay the kprobes jump optimization

[Uladzislau Rezki]

On Mon, Feb 22, 2021 at 07:09:03AM -0800, Paul E. McKenney wrote:
> On Mon, Feb 22, 2021 at 01:54:31PM +0100, Uladzislau Rezki wrote:
> > On Mon, Feb 22, 2021 at 11:21:04AM +0100, Sebastian Andrzej Siewior wrote:
> > > On 2021-02-19 10:33:36 [-0800], Paul E. McKenney wrote:
> > > > For definiteness, here is the first part of the change, posted earlier.
> > > > The commit log needs to be updated.  I will post the change that keeps
> > > > the tick going as a reply to this email.
> > > …
> > > > diff --git a/kernel/softirq.c b/kernel/softirq.c
> > > > index 9d71046..ba78e63 100644
> > > > --- a/kernel/softirq.c
> > > > +++ b/kernel/softirq.c
> > > > @@ -209,7 +209,7 @@ static inline void invoke_softirq(void)
> > > >  	if (ksoftirqd_running(local_softirq_pending()))
> > > >  		return;
> > > >  
> > > > -	if (!force_irqthreads) {
> > > > +	if (!force_irqthreads || !__this_cpu_read(ksoftirqd)) {
> > > >  #ifdef CONFIG_HAVE_IRQ_EXIT_ON_IRQ_STACK
> > > >  		/*
> > > >  		 * We can safely execute softirq on the current stack if
> > > > @@ -358,8 +358,8 @@ asmlinkage __visible void __softirq_entry __do_softirq(void)
> > > >  
> > > >  	pending = local_softirq_pending();
> > > >  	if (pending) {
> > > > -		if (time_before(jiffies, end) && !need_resched() &&
> > > > -		    --max_restart)
> > > > +		if (!__this_cpu_read(ksoftirqd) ||
> > > > +		    (time_before(jiffies, end) && !need_resched() && --max_restart))
> > > >  			goto restart;
> > > 
> > > This is hunk shouldn't be needed. The reason for it is probably that the
> > > following wakeup_softirqd() would avoid further invoke_softirq()
> > > performing the actual softirq work. It would leave early due to
> > > ksoftirqd_running(). Unless I'm wrong, any raise_softirq() invocation
> > > outside of an interrupt would do the same. 
> 
> And it does pass the rcutorture test without that hunk:
> 
> tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 2 --configs "TREE03" --kconfig "CONFIG_DEBUG_LOCK_ALLOC=y CONFIG_PROVE_LOCKING=y" --bootargs "threadirqs=1" --trust-make
> 
Yep. I have tested that patch also. It works for me as well. So
technically i do not see any issues from the first glance but of
course it should be reviewed by the softirq people to hear their
opinion.

IRQs are enabled, so it can be handled from an IRQ tail until
ksoftirqd threads are spawned.

> > > I would like PeterZ / tglx to comment on this one. Basically I'm not
> > > sure if it is okay to expect softirqs beeing served and waited on that
> > > early in the boot.
> 
> It would be good to get other eyes on this.
> 
> I do agree that "don't wait on softirq handlers until after completion
> of all early_initcall() handlers" is a nice simple rule, but debugging
> violations of it is not so simple.  Adding warnings to ease debugging
> of violations of this rule is quite a bit more complex than is either of
> the methods of making the rule unnecessary, at least from what I can see
> at this point.  The complexity of the warnings is exactly what Sebastian
> pointed out earlier, that it is currently legal to raise_softirq() but
> not to wait on the resulting handlers.  But even waiting is OK if that
> waiting does not delay the boot sequence.  But if the boot kthread waits
> on the kthread that does the waiting, it is once again not OK.
> 
> So am I missing something subtle here?
>
I agree here. Seems like we are on the same page in understanding :)

> > The ksoftirqd threads get spawned during early_initcall() phase. Why not
> > just spawn them one step earlier what is totally safe? I mean before
> > do_pre_smp_initcalls() that calls early callbacks.
> > 
> > +       spawn_ksoftirqd();
> >         rcu_init_tasks_generic();
> >         do_pre_smp_initcalls();
> > 
> > With such change the spawning will not be depended on linker/compiler
> > i.e. when and in which order an early_initcall(spawn_ksoftirqd) callback
> > is executed.
> 
> We both posted patches similar to this, so I am not opposed.  One caveat,
> though, namely that this narrows the window quite a bit but does not
> entirely close it.  But it does allow the early_initcall()s to wait on
> softirq handlers.
> 
Yep, that was an intention. At least to provide such functionality for early
callbacks. What happens before it(init/main.c) is pretty controllable.

--
Vlad Rezki