Re: Query regarding synchronize_sched_expedited and resched_cpu

[Peter Zijlstra]

On Thu, Sep 21, 2017 at 09:00:48AM -0700, Paul E. McKenney wrote:
> commit c21c9b78182e35eb0e72ef4e3bba3054f26eaaea
> Author: Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx>
> Date:   Mon Sep 18 08:54:40 2017 -0700
> 
>     sched: Make resched_cpu() unconditional
>     
>     The current implementation of synchronize_sched_expedited() incorrectly
>     assumes that resched_cpu() is unconditional, which it is not.  This means
>     that synchronize_sched_expedited() can hang when resched_cpu()'s trylock
>     fails as follows (analysis by Neeraj Upadhyay):
>     
>     o	CPU1 is waiting for expedited wait to complete:
>     
>     	sync_rcu_exp_select_cpus
>     	     rdp->exp_dynticks_snap & 0x1   // returns 1 for CPU5
>     	     IPI sent to CPU5
>     
>     	synchronize_sched_expedited_wait
>     		 ret = swait_event_timeout(rsp->expedited_wq,
>     					   sync_rcu_preempt_exp_done(rnp_root),
>     					   jiffies_stall);
>     
>     	expmask = 0x20, CPU 5 in idle path (in cpuidle_enter())
>     
>     o	CPU5 handles IPI and fails to acquire rq lock.
>     
>     	Handles IPI
>     	     sync_sched_exp_handler
>     		 resched_cpu
>     		     returns while failing to try lock acquire rq->lock
>     		 need_resched is not set
>     
>     o    CPU5 calls  rcu_idle_enter() and as need_resched is not set, goes to
>          idle (schedule() is not called).
>     
>     o    CPU 1 reports RCU stall.


Inconsistent spacing after your bullet 'o', first two points have a
space the last two a tab or so.

>     Given that resched_cpu() is now used only by RCU, this commit fixes the
>     assumption by making resched_cpu() unconditional.

Other than that, yes looks _much_ better, thanks!

Acked-by: Peter Zijlstra (Intel) <peterz@xxxxxxxxxxxxx>

Also, you might want to tag it for stable.