Re: Question on comment header for for_each_domain()

[Peter Zijlstra]

On Thu, Nov 08, 2018 at 07:31:09AM -0800, Paul E. McKenney wrote:
> On Thu, Nov 08, 2018 at 10:21:51AM +0100, Peter Zijlstra wrote:
> > On Wed, Nov 07, 2018 at 03:00:02PM -0800, Paul E. McKenney wrote:
> > > Hello!
> > > 
> > > The header comment for for_each_domain() talks about a call to
> > > synchronize_sched() within detach_destroy_domains(), but I am not
> > > seeing any such call.  Because synchronize_sched() is now folded into
> > > synchronize_rcu(), I have a patch that edits the comment, but it looks
> > > like a larger change is needed.
> > > 
> > > Or am I blind today?
> > 
> > I think you're quite right and that comment is a wee bit stale.
> > 
> > The sched domain tree is indeed protected by regular RCU (not RCU-sched
> > as the comment seems to imply) and this is per destroy_sched_domains()
> > using call_rcu().
> > 
> > And most (I didn't look at all) uses for the sched-domain tree do indeed
> > employ rcu_read_lock().
> 
> Ah, thank you for the info!  Would this patch do the trick?

> diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
> index 618577fc9aa8..00b91d16af9f 100644
> --- a/kernel/sched/sched.h
> +++ b/kernel/sched/sched.h
> @@ -1237,7 +1237,7 @@ extern void sched_ttwu_pending(void);
>  
>  /*
>   * The domain tree (rq->sd) is protected by RCU's quiescent state transition.
> - * See detach_destroy_domains: synchronize_sched for details.
> + * See destroy_sched_domains: call_rcu for details.
>   *
>   * The domain tree of any CPU may only be accessed from within
>   * preempt-disabled sections.
> diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
> index 8d7f15ba5916..04d458faf2c1 100644
> --- a/kernel/sched/topology.c
> +++ b/kernel/sched/topology.c
> @@ -248,7 +248,7 @@ void rq_attach_root(struct rq *rq, struct root_domain *rd)
>  	raw_spin_unlock_irqrestore(&rq->lock, flags);
>  
>  	if (old_rd)
> -		call_rcu_sched(&old_rd->rcu, free_rootdomain);
> +		call_rcu(&old_rd->rcu, free_rootdomain);
>  }
>  
>  void sched_get_rd(struct root_domain *rd)
> @@ -261,7 +261,7 @@ void sched_put_rd(struct root_domain *rd)
>  	if (!atomic_dec_and_test(&rd->refcount))
>  		return;
>  
> -	call_rcu_sched(&rd->rcu, free_rootdomain);
> +	call_rcu(&rd->rcu, free_rootdomain);
>  }
>  
>  static int init_rootdomain(struct root_domain *rd)
> 

Argh, that is the rootdomain, not the regular sched-domain tree. Now
I'll have to go audit that stuff again.

ISTR there being slightly different rules for rootdomain, and with a
reason.

Can we hold onto this until after LPC?