Re: [PATCH 1/2] srcu: Fix broken node geometry after early ssp init

[Frederic Weisbecker]

On Wed, Apr 14, 2021 at 08:55:38AM -0700, Paul E. McKenney wrote:
> On Wed, Apr 14, 2021 at 03:24:12PM +0200, Frederic Weisbecker wrote:
> > diff --git a/kernel/rcu/rcu.h b/kernel/rcu/rcu.h
> > index 75ed367d5b60..24db97cbf76b 100644
> > --- a/kernel/rcu/rcu.h
> > +++ b/kernel/rcu/rcu.h
> > @@ -278,6 +278,7 @@ extern void resched_cpu(int cpu);
> >  extern int rcu_num_lvls;
> >  extern int num_rcu_lvl[];
> >  extern int rcu_num_nodes;
> > +extern bool rcu_geometry_initialized;
> 
> Can this be a static local variable inside rcu_init_geometry()?
> 
> After all, init_srcu_struct() isn't called all that often, and its overhead
> is such that an extra function call and check is going to hurt it.  This
> of course requires removing __init from rcu_init_geometry(), but it is not
> all that large, so why not just remove the __init?
> 
> But if we really are worried about reclaiming rcu_init_geometry()'s
> instructions (maybe we are?), then rcu_init_geometry() can be split
> into a function that just does the check (which is not __init) and the
> remainder of the function, which could remain __init.

Indeed that makes sense, I'll move the variable inside rcu_init_geometry().
Also since rcu_init_geometry() can now be called anytime after the boot, I
already removed the __init. I don't think we can do the split trick because a
non-init function can't call an __init function. That would trigger a section
mismatch.


> > @@ -171,6 +171,8 @@ static int init_srcu_struct_fields(struct srcu_struct *ssp, bool is_static)
> >  		ssp->sda = alloc_percpu(struct srcu_data);
> >  	if (!ssp->sda)
> >  		return -ENOMEM;
> > +	if (!rcu_geometry_initialized)
> > +		rcu_init_geometry();
> 
> With the suggested change above, this just becomes an unconditional call
> to rcu_init_geometry().

Right.

> > -static void __init rcu_init_geometry(void)
> > +void rcu_init_geometry(void)
> >  {
> >  	ulong d;
> >  	int i;
> > +	static unsigned long old_nr_cpu_ids;
> >  	int rcu_capacity[RCU_NUM_LVLS];
> 
> And then rcu_geometry_initialized is declared static here.
> 
> Or am I missing something?

Looks good, I'll resend with that.

Thanks!

> 
> > +	if (rcu_geometry_initialized) {
> > +		/*
> > +		 * Arrange for warning if rcu_init_geometry() was called before
> > +		 * setup_nr_cpu_ids(). We may miss cases when
> > +		 * nr_cpus_ids == NR_CPUS but that shouldn't matter too much.
> > +		 */
> > +		WARN_ON_ONCE(old_nr_cpu_ids != nr_cpu_ids);
> > +		return;
> > +	}
> > +
> > +	old_nr_cpu_ids = nr_cpu_ids;
> > +	rcu_geometry_initialized = true;
> > +
> >  	/*
> >  	 * Initialize any unspecified boot parameters.
> >  	 * The default values of jiffies_till_first_fqs and
> > -- 
> > 2.25.1
> >