Re: [RFCv7 PATCH 03/10] sched: scheduler-driven cpu frequency selection

[Rafael J. Wysocki]

On Thursday, February 25, 2016 10:28:37 AM Peter Zijlstra wrote:
> On Thu, Feb 25, 2016 at 04:55:57AM +0100, Rafael J. Wysocki wrote:
> > > +static void dummy(void *info) {}
> > > +
> > > +static int cpufreq_sched_stop(struct cpufreq_policy *policy)
> > > +{
> > > +	struct gov_data *gd = policy->governor_data;
> > > +	int cpu;
> > > +
> > > +	/*
> > > +	 * The schedfreq static key is managed here so the global schedfreq
> > > +	 * lock must be taken - a per-policy lock such as policy->rwsem is
> > > +	 * not sufficient.
> > > +	 */
> > > +	mutex_lock(&gov_enable_lock);
> > > +
> > > +	/*
> > > +	 * The governor stop path may or may not hold policy->rwsem. There
> > > +	 * must be synchronization with the slow path however.
> > > +	 */
> > > +	mutex_lock(&gd->slowpath_lock);
> > > +
> > > +	/*
> > > +	 * Stop new entries into the hot path for all CPUs. This will
> > > +	 * potentially affect other policies which are still running but
> > > +	 * this is an infrequent operation.
> > > +	 */
> > > +	static_key_slow_dec(&__sched_freq);
> > > +	enabled_policies--;
> > > +
> > > +	/*
> > > +	 * Ensure that all CPUs currently part of this policy are out
> > > +	 * of the hot path so that if this policy exits we can free gd.
> > > +	 */
> > > +	preempt_disable();
> > > +	smp_call_function_many(policy->cpus, dummy, NULL, true);
> > > +	preempt_enable();
> > 
> > I'm not sure how this works, can you please tell me?
> 
> I think it relies on the fact that rq->lock disables IRQs, so if we've
> managed to IPI all relevant CPUs, it means they cannot be inside a
> rq->lock section.
> 
> Its vile though; one should not spray IPIs if one can avoid it. Such
> things are much better done with RCU. Sure sync_sched() takes a little
> longer, but this isn't a fast path by any measure.

I see, thanks!

BTW, when cpufreq_update_util() callbacks are removed, I use synchronize_rcu()
to wait for the running ones, but would it be better to use synchronize_sched()
in there instead?

Thanks,
Rafael