Re: [PATCH V4 1/2] sched/fair: Fix load_balance() affinity redo path

[Jeffrey Hugo]

On 6/6/2017 6:30 AM, Peter Zijlstra wrote:
> On Fri, Jun 02, 2017 at 04:27:11PM -0600, Jeffrey Hugo wrote:
> > If load_balance() fails to migrate any tasks because all tasks were
> > affined, load_balance() removes the source cpu from consideration and
> > attempts to redo and balance among the new subset of cpus.
> >
> > There is a bug in this code path where the algorithm considers all active
> > cpus in the system (minus the source that was just masked out).  This is
> > not valid for two reasons: some active cpus may not be in the current
> > scheduling domain and one of the active cpus is dst_cpu. These cpus should
> > not be considered, as we cannot pull load from them.
> >
> > Instead of failing out of load_balance(), we may end up redoing the search
> > with no valid cpus and incorrectly concluding the domain is balanced.
> > Additionally, if the group_imbalance flag was just set, it may also be
> > incorrectly unset, thus the flag will not be seen by other cpus in future
> > load_balance() runs as that algorithm intends.
> >
> > Fix the check by removing cpus not in the current domain and the dst_cpu
> > from considertation, thus limiting the evaluation to valid remaining cpus
> > from which load might be migrated.
> >
> > Co-authored-by: Austin Christ <austinwc@xxxxxxxxxxxxxx>
> > Co-authored-by: Dietmar Eggemann <dietmar.eggemann@xxxxxxx>
> > Signed-off-by: Jeffrey Hugo <jhugo@xxxxxxxxxxxxxx>
> > Tested-by: Tyler Baicar <tbaicar@xxxxxxxxxxxxxx>
> > ---
> >   kernel/sched/fair.c | 22 ++++++++++------------
> >   1 file changed, 10 insertions(+), 12 deletions(-)
> >
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index d711093..84255ab 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -6737,10 +6737,10 @@ int can_migrate_task(struct task_struct *p, struct lb_env *env)
> >   		 * our sched_group. We may want to revisit it if we couldn't
> >   		 * meet load balance goals by pulling other tasks on src_cpu.
> >   		 *
> > -		 * Also avoid computing new_dst_cpu if we have already computed
> > -		 * one in current iteration.
> > +		 * Avoid computing new_dst_cpu for NEWLY_IDLE or if we have
> > +		 * already computed one in current iteration.
> >   		 */
> > -		if (!env->dst_grpmask || (env->flags & LBF_DST_PINNED))
> > +		if (env->idle == CPU_NEWLY_IDLE || (env->flags & LBF_DST_PINNED))
> >   			return 0;
> >   
> >   		/* Prevent to re-select dst_cpu via env's cpus */
> > @@ -8091,14 +8091,7 @@ static int load_balance(int this_cpu, struct rq *this_rq,
> >   		.tasks		= LIST_HEAD_INIT(env.tasks),
> >   	};
> >   
> > -	/*
> > -	 * For NEWLY_IDLE load_balancing, we don't need to consider
> > -	 * other cpus in our group
> > -	 */
> > -	if (idle == CPU_NEWLY_IDLE)
> > -		env.dst_grpmask = NULL;
> > -
> > -	cpumask_copy(cpus, cpu_active_mask);
> > +	cpumask_and(cpus, sched_domain_span(sd), cpu_active_mask);
> >   
> >   	schedstat_inc(sd->lb_count[idle]);
> >   
> > @@ -8220,7 +8213,12 @@ static int load_balance(int this_cpu, struct rq *this_rq,
> >   		/* All tasks on this runqueue were pinned by CPU affinity */
> >   		if (unlikely(env.flags & LBF_ALL_PINNED)) {
> >   			cpumask_clear_cpu(cpu_of(busiest), cpus);
> > -			if (!cpumask_empty(cpus)) {
> > +			/*
> > +			 * Go back to "redo" iff the load-balance cpumask
> > +			 * contains other potential busiest cpus for the
> > +			 * current sched domain.
> > +			 */
> > +			if (!cpumask_subset(cpus, env.dst_grpmask)) {
> >   				env.loop = 0;
> >   				env.loop_break = sched_nr_migrate_break;
> >   				goto redo;
>
> So I was struggling with that subset condition. You want to ensure there
> are CPUs outside of dst_grpmask left, otherwise balancing at this SD
> level doesn't make sense anymore, right?

Correct.

> I think you might want to spell that out a little in that comment.
> Currently the comment only explains what it does, which is something we
> can read from the code. Comments should explain _why_ we do things and
> its failing there.

Agreed.  We will attempt to improve the comment in the next version.

> So with that the problem is that active_load_balance_cpu_stop() calls
> into can_migrate_task() with ->idle = CPU_IDLE and !dst_grpmask, which
> then goes *bang*. Now active_load_balance_cpu_stop() doesn't need to
> re-evaluate anything, so ideally it would just skip this entirely,
> right?

Correct.  Your #3 option below would also work based on our 
understanding, and seems cleaner.  We will validate it as a solution, 
and incorporate into the next version.

Thanks

> So why not do #3:
>
> ---
>   kernel/sched/fair.c | 7 +++++++
>   1 file changed, 7 insertions(+)
>
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 47a0c552c77b..fd639d32fa4c 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -8523,6 +8523,13 @@ static int active_load_balance_cpu_stop(void *data)
>   			.src_cpu	= busiest_rq->cpu,
>   			.src_rq		= busiest_rq,
>   			.idle		= CPU_IDLE,
> +			/*
> +			 * can_migrate_task() doesn't need to compute new_dst_cpu
> +			 * for active balancing. Since we have CPU_IDLE, but no
> +			 * @dst_grpmask we need to make that test go away with lying
> +			 * about DST_PINNED.
> +			 */
> +			.flags		= LBF_DST_PINNED,
>   		};
>   
>   		schedstat_inc(sd->alb_count);


--
Jeffrey Hugo
Qualcomm Datacenter Technologies as an affiliate of Qualcomm 
Technologies, Inc.
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.