Re: [PATCH RESEND] sched: prefer an idle cpu vs an idle sibling for BALANCE_WAKE

[Josef Bacik]

On 06/11/2015 10:35 PM, Mike Galbraith wrote:
> On Thu, 2015-05-28 at 13:05 +0200, Peter Zijlstra wrote:
>
> > @@ -5022,22 +5026,28 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int sd_flag, int wake_f
> >   		 * If both cpu and prev_cpu are part of this domain,
> >   		 * cpu is a valid SD_WAKE_AFFINE target.
> >   		 */
> > -		if (want_affine && (tmp->flags & SD_WAKE_AFFINE) &&
> > -		    cpumask_test_cpu(prev_cpu, sched_domain_span(tmp))) {
> > +		if (want_affine && !affine_sd &&
> > +		    (tmp->flags & SD_WAKE_AFFINE) &&
> > +		    cpumask_test_cpu(prev_cpu, sched_domain_span(tmp)))
> >   			affine_sd = tmp;
> > -			break;
> > -		}
> >
> >   		if (tmp->flags & sd_flag)
> >   			sd = tmp;
> > +		else if (!want_affine || (want_affine && affine_sd))
> > +			break;
> >   	}
>
> Hm, new_cpu == cpu.
>
> > -	if (affine_sd && cpu != prev_cpu && wake_affine(affine_sd, p, sync))
> > +	if (affine_sd && cpu != prev_cpu && wake_affine(affine_sd, p, sync)) {
> >   		prev_cpu = cpu;
> > +		sd = NULL; /* WAKE_AFFINE trumps BALANCE_WAKE */
> > +	}
>
> If branch above is not taken, new_cpu remains cpu.
>
> >   	if (sd_flag & SD_BALANCE_WAKE) {
> > -		new_cpu = select_idle_sibling(p, prev_cpu);
> > -		goto unlock;
> > +		int tmp = select_idle_sibling(p, prev_cpu);
> > +		if (tmp >= 0) {
> > +			new_cpu = tmp;
> > +			goto unlock;
> > +		}
> >   	}
>
> If select_idle_sibling() returns -1, new_cpu remains cpu.
>
> >   	while (sd) {
>
> If sd == NULL, we fall through and try to pull wakee despite nacked-by
> tsk_cpus_allowed() or wake_affine().

So maybe add a check in the if (sd_flag & SD_BALANCE_WAKE) for something 
like this

if (tmp >= 0) {
	new_cpu = tmp;
	goto unlock;
} else if (!want_affine) {
	new_cpu = prev_cpu;
}

so we can make sure we're not being pushed onto a cpu that we aren't 
allowed on?  Thanks,

Josef

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/