Re: [PATCH v3 10/14] sched: Introduce arch_cpu_allowed_mask() to limit fallback rq selection

[Will Deacon]

On Thu, Nov 19, 2020 at 11:07:09AM +0000, Will Deacon wrote:
> On Thu, Nov 19, 2020 at 09:38:50AM +0000, Quentin Perret wrote:
> > On Friday 13 Nov 2020 at 09:37:15 (+0000), Will Deacon wrote:
> > > Asymmetric systems may not offer the same level of userspace ISA support
> > > across all CPUs, meaning that some applications cannot be executed by
> > > some CPUs. As a concrete example, upcoming arm64 big.LITTLE designs do
> > > not feature support for 32-bit applications on both clusters.
> > > 
> > > On such a system, we must take care not to migrate a task to an
> > > unsupported CPU when forcefully moving tasks in select_fallback_rq()
> > > in response to a CPU hot-unplug operation.
> > > 
> > > Introduce an arch_cpu_allowed_mask() hook which, given a task argument,
> > > allows an architecture to return a cpumask of CPUs that are capable of
> > > executing that task. The default implementation returns the
> > > cpu_possible_mask, since sane machines do not suffer from per-cpu ISA
> > > limitations that affect scheduling. The new mask is used when selecting
> > > the fallback runqueue as a last resort before forcing a migration to the
> > > first active CPU.
> > > 
> > > Signed-off-by: Will Deacon <will@xxxxxxxxxx>
> > > ---
> > >  kernel/sched/core.c | 13 ++++++++++---
> > >  1 file changed, 10 insertions(+), 3 deletions(-)
> > > 
> > > diff --git a/kernel/sched/core.c b/kernel/sched/core.c
> > > index 818c8f7bdf2a..8df38ebfe769 100644
> > > --- a/kernel/sched/core.c
> > > +++ b/kernel/sched/core.c
> > > @@ -1696,6 +1696,11 @@ void check_preempt_curr(struct rq *rq, struct task_struct *p, int flags)
> > >  
> > >  #ifdef CONFIG_SMP
> > >  
> > > +/* Must contain at least one active CPU */
> > > +#ifndef arch_cpu_allowed_mask
> > > +#define  arch_cpu_allowed_mask(p)	cpu_possible_mask
> > > +#endif
> > > +
> > >  /*
> > >   * Per-CPU kthreads are allowed to run on !active && online CPUs, see
> > >   * __set_cpus_allowed_ptr() and select_fallback_rq().
> > > @@ -1708,7 +1713,10 @@ static inline bool is_cpu_allowed(struct task_struct *p, int cpu)
> > >  	if (is_per_cpu_kthread(p))
> > >  		return cpu_online(cpu);
> > >  
> > > -	return cpu_active(cpu);
> > > +	if (!cpu_active(cpu))
> > > +		return false;
> > > +
> > > +	return cpumask_test_cpu(cpu, arch_cpu_allowed_mask(p));
> > >  }
> > >  
> > >  /*
> > > @@ -2361,10 +2369,9 @@ static int select_fallback_rq(int cpu, struct task_struct *p)
> > >  			}
> > >  			fallthrough;
> > >  		case possible:
> > > -			do_set_cpus_allowed(p, cpu_possible_mask);
> > > +			do_set_cpus_allowed(p, arch_cpu_allowed_mask(p));
> > 
> > Nit: I'm wondering if this should be called arch_cpu_possible_mask()
> > instead?
> 
> I'm open to renaming it, so if nobody else has any better ideas then I'll
> go with this.

Ah, so in doing this I realised I don't like arch_cpu_possible_mask() so
much because it makes it sound like a back-end to cpu_possible_mask, but
the two are really different things.

arch_task_cpu_possible_mask() might work?

Will