Re: [PATCH] sched, fair: try to prevent migration thread from preempting non-cfs task

[Vincent Guittot]

On Tue, 15 Jun 2021 at 16:55, Valentin Schneider
<valentin.schneider@xxxxxxx> wrote:
>
>
> Hi,
>
> On 15/06/21 20:15, Yafang Shao wrote:
>
> > - Prev version
> >   https://lore.kernel.org/lkml/CAKfTPtBd349eyDhA5ThCAHFd83cGMQKb_LDxD4QvyP-cJOBjqA@xxxxxxxxxxxxxx/
> >
> > - Similar discussion
> >   https://lore.kernel.org/lkml/CAKfTPtBygNcVewbb0GQOP5xxO96am3YeTZNP5dK9BxKHJJAL-g@xxxxxxxxxxxxxx/
>
> I knew that sounded familiar :-)
>
> > ---
> >  kernel/sched/fair.c | 14 ++++++++++++++
> >  1 file changed, 14 insertions(+)
> >
> > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > index 3248e24a90b0..597c7a940746 100644
> > --- a/kernel/sched/fair.c
> > +++ b/kernel/sched/fair.c
> > @@ -9797,6 +9797,20 @@ static int load_balance(int this_cpu, struct rq *this_rq,
> >                       /* Record that we found at least one task that could run on this_cpu */
> >                       env.flags &= ~LBF_ALL_PINNED;
> >
> > +                     /*
> > +                      * There may be a race between load balance starting migration
> > +                      * thread to pull the cfs running thread and the RT thread
> > +                      * waking up and preempting cfs task before migration threads
> > +                      * which then preempt the RT thread.
> > +                      * We'd better do the last minute check before starting
> > +                      * migration thread to avoid preempting latency-sensitive thread.
> > +                      */
>
> This can be summarized as in the below, no?
>
>                         /*
>                          * Don't cause a higher-than-CFS task to be preempted by
>                          * the CPU stopper.
>                          */

IMO, it's worth keeping the explanation that we are there because:
- a CFS task that was running during the 1st step :  if
(busiest->nr_running > 1) { ...
so we didn't pull the task
- but it has been preempted while lb was deciding if it needs an
active load balance

so maybe something like:
                         /*
                          * Don't kick the active_load_balance_cpu_stop,
                          * if the CFS task has been preempted by higher
                          * priority task in the meantime.
                         */


>
> > +                     if (busiest->curr->sched_class != &fair_sched_class) {
> > +                             raw_spin_unlock_irqrestore(&busiest->lock,
> > +                                                        flags);
> > +                             goto out;
>
> Since you goto out this could be moved before the
>
>   env.flags &= ~LBF_ALL_PINNED;
>
> above (it only has an impact if you'd goto out_balanced).

Good point. My comment to move this test after  env.flags &=
~LBF_ALL_PINNED; was valid only with goto out_one_pinned

>
> > +                     }
> > +
>
> Other than the above, this looks OK to me.
>
> Back then I had argued that having a >CFS task and holding the remote rq
> lock could let us invoke detach_one_task() locally (rather than on the
> stopper thread), but realistically if we got to this !ld_moved condition
> then the chances of us actually pulling something here are very slim (we'd
> depend on enqueues happening between ~detach_tasks() and here).
>
> >                       /*
> >                        * ->active_balance synchronizes accesses to
> >                        * ->active_balance_work.  Once set, it's cleared
> > --
> > 2.17.1