Re: [PATCH v2] sched/fair: Introduce priority load balance to reduce interference from IDLE tasks

[Abel Wu]

On 8/17/22 8:58 PM, Vincent Guittot Wrote:
> On Tue, 16 Aug 2022 at 04:53, zhangsong (J) <zhangsong34@xxxxxxxxxx> wrote:
> > For co-location with NORMAL and IDLE tasks, when CFS trigger load balance,
> > it is reasonable to prefer migrating NORMAL(Latency Sensitive) tasks from
> > the busy src CPU to dst CPU, and migrating IDLE tasks lastly.
> >
> >
> > Considering the large weight difference between normal and idle tasks,
> > does the re-ordering really change things? It would be helpful if you
> > can offer more detailed info.
> >
> > Please consider the situation that CPU A has several normal tasks and hundreds of idle tasks
> > while CPU B is idle, and CPU B needs to pull some tasks from CPU A, but the cfs_tasks in CPU A
> > are not in order of priority, and the max number of pulling tasks depends on env->loop_max,
> > which value is sysctl_sched_nr_migrate, i.e. 32.
> >
> >
> > The case you elaborated above is really rare, the only possibility I
> > can imagine is that all these tasks are affined to one single cpu and
> > suddenly remove the affinity constrain. Otherwise, the load balancing
> > including wakeup cpu selection logic will make things right.
> >
> >
> > Yes, this is usually a corner case, but suppose that some non-idle tasks bounds to CPU 1-2
> >
> > and idle tasks bounds to CPU 0-1, so CPU 1 may has many idle tasks and some non-idle
> >
> > tasks while idle tasks on CPU 1 can not be pulled to CPU 2, when trigger load balance if
> >
> > CPU 2 should pull some tasks from CPU 1, the bad result is idle tasks of CPU 1 cannot be
> >
> > migrated and non-idle tasks also cannot be migrated in case of env->loop_max constraint.
>
> env->loop_max adds a break but load_balance will continue with next
> tasks so it also tries to pull your non idle task at the end after
> several breaks.

Loop will be terminated without LBF_NEED_BREAK if exceeds loop_max :)

> > This will cause non-idle  tasks cannot achieve  more CPU utilization.
>
> Your problem is not linked to IDLE vs NORMAL tasks but to the large
> number of pinned tasks that can't migrate on CPU2. You can end with
> the same behavior without using IDLE tasks but only NORMAL tasks.

I feel the same thing.

Best,
Abel