Re: [RFC PATCH 2/2] sched/fair: Do not specialcase SCHED_IDLE cpus in select slowpath

[Josh Don]

Thanks Abel,

> @@ -7481,12 +7481,13 @@ sched_balance_find_dst_group_cpu(struct sched_group *group, struct task_struct *
>                                 latest_idle_timestamp = rq->idle_stamp;
>                                 shallowest_idle_cpu = i;
>                         }
> -               } else if (shallowest_idle_cpu == -1 && si_cpu == -1) {
> -                       if (sched_idle_cpu(i)) {
> -                               si_cpu = i;
> -                               continue;
> -                       }
> -
> +               } else if (shallowest_idle_cpu == -1) {
> +                       /*
> +                        * The SCHED_IDLE cpus do not necessarily means anything
> +                        * to @p due to the cgroup hierarchical behavior. But it
> +                        * is almost certain that the wakee will get better served
> +                        * if the cpu is less loaded.
> +                        */
>                         load = cpu_load(cpu_rq(i));
>                         if (load < min_load) {
>                                 min_load = load;

This seems reasonable due to the case you describe. However, I'm
wondering if you considered any heuristics here to help identify when
a target cpu should really be considered sched_idle from the
perspective of the incoming task. For example, in your cgroup
hierarchy, if you have a cpu currently only running tasks in your
besteffort container (and all cpus in the system are busy running
something), then that cpu should be considered as a good target for a
waking task in the "guaranteed" container, and not a good target for a
waking task in the "containerd" container.  A simple way to do this
would be to do a find_matching_se on the incoming task and the current
on_cpu task. That does have a drawback of cgroup pointer chasing in a
hot wakeup path, so I don't love it.

In any case, I'm fine with the change as-is, mostly curious if you
gave any additional thought to the case mentioned above.

Best,
Josh