Re: [RFC PATCH] sched/fair: correct llc shared domain's number of busy CPUs

[Vincent Guittot]

On Mon, 4 May 2020 at 03:57, Hillf Danton <hdanton@xxxxxxxx> wrote:
>
>
> The comment says, if there is an imbalance between LLC domains (IOW we
> could increase the overall cache use),  we need some less-loaded LLC
> domain to pull some load.
>
> To show that imbalance, record busy CPUs as they come and go by doing
> a minor cleanup for sd::nohz_idle.

Your comment failed to explain why we can get rid of sd->nohz_idle

>
> Cc: Mel Gorman <mgorman@xxxxxxxxxxxxxxxxxxx>
> Cc: Vincent Guittot <vincent.guittot@xxxxxxxxxx>
> Cc: Valentin Schneider <valentin.schneider@xxxxxxx>
> Cc: Dietmar Eggemann <dietmar.eggemann@xxxxxxx>
> Signed-off-by: Hillf Danton <hdanton@xxxxxxxx>
> ---
>
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -10138,13 +10138,8 @@ static void set_cpu_sd_state_busy(int cp
>
>         rcu_read_lock();
>         sd = rcu_dereference(per_cpu(sd_llc, cpu));
> -
> -       if (!sd || !sd->nohz_idle)
> -               goto unlock;
> -       sd->nohz_idle = 0;

you remove the use of sd->nohz_idle but you don't remove it from
struct sched_domain

> -
> -       atomic_inc(&sd->shared->nr_busy_cpus);
> -unlock:
> +       if (sd)
> +               atomic_inc(&sd->shared->nr_busy_cpus);
>         rcu_read_unlock();
>  }
>
> @@ -10168,13 +10163,8 @@ static void set_cpu_sd_state_idle(int cp
>
>         rcu_read_lock();
>         sd = rcu_dereference(per_cpu(sd_llc, cpu));
> -
> -       if (!sd || sd->nohz_idle)
> -               goto unlock;
> -       sd->nohz_idle = 1;
> -
> -       atomic_dec(&sd->shared->nr_busy_cpus);
> -unlock:
> +       if (sd)
> +               atomic_dec(&sd->shared->nr_busy_cpus);
>         rcu_read_unlock();
>  }
>
>