Re: [RFCv3 PATCH 48/48] sched: Disable energy-unfriendly nohz kicks

[Dietmar Eggemann]

Hi Morten,

On 04/02/15 18:31, Morten Rasmussen wrote:
> With energy-aware scheduling enabled nohz_kick_needed() generates many
> nohz idle-balance kicks which lead to nothing when multiple tasks get
> packed on a single cpu to save energy. This causes unnecessary wake-ups
> and hence wastes energy. Make these conditions depend on !energy_aware()
> for now until the energy-aware nohz story gets sorted out.
> 
> cc: Ingo Molnar <mingo@xxxxxxxxxx>
> cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
> 
> Signed-off-by: Morten Rasmussen <morten.rasmussen@xxxxxxx>
> ---
>  kernel/sched/fair.c | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 1c248f8..cfe65ae 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -8195,6 +8195,8 @@ static void nohz_idle_balance(struct rq *this_rq, enum cpu_idle_type idle)
>  	clear_bit(NOHZ_BALANCE_KICK, nohz_flags(this_cpu));
>  }
>  
> +static int cpu_overutilized(int cpu, struct sched_domain *sd);
> +
>  /*
>   * Current heuristic for kicking the idle load balancer in the presence
>   * of an idle cpu in the system.
> @@ -8234,12 +8236,13 @@ static inline bool nohz_kick_needed(struct rq *rq)
>  	if (time_before(now, nohz.next_balance))
>  		return false;
>  
> -	if (rq->nr_running >= 2)
> +	sd = rcu_dereference(rq->sd);
> +	if (rq->nr_running >= 2 && (!energy_aware() || cpu_overutilized(cpu, sd)))
>  		return true;

CONFIG_PROVE_RCU checking revealed this one:

[    3.814454] ===============================
[    3.826989] [ INFO: suspicious RCU usage. ]
[    3.839526] 3.19.0-rc7+ #10 Not tainted
[    3.851018] -------------------------------
[    3.863554] kernel/sched/fair.c:8239 suspicious
rcu_dereference_check() usage!
[    3.885216]
[    3.885216] other info that might help us debug this:
[    3.885216]
[    3.909236]
[    3.909236] rcu_scheduler_active = 1, debug_locks = 1
[    3.928817] no locks held by kthreadd/437.

The RCU read-side critical section has to be extended to incorporate
this sd = rcu_dereference(rq->sd):

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index cfe65aec3237..145360ee6e4a 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -8236,11 +8236,13 @@ static inline bool nohz_kick_needed(struct rq *rq)
        if (time_before(now, nohz.next_balance))
                return false;

+       rcu_read_lock();
        sd = rcu_dereference(rq->sd);
-       if (rq->nr_running >= 2 && (!energy_aware() ||
cpu_overutilized(cpu, sd)))
-               return true;
+       if (rq->nr_running >= 2 && (!energy_aware() ||
cpu_overutilized(cpu, sd))) {
+               kick = true;
+               goto unlock;
+       }

-       rcu_read_lock();
        sd = rcu_dereference(per_cpu(sd_busy, cpu));
        if (sd && !energy_aware()) {
                sgc = sd->groups->sgc;

-- Dietmar

>  
>  	rcu_read_lock();
>  	sd = rcu_dereference(per_cpu(sd_busy, cpu));
> -	if (sd) {
> +	if (sd && !energy_aware()) {
>  		sgc = sd->groups->sgc;
>  		nr_busy = atomic_read(&sgc->nr_busy_cpus);
>  
> 


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/