Re: [PATCH 9/7] sched,numa: remove task_h_load from task_numa_compare

[Peter Zijlstra]

On Wed, Jun 25, 2014 at 07:07:35AM +0200, Peter Zijlstra wrote:
> Shall I merge this into patch 3?

Which gets me the below; which is has a wrong changelog.

task_h_load() already computes the load as seen from the root group.
effective_load() just does a better (and more expensive) job of
computing the task movement implications of a move.

So the total effect of this patch shouldn't be very big; regular load
balancing also only uses task_h_load(), see move_tasks().

Now, we don't run with preemption disabled, don't run as often, etc..,
so maybe we can indeed use the more expensive variant just fine, but
does it really matter?

---
Subject: sched,numa: use effective_load to balance NUMA loads
From: Rik van Riel <riel@xxxxxxxxxx>
Date: Mon, 23 Jun 2014 11:46:14 -0400

When CONFIG_FAIR_GROUP_SCHED is enabled, the load that a task places
on a CPU is determined by the group the task is in. This is conveniently
calculated for us by effective_load(), which task_numa_compare should
use.

The active groups on the source and destination CPU can be different,
so the calculation needs to be done separately for each CPU.

Cc: mgorman@xxxxxxx
Cc: mingo@xxxxxxxxxx
Cc: chegu_vinod@xxxxxx
Signed-off-by: Rik van Riel <riel@xxxxxxxxxx>
Signed-off-by: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
Link: http://lkml.kernel.org/r/1403538378-31571-3-git-send-email-riel@xxxxxxxxxx
---
 kernel/sched/fair.c |   20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)

--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -1151,6 +1151,7 @@ static void task_numa_compare(struct tas
 	struct rq *src_rq = cpu_rq(env->src_cpu);
 	struct rq *dst_rq = cpu_rq(env->dst_cpu);
 	struct task_struct *cur;
+	struct task_group *tg;
 	long src_load, dst_load;
 	long load;
 	long imp = (groupimp > 0) ? groupimp : taskimp;
@@ -1225,14 +1226,21 @@ static void task_numa_compare(struct tas
 	 * In the overloaded case, try and keep the load balanced.
 	 */
 balance:
-	load = task_h_load(env->p);
-	dst_load = env->dst_stats.load + load;
-	src_load = env->src_stats.load - load;
+	src_load = env->src_stats.load;
+	dst_load = env->dst_stats.load;
+
+	/* Calculate the effect of moving env->p from src to dst. */
+	load = env->p->se.load.weight;
+	tg = task_group(env->p);
+	src_load += effective_load(tg, env->src_cpu, -load, -load);
+	dst_load += effective_load(tg, env->dst_cpu, load, load);
 
 	if (cur) {
-		load = task_h_load(cur);
-		dst_load -= load;
-		src_load += load;
+		/* Cur moves in the opposite direction. */
+		load = cur->se.load.weight;
+		tg = task_group(cur);
+		src_load += effective_load(tg, env->src_cpu, load, load);
+		dst_load += effective_load(tg, env->dst_cpu, -load, -load);
 	}
 
 	if (load_too_imbalanced(src_load, dst_load, env))

Attachment: pgp3aDtV_npgV.pgp
Description: PGP signature