Re: [rfc patch] sched/fair: Use instantaneous load for fork/exec balancing

[Dietmar Eggemann]

On 14/06/16 08:58, Mike Galbraith wrote:
> SUSE's regression testing noticed that...
> 
> 0905f04eb21f sched/fair: Fix new task's load avg removed from source CPU in wake_up_new_task()
> 
> ...introduced a hackbench regression, and indeed it does.  I think this
> regression has more to do with randomness than anything else, but in
> general...
> 
> While averaging calms down load balancing, helping to keep migrations
> down to a dull roar, it's not completely wonderful when it comes to
> things that live in the here and now, hackbench being one such.
> 
> time sh -c 'for i in `seq 1000`; do hackbench -p -P > /dev/null; done'
> 
> real    0m55.397s
> user    0m8.320s
> sys     5m40.789s
> 
> echo LB_INSTANTANEOUS_LOAD > /sys/kernel/debug/sched_features
> 
> real    0m48.049s
> user    0m6.510s
> sys     5m6.291s
> 
> Signed-off-by: Mike Galbraith <umgwanakikbuti@xxxxxxxxx>

I see similar values on ARM64 (Juno r0: 2xCortex-A57 4xCortex-A53). OK,
1000 invocations of hackbench take a little bit longer but I guess it's
the fork's we're after.

- echo NO_LB_INSTANTANEOUS_LOAD > /sys/kernel/debug/sched_features

time sh -c 'for i in `seq 1000`; do hackbench -p -P > /dev/null; done'

root@juno:~# time sh -c 'for i in `seq 1000`; do hackbench -p -P >
/dev/null; done'

real	10m17.155s
user	2m56.976s
sys	38m0.324s

- echo LB_INSTANTANEOUS_LOAD > /sys/kernel/debug/sched_features

time sh -c 'for i in `seq 1000`; do hackbench -p -P > /dev/null; done'

real	9m49.832s
user	2m42.896s
sys	34m51.452s

- But I get a similar effect in case I initialize se->avg.load_avg w/ 0:

--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -680,7 +680,7 @@ void init_entity_runnable_average(struct
sched_entity *se)
         * will definitely be update (after enqueue).
         */
        sa->period_contrib = 1023;
-       sa->load_avg = scale_load_down(se->load.weight);
+       sa->load_avg = scale_load_down(0);
        sa->load_sum = sa->load_avg * LOAD_AVG_MAX;

root@juno:~# time sh -c 'for i in `seq 1000`; do hackbench -p -P >
/dev/null; done'

real	9m55.396s
user	2m41.192s
sys	35m6.196s


IMHO, the hackbench performance "boost" w/o 0905f04eb21f is due to the
fact that a new task gets all it's load decayed (making it a small task)
in the __update_load_avg() call in remove_entity_load_avg() because its
se->avg.last_update_time value is 0 which creates a huge time difference
comparing it to cfs_rq->avg.last_update_time. The patch 0905f04eb21f
avoids this and thus the task stays big se->avg.load_avg = 1024.

It can't be a difference in the value of cfs_rq->removed_load_avg
because w/o the patch 0905f04eb21f, we atomic_long_add 0 and with the
patch we bail before the atomic_long_add().

[...]