[sched/balance] 7511dd0a7: +2.1e+05% context switches

From: Fengguang Wu
Date: Wed Feb 19 2014 - 08:01:22 EST

Alex,

We noticed the below changes for commit

commit 7511dd0a73aaf2ca4bcd829f991dcd2c839c5417
Author: Alex Shi <alex.shi@xxxxxxxxxx>
AuthorDate: Fri Jan 24 15:17:02 2014 +0800
Commit: Alex Shi <alex.shi@xxxxxxxxxx>
CommitDate: Fri Feb 14 15:09:01 2014 +0800

sched: only do load balance on tick_do_timer_cpu

that is a central balance.

Signed-off-by: Alex Shi <alex.shi@xxxxxxxxxx>

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
83892 ~ 5% -96.1% 3245 ~ 7% lkp-snb01/micro/will-it-scale/pthread_mutex2
86509 ~ 5% -96.2% 3271 ~ 6% lkp-snb01/micro/will-it-scale/sched_yield
170401 ~ 5% -96.2% 6517 ~ 7% TOTAL softirqs.SCHED

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
60321 ~ 0% -78.5% 12962 ~ 3% lkp-snb01/micro/will-it-scale/pthread_mutex2
60358 ~ 0% -80.5% 11797 ~ 4% lkp-snb01/micro/will-it-scale/sched_yield
120679 ~ 0% -79.5% 24759 ~ 4% TOTAL interrupts.RES

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
105019 ~ 0% -41.8% 61097 ~ 3% lkp-snb01/micro/will-it-scale/pthread_mutex2
103938 ~ 1% -35.4% 67183 ~ 1% lkp-snb01/micro/will-it-scale/sched_yield
208958 ~ 0% -38.6% 128281 ~ 2% TOTAL cpuidle.C7-SNB.usage

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
2492 ~23% +39.8% 3483 ~14% lkp-snb01/micro/will-it-scale/sched_yield
2492 ~23% +39.8% 3483 ~14% TOTAL cpuidle.C1E-SNB.usage

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
543 ~10% +30.8% 710 ~11% lkp-snb01/micro/will-it-scale/sched_yield
543 ~10% +30.8% 710 ~11% TOTAL cpuidle.C3-SNB.usage

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
20.65 ~ 0% +24.1% 25.63 ~ 2% lkp-snb01/micro/will-it-scale/sched_yield
20.65 ~ 0% +24.1% 25.63 ~ 2% TOTAL turbostat.%c1

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
1268 ~13% +17.0% 1483 ~ 5% lkp-snb01/micro/will-it-scale/sched_yield
1268 ~13% +17.0% 1483 ~ 5% TOTAL numa-vmstat.node0.nr_alloc_batch

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
7255 ~ 2% +14.6% 8317 ~ 2% lkp-snb01/micro/will-it-scale/pthread_mutex2
7255 ~ 2% +14.6% 8317 ~ 2% TOTAL slabinfo.kmalloc-192.num_objs

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
7255 ~ 2% +14.6% 8317 ~ 2% lkp-snb01/micro/will-it-scale/pthread_mutex2
7255 ~ 2% +14.6% 8317 ~ 2% TOTAL slabinfo.kmalloc-192.active_objs

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
683581 ~ 0% -9.3% 620114 ~ 1% lkp-snb01/micro/will-it-scale/sched_yield
683581 ~ 0% -9.3% 620114 ~ 1% TOTAL softirqs.TIMER

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
204033 ~ 1% -8.6% 186557 ~ 2% lkp-snb01/micro/will-it-scale/sched_yield
204033 ~ 1% -8.6% 186557 ~ 2% TOTAL softirqs.RCU

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
4.902e+09 ~ 0% +9.8% 5.385e+09 ~ 1% lkp-snb01/micro/will-it-scale/sched_yield
4.902e+09 ~ 0% +9.8% 5.385e+09 ~ 1% TOTAL cpuidle.C7-SNB.time

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
8204 ~ 2% +11.3% 9134 ~ 4% lkp-snb01/micro/will-it-scale/sched_yield
8204 ~ 2% +11.3% 9134 ~ 4% TOTAL slabinfo.kmalloc-256.num_objs

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
2405 ~25% +2.1e+05% 5058323 ~13% lkp-snb01/micro/will-it-scale/sched_yield
2405 ~25% +2.1e+05% 5058323 ~13% TOTAL vmstat.system.cs

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
2029 ~ 0% +222.9% 6551 ~17% lkp-snb01/micro/will-it-scale/pthread_mutex2
143678 ~42% +4.8e+05% 6.927e+08 ~ 0% lkp-snb01/micro/will-it-scale/sched_yield
145708 ~42% +4.8e+05% 6.927e+08 ~ 0% TOTAL time.involuntary_context_switches

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
1267 ~ 0% -22.3% 985 ~ 1% lkp-snb01/micro/will-it-scale/sched_yield
1267 ~ 0% -22.3% 985 ~ 1% TOTAL time.system_time

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
1518 ~ 0% -4.9% 1444 ~ 1% lkp-snb01/micro/will-it-scale/pthread_mutex2
250 ~ 4% +26.3% 316 ~ 5% lkp-snb01/micro/will-it-scale/sched_yield
1769 ~ 0% -0.5% 1761 ~ 2% TOTAL time.user_time

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
489 ~ 0% -4.9% 464 ~ 1% lkp-snb01/micro/will-it-scale/pthread_mutex2
489 ~ 0% -14.3% 419 ~ 0% lkp-snb01/micro/will-it-scale/sched_yield
978 ~ 0% -9.6% 883 ~ 0% TOTAL time.percent_of_cpu_this_job_got

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
50.58 ~ 0% -2.0% 49.56 ~ 0% lkp-snb01/micro/will-it-scale/pthread_mutex2
50.62 ~ 0% -9.9% 45.62 ~ 1% lkp-snb01/micro/will-it-scale/sched_yield
101.20 ~ 0% -5.9% 95.19 ~ 0% TOTAL turbostat.%c0

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
4386 ~ 0% -5.2% 4158 ~ 0% lkp-snb01/micro/will-it-scale/pthread_mutex2
4386 ~ 0% -5.2% 4158 ~ 0% TOTAL vmstat.system.in

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
148 ~ 0% -1.9% 145 ~ 0% lkp-snb01/micro/will-it-scale/sched_yield
148 ~ 0% -1.9% 145 ~ 0% TOTAL turbostat.Pkg_W

bc575710efe937e 7511dd0a73aaf2ca4bcd829f9
--------------- -------------------------
109 ~ 0% -0.1% 109 ~ 0% lkp-snb01/micro/will-it-scale/pthread_mutex2
121 ~ 0% -2.3% 119 ~ 0% lkp-snb01/micro/will-it-scale/sched_yield
231 ~ 0% -1.3% 228 ~ 0% TOTAL turbostat.Cor_W

time.percent_of_cpu_this_job_got

490 *+-----*---*---*--*---*--*---*---*--*---*---*--*---*--*---*---*--*---*
| .. + |
480 ++ + |
470 ++ * |
| |
460 ++ |
| |
450 ++ |
| |
440 ++ |
430 ++ |
| O |
420 O+ O O O O O O O O O O O O O O O O O O
| |
410 ++-------------------------------------------------------------------+


time.involuntary_context_switches

8e+08 ++-----------------------------------------------------------------+
| |
7e+08 O+ O O O O O O O O O O O O O O O O O O
6e+08 ++ O |
| |
5e+08 ++ |
| |
4e+08 ++ |
| |
3e+08 ++ |
2e+08 ++ |
| |
1e+08 ++ |
| |
0 *+--*--*---*--*---*--*---*--*---*--*---*--*---*--*---*--*---*--*---*


turbostat.%c0

51 *+--------------------------------------------------------------------+
| . .*...*...*..*...*...*..*...*...*...*..*...*...*..*...*...*..*...*
50 ++ *. |
| |
49 ++ |
| |
48 ++ |
| |
47 ++ |
| O
46 ++ O O O O |
O O O O O O O O O O O O O O |
45 ++ |
| O |
44 ++--------------------------------------------------------------------+


vmstat.system.cs

6e+06 ++------------------------------------------------------O----------+
| O O O |
5e+06 O+ O O O O O O O O O O |
| O O O |
| O O
4e+06 ++ |
| |
3e+06 ++ |
| |
2e+06 ++ |
| |
| |
1e+06 ++ |
| |
0 *+--*--*---*--*---*--*---*--*---*--*---*--*---*--*---*--*---*--*---*

Thanks,
Fengguang
>From 7511dd0a73aaf2ca4bcd829f991dcd2c839c5417 Mon Sep 17 00:00:00 2001
From: Alex Shi <alex.shi@xxxxxxxxxx>
Date: Fri, 24 Jan 2014 15:17:02 +0800
Subject: [PATCH] sched: only do load balance on tick_do_timer_cpu

that is a central balance.

Signed-off-by: Alex Shi <alex.shi@xxxxxxxxxx>
---
kernel/sched/core.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 7368964..1be85a7 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2425,6 +2425,7 @@ unsigned long long task_sched_runtime(struct task_struct *p)
return ns;
}

+extern int tick_do_timer_cpu __read_mostly;
/*
* This function gets called by the timer code, with HZ frequency.
* We call it with interrupts disabled.
@@ -2446,8 +2447,11 @@ void scheduler_tick(void)
perf_event_task_tick();

#ifdef CONFIG_SMP
- rq->idle_balance = idle_cpu(cpu);
- trigger_load_balance(rq);
+ /* only do load balance on tick_do_timer_cpu */
+ if (cpu == tick_do_timer_cpu) {
+ rq->idle_balance = idle_cpu(cpu);
+ trigger_load_balance(rq);
+ }
#endif
rq_last_tick_reset(rq);
}
--
1.8.5.2

Attachment: 2014-02-16_10:06:49.png
Description: PNG image

2014-02-16 04:18:41 /lkp/lkp/src/sbin/perf-bisect /lkp/.bisect-descs/bisect-desc.vmstat.system.cs.2014-02-16.03:54:18
---
stat: vmstat.system.cs
interval: ! '[ 5.1883e+06 - 5.1883e+06 ] -- [ 2474 - 4961 ]'
a:
- 5188285
b:
- 2474
- 2919
- 3479
- 3594
- 3669
- 3802
- 4124
- 4194
- 4961
ttl: 2014-02-15 17:33:53.961378385 +08:00
ratio: 5
delta: 5184616
mean_a: 5188285
mean_b: 3669
x: 0
y: 5183324
z: 2487
min_a: 5188285
max_a: 5188285
min_b: 2474
max_b: 4961
max: 5188285
nr_run: 1469
good_commit: b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed
tbox_group: lkp-snb01
run_time: 311
result_root: /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/0
_result_root: /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64
head_commit: 5abff587f6ba8de0adf41a19c4add6a3a4b37b64
base_commit: b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed
commit: 5abff587f6ba8de0adf41a19c4add6a3a4b37b64
branch: linux-devel/devel-hourly-2014021515
kconfig: x86_64-lkp
testbox: lkp-snb01
score_perf_change: 0
change_percent: 0.999292829904294
variation: 0.00047957783266686733
avg_run_time: 312
score_change: 99.96463524188411
score_var: 9.985619563763917
score_time: 90
score_metric: 4
prio: 203.9503
2014-02-16 04:18:42 [perf-bisect 98512] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed -q reconfirm -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 3 times: /lkp/jobs/reconfirm/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed-2.yaml
2014-02-16 04:24:23 [perf-bisect 98512] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:5abff587f6ba8de0adf41a19c4add6a3a4b37b64 -q reconfirm -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 3 times: /lkp/jobs/reconfirm/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-5abff587f6ba8de0adf41a19c4add6a3a4b37b64-2.yaml
2014-02-16 04:44:24 [perf-bisect 98512] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed -q reconfirm -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 5 times: /lkp/jobs/reconfirm/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed-4.yaml
2014-02-16 04:56:44 [perf-bisect 98512] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:5abff587f6ba8de0adf41a19c4add6a3a4b37b64 -q reconfirm -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 5 times: /lkp/jobs/reconfirm/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-5abff587f6ba8de0adf41a19c4add6a3a4b37b64-4.yaml
git checkout b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed
2014-02-16 05:15:06 [perf-bisect-test 73445] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed-1.yaml
2014-02-16 05:22:06 [perf-bisect-test 73445] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json
bisect GOOD: [2734, 3802, 7773] [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

bisect: good commit b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed
git bisect start 5abff587f6ba8de0adf41a19c4add6a3a4b37b64 b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed --
/c/kernel-tests/linear-bisect: ["-b", "5abff587f6ba8de0adf41a19c4add6a3a4b37b64", "-g", "b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed", "/lkp/lkp/src/sbin/perf-bisect-test", "/cc/perf-bisect-obj/lkp/x86_64-lkp"]
Bisecting: 1944 revisions left to test after this (roughly 11 steps)
[7aaf3caaab7f5ddc8da4359a93a8a6be86946169] Merge 'at91/at91-3.15-cleanup' into devel-hourly-2014021515
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 05:22:07 [perf-bisect-test 10038] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:7aaf3caaab7f5ddc8da4359a93a8a6be86946169 -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-7aaf3caaab7f5ddc8da4359a93a8a6be86946169-1.yaml
2014-02-16 05:40:58 [perf-bisect-test 10038] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/7aaf3caaab7f5ddc8da4359a93a8a6be86946169/matrix.json
bisect BAD: [5152320, 5470320, 5470320] [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

Bisecting: 1076 revisions left to test after this (roughly 11 steps)
[f6afc6d8b9d433388365d1567babe4b1738400bb] Merge 'pci/pci/host-mvebu' into devel-hourly-2014021515
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 05:40:58 [perf-bisect-test 40970] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:f6afc6d8b9d433388365d1567babe4b1738400bb -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
/lkp/lkp/src/sbin/kbuild-queue -c x86_64-lkp -b linux-devel/devel-hourly-2014021515 -k f6afc6d8b9d433388365d1567babe4b1738400bb -w
Queued build task to /kernel-tests/build-queue/x86_64-lkp-f6afc6d8b9d433388365d1567babe4b1738400bb
Check for kernel in /kernel/x86_64-lkp/f6afc6d8b9d433388365d1567babe4b1738400bb
waiting for completion of /kernel-tests/build-queue/x86_64-lkp-f6afc6d8b9d433388365d1567babe4b1738400bb
waiting for completion of /kernel-tests/build-queue/.x86_64-lkp-f6afc6d8b9d433388365d1567babe4b1738400bb
kernel: /kernel/x86_64-lkp/f6afc6d8b9d433388365d1567babe4b1738400bb/vmlinuz-3.14.0-rc2-wl-ath-00835-gf6afc6d
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-f6afc6d8b9d433388365d1567babe4b1738400bb-1.yaml
2014-02-16 06:00:19 [perf-bisect-test 40970] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/f6afc6d8b9d433388365d1567babe4b1738400bb/matrix.json
bisect GOOD: [2868, 3166, 3166] [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

Bisecting: 241 revisions left to test after this (roughly 8 steps)
[d67497e37737967e5edb30ff331e7479e19a13f3] Merge 'char-misc/char-misc-linus' into devel-hourly-2014021515
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 06:00:19 [perf-bisect-test 797] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:d67497e37737967e5edb30ff331e7479e19a13f3 -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
/lkp/lkp/src/sbin/kbuild-queue -c x86_64-lkp -b linux-devel/devel-hourly-2014021515 -k d67497e37737967e5edb30ff331e7479e19a13f3 -w
Queued build task to /kernel-tests/build-queue/x86_64-lkp-d67497e37737967e5edb30ff331e7479e19a13f3
Check for kernel in /kernel/x86_64-lkp/d67497e37737967e5edb30ff331e7479e19a13f3
waiting for completion of /kernel-tests/build-queue/x86_64-lkp-d67497e37737967e5edb30ff331e7479e19a13f3
waiting for completion of /kernel-tests/build-queue/.x86_64-lkp-d67497e37737967e5edb30ff331e7479e19a13f3
kernel: /kernel/x86_64-lkp/d67497e37737967e5edb30ff331e7479e19a13f3/vmlinuz-3.14.0-rc2-wl-ath-00912-gd67497e
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-d67497e37737967e5edb30ff331e7479e19a13f3-1.yaml
2014-02-16 06:18:30 [perf-bisect-test 797] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/d67497e37737967e5edb30ff331e7479e19a13f3/matrix.json
bisect BAD: [5209415, 5211906, 5211906] [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

Bisecting: 77 revisions left to test after this (roughly 7 steps)
[933530ed7e473b3c0efa0a0b39c09c65c04beb37] Merge 'arm-perf/for-next/perf' into devel-hourly-2014021515
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 06:18:31 [perf-bisect-test 41141] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:933530ed7e473b3c0efa0a0b39c09c65c04beb37 -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
/lkp/lkp/src/sbin/kbuild-queue -c x86_64-lkp -b linux-devel/devel-hourly-2014021515 -k 933530ed7e473b3c0efa0a0b39c09c65c04beb37 -w
Queued build task to /kernel-tests/build-queue/x86_64-lkp-933530ed7e473b3c0efa0a0b39c09c65c04beb37
Check for kernel in /kernel/x86_64-lkp/933530ed7e473b3c0efa0a0b39c09c65c04beb37
waiting for completion of /kernel-tests/build-queue/x86_64-lkp-933530ed7e473b3c0efa0a0b39c09c65c04beb37
waiting for completion of /kernel-tests/build-queue/.x86_64-lkp-933530ed7e473b3c0efa0a0b39c09c65c04beb37
kernel: /kernel/x86_64-lkp/933530ed7e473b3c0efa0a0b39c09c65c04beb37/vmlinuz-3.14.0-rc2-wl-ath-00868-g933530e
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-933530ed7e473b3c0efa0a0b39c09c65c04beb37-1.yaml
2014-02-16 06:36:23 [perf-bisect-test 41141] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/933530ed7e473b3c0efa0a0b39c09c65c04beb37/matrix.json
bisect GOOD: [2499, 2714, 2714] [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

Bisecting: 44 revisions left to test after this (roughly 6 steps)
[9fd1bb34f21472d352fbf49bbdf1f07a320393ca] Merge 'kvm/master' into devel-hourly-2014021515
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 06:36:23 [perf-bisect-test 69581] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:9fd1bb34f21472d352fbf49bbdf1f07a320393ca -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
/lkp/lkp/src/sbin/kbuild-queue -c x86_64-lkp -b linux-devel/devel-hourly-2014021515 -k 9fd1bb34f21472d352fbf49bbdf1f07a320393ca -w
Queued build task to /kernel-tests/build-queue/x86_64-lkp-9fd1bb34f21472d352fbf49bbdf1f07a320393ca
Check for kernel in /kernel/x86_64-lkp/9fd1bb34f21472d352fbf49bbdf1f07a320393ca
waiting for completion of /kernel-tests/build-queue/x86_64-lkp-9fd1bb34f21472d352fbf49bbdf1f07a320393ca
waiting for completion of /kernel-tests/build-queue/.x86_64-lkp-9fd1bb34f21472d352fbf49bbdf1f07a320393ca
kernel: /kernel/x86_64-lkp/9fd1bb34f21472d352fbf49bbdf1f07a320393ca/vmlinuz-3.14.0-rc2-wl-ath-00876-g9fd1bb3
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-9fd1bb34f21472d352fbf49bbdf1f07a320393ca-1.yaml
2014-02-16 06:55:04 [perf-bisect-test 69581] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/9fd1bb34f21472d352fbf49bbdf1f07a320393ca/matrix.json
bisect GOOD: [3523, 7441, 7441] [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

Bisecting: 36 revisions left to test after this (roughly 6 steps)
[1a9a97cd15230efd44ff9957326dd245f89bdbd6] Merge 'alexshi/topdown' into devel-hourly-2014021515
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 06:55:05 [perf-bisect-test 101819] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:1a9a97cd15230efd44ff9957326dd245f89bdbd6 -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
/lkp/lkp/src/sbin/kbuild-queue -c x86_64-lkp -b linux-devel/devel-hourly-2014021515 -k 1a9a97cd15230efd44ff9957326dd245f89bdbd6 -w
Queued build task to /kernel-tests/build-queue/x86_64-lkp-1a9a97cd15230efd44ff9957326dd245f89bdbd6
Check for kernel in /kernel/x86_64-lkp/1a9a97cd15230efd44ff9957326dd245f89bdbd6
waiting for completion of /kernel-tests/build-queue/x86_64-lkp-1a9a97cd15230efd44ff9957326dd245f89bdbd6
waiting for completion of /kernel-tests/build-queue/.x86_64-lkp-1a9a97cd15230efd44ff9957326dd245f89bdbd6
kernel: /kernel/x86_64-lkp/1a9a97cd15230efd44ff9957326dd245f89bdbd6/vmlinuz-3.14.0-rc2-wl-ath-00895-g1a9a97c
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-1a9a97cd15230efd44ff9957326dd245f89bdbd6-1.yaml
2014-02-16 07:21:36 [perf-bisect-test 101819] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/1a9a97cd15230efd44ff9957326dd245f89bdbd6/matrix.json
bisect BAD: [5211562, 5215236, 5215236] [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

linear-bisect: bad branch may be alexshi/topdown
[v3.14-rc2] Linux 3.14-rc2
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 07:21:39 [perf-bisect-test 43744] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed-1.yaml
2014-02-16 07:25:30 [perf-bisect-test 43744] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json
bisect GOOD: [2734, 3802, 7773] [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

[d2f8e017fdd913fc150160508d2e1161cab5fef6] sched: remove un-used functions
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 07:25:30 [perf-bisect-test 115104] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:d2f8e017fdd913fc150160508d2e1161cab5fef6 -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-d2f8e017fdd913fc150160508d2e1161cab5fef6-1.yaml
2014-02-16 07:45:41 [perf-bisect-test 115104] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/d2f8e017fdd913fc150160508d2e1161cab5fef6/matrix.json
bisect BAD: [4736446, 5001400, 5001400] [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

linear-bisect: recursive descent into merged branch
/c/kernel-tests/linear-bisect: ["-g", "v3.14-rc2", "-b", "d2f8e017fdd913fc150160508d2e1161cab5fef6", "/lkp/lkp/src/sbin/perf-bisect-test", "/cc/perf-bisect-obj/lkp/x86_64-lkp"]
Bisecting: 18 revisions left to test after this (roughly 5 steps)
[f6ab232915356bd6649052c1672c10424feab995] sched: remove rq->cpu_load and rq->nr_load_updates
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 07:45:42 [perf-bisect-test 74444] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:f6ab232915356bd6649052c1672c10424feab995 -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-f6ab232915356bd6649052c1672c10424feab995-1.yaml
2014-02-16 08:04:02 [perf-bisect-test 74444] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/f6ab232915356bd6649052c1672c10424feab995/matrix.json
bisect GOOD: [2329, 4551, 4551] [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

Bisecting: 9 revisions left to test after this (roughly 4 steps)
[7511dd0a73aaf2ca4bcd829f991dcd2c839c5417] sched: only do load balance on tick_do_timer_cpu
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 08:04:03 [perf-bisect-test 53445] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:7511dd0a73aaf2ca4bcd829f991dcd2c839c5417 -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-7511dd0a73aaf2ca4bcd829f991dcd2c839c5417-1.yaml
2014-02-16 08:21:33 [perf-bisect-test 53445] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/7511dd0a73aaf2ca4bcd829f991dcd2c839c5417/matrix.json
bisect BAD: [5373928, 5996496, 5996496] [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

Bisecting: 4 revisions left to test after this (roughly 3 steps)
[6de9950778921604a8bd0fbab09911df5d98bbeb] sched: add statistic for newidle load balance cost
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 08:21:34 [perf-bisect-test 112510] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:6de9950778921604a8bd0fbab09911df5d98bbeb -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
/lkp/lkp/src/sbin/kbuild-queue -c x86_64-lkp -b linux-devel/devel-hourly-2014021515 -k 6de9950778921604a8bd0fbab09911df5d98bbeb -w
Queued build task to /kernel-tests/build-queue/x86_64-lkp-6de9950778921604a8bd0fbab09911df5d98bbeb
Check for kernel in /kernel/x86_64-lkp/6de9950778921604a8bd0fbab09911df5d98bbeb
waiting for completion of /kernel-tests/build-queue/x86_64-lkp-6de9950778921604a8bd0fbab09911df5d98bbeb
waiting for completion of /kernel-tests/build-queue/.x86_64-lkp-6de9950778921604a8bd0fbab09911df5d98bbeb
kernel: /kernel/x86_64-lkp/6de9950778921604a8bd0fbab09911df5d98bbeb/vmlinuz-3.14.0-rc2-00011-g6de9950
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-6de9950778921604a8bd0fbab09911df5d98bbeb-1.yaml
2014-02-16 08:46:04 [perf-bisect-test 112510] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/6de9950778921604a8bd0fbab09911df5d98bbeb/matrix.json
bisect GOOD: [2622, 2906, 2906] [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

Bisecting: 2 revisions left to test after this (roughly 2 steps)
[bc575710efe937e26d2bd541d92362b240ef0559] sched: clean up task_hot function
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 08:46:05 [perf-bisect-test 5884] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:bc575710efe937e26d2bd541d92362b240ef0559 -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
/lkp/lkp/src/sbin/kbuild-queue -c x86_64-lkp -b linux-devel/devel-hourly-2014021515 -k bc575710efe937e26d2bd541d92362b240ef0559 -w
Queued build task to /kernel-tests/build-queue/x86_64-lkp-bc575710efe937e26d2bd541d92362b240ef0559
Check for kernel in /kernel/x86_64-lkp/bc575710efe937e26d2bd541d92362b240ef0559
waiting for completion of /kernel-tests/build-queue/x86_64-lkp-bc575710efe937e26d2bd541d92362b240ef0559
waiting for completion of /kernel-tests/build-queue/.x86_64-lkp-bc575710efe937e26d2bd541d92362b240ef0559
kernel: /kernel/x86_64-lkp/bc575710efe937e26d2bd541d92362b240ef0559/vmlinuz-3.14.0-rc2-00012-gbc57571
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-bc575710efe937e26d2bd541d92362b240ef0559-1.yaml
2014-02-16 09:17:05 [perf-bisect-test 5884] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/bc575710efe937e26d2bd541d92362b240ef0559/matrix.json
bisect GOOD: [1794, 2328, 2328] [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

7511dd0a73aaf2ca4bcd829f991dcd2c839c5417 is the first bad commit
commit 7511dd0a73aaf2ca4bcd829f991dcd2c839c5417
Author: Alex Shi <alex.shi@xxxxxxxxxx>
AuthorDate: Fri Jan 24 15:17:02 2014 +0800
Commit: Alex Shi <alex.shi@xxxxxxxxxx>
CommitDate: Fri Feb 14 15:09:01 2014 +0800

sched: only do load balance on tick_do_timer_cpu

that is a central balance.

Signed-off-by: Alex Shi <alex.shi@xxxxxxxxxx>

kernel/sched/core.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
exit
/c/kernel-tests/linear-bisect:274:in `exit'
/c/kernel-tests/linear-bisect:274:in `linear_bisect'
/c/kernel-tests/linear-bisect:294:in `<main>'
exit
/c/kernel-tests/linear-bisect:181:in `exit'
/c/kernel-tests/linear-bisect:181:in `recursive_bisect_into'
/c/kernel-tests/linear-bisect:210:in `perf_test_merged_branch'
/c/kernel-tests/linear-bisect:286:in `block in linear_bisect'
/c/kernel-tests/linear-bisect:286:in `each'
/c/kernel-tests/linear-bisect:286:in `linear_bisect'
/c/kernel-tests/linear-bisect:294:in `<main>'
2014-02-16 09:17:06 [perf-bisect 98512] validating first bad commit 7511dd0a73aaf2ca4bcd829f991dcd2c839c5417 ...
2014-02-16 09:17:06 [perf-bisect 98512] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:7511dd0a73aaf2ca4bcd829f991dcd2c839c5417 -q validate -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 5 times: /lkp/jobs/validate/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-7511dd0a73aaf2ca4bcd829f991dcd2c839c5417-4.yaml
bisect BAD: [4110714, 5373928, 5996496] [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

2014-02-16 09:41:56 [perf-bisect 98512] validating parent commit bc575710efe937e26d2bd541d92362b240ef0559 ...
2014-02-16 09:41:56 [perf-bisect 98512] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:bc575710efe937e26d2bd541d92362b240ef0559 -q validate -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 5 times: /lkp/jobs/validate/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-bc575710efe937e26d2bd541d92362b240ef0559-4.yaml
bisect GOOD: [1794, 2316, 3578] [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

2014-02-16 10:06:47 [perf-bisect 98512] first bad commit validated successfully
==================== PARENT FIRST_BAD KCONFIGS ====================
stats changes plotted at /lkp/bisect-plot/vmstat.system.cs/2014-02-16_10:06:49.png
_______________________________________________
LKP mailing list
LKP@xxxxxxxxxxxxxxx