Re: sched: tweak select_idle_sibling to look for idle threads

[Mike Galbraith]

On Sat, 2016-04-30 at 14:47 +0200, Peter Zijlstra wrote:
> On Sat, Apr 09, 2016 at 03:05:54PM -0400, Chris Mason wrote:
> > select_task_rq_fair() can leave cpu utilization a little lumpy,
> > especially as the workload ramps up to the maximum capacity of the
> > machine.  The end result can be high p99 response times as apps
> > wait to get scheduled, even when boxes are mostly idle.
> > 
> > I wrote schbench to try and measure this:
> > 
> > git://git.kernel.org/pub/scm/linux/kernel/git/mason/schbench.git
> 
> Can you guys have a play with this; I think one and two node tbench are
> good, but I seem to be getting significant run to run variance on that,
> so maybe I'm not doing it right.

Nah, tbench is just variance prone.  It got dinged up at clients=cores
on my desktop box, on 4 sockets the high end got seriously dinged up.

tbench

1 x i4790
master                          avg
1     714       684     688     695   1.000
2    1260      1234    1284    1259   1.000
4    2238      2301    2286    2275   1.000
8    3388      3418    3396    3400   1.000

masterx
1     690       701     701     697   1.002
2    1287      1332    1235    1284   1.019
4    2014      2006    1999    2006   0.881
8    3388      3385    3404    3392   0.997

4 x E7-8890
master                          avg
1     524      524      523     523   1.000
2    1049     1053     1045    1049   1.000
4    2064     2081     2091    2078   1.000
8    3737     3813     3746    3765   1.000
16   7129     7028     7082    7079   1.000
32  13718    13730    13578   13675   1.000
64  21397    21435    21519   21450   1.000
128 39846    38397    39026   39089   1.000
256 59509    59797    59344   59550   1.000

masterx                         avg
1     505      507      501     504   0.963   1.000
2    1036     1027     1039    1034   0.985   1.000
4    1977     2001     1992    1990   0.957   1.000
8    3734     3802     3778    3771   1.001   1.000
16   7124     7079     7071    7091   1.001   1.000
32  13549    13758    13364   13557   0.991   1.000
64  21975    22161    22100   22078   1.029   1.000
128 23066    23044    23028   23046   0.589   1.000
256 29905    29630    30753   30096   0.505   1.000

masterx NO_IDLE_CORE            avg
1     500      521      502     507   0.969   1.005
2    1012      996     1043    1017   0.969   0.983
4    1988     1992     1995    1991   0.958   1.000
8    3834     3758     3671    3754   0.997   0.995
16   7160     7206     7168    7178   1.013   1.012
32  13788    13773    13672   13744   1.005   1.013
64  21771    21845    21826   21814   1.016   0.988
128 23248    23136    23133   23172   0.592   1.005
256 28683    30013    31850   30182   0.506   1.002