Re: [RFC PATCH v3 00/16] Core scheduling v3

From: Dario Faggioli
Date: Tue Oct 29 2019 - 05:17:33 EST

On Tue, 2019-10-29 at 10:11 +0100, Dario Faggioli wrote:
> On Sun, 2019-09-15 at 22:14 +0800, Aaron Lu wrote:
> > I'm using the following branch as base which is v5.1.5 based:
> > https://github.com/digitalocean/linux-coresched coresched-v3-
> > v5.1.5-
> > test
> >
> > And I have pushed Tim's branch to:
> > https://github.com/aaronlu/linux coresched-v3-v5.1.5-test-tim
> >
> > Mine:
> > https://github.com/aaronlu/linux coresched-v3-v5.1.5-test-
> > core_vruntime
> >
> Hello,
>
> As anticipated, I've been trying to follow the development of this
> feature and, in the meantime, I have done some benchmarks.
>
> I actually have a lot of data (and am planning for more), so I am
> sending a few emails, each one with a subset of the numbers in it,
> instead than just one which would be beyond giant! :-)
>
SYSBENCHCPU
===========

http://xenbits.xen.org/people/dariof/benchmarks/results/linux/core-sched/mmtests/boxes/wayrath/coresched-email-4_sysbenchcpu.txt

v v BM BM BM BM BM BM
BM-HT BM-noHT HT noHT csc-HT csc_stallfix-HT csc_tim-HT csc_vruntime-HT
Amean 1 22.55 ( 0.00%) 22.55 * -0.03%* 22.55 ( 0.00%) 22.55 ( -0.02%) 22.56 * -0.04%* 22.56 * -0.03%* 22.55 ( 0.01%) 22.55 ( -0.03%)
Amean 3 7.68 ( 0.00%) 7.69 * -0.07%* 7.68 ( -0.02%) 7.69 * -0.09%* 7.69 * -0.07%* 7.68 ( -0.02%) 7.68 ( -0.02%) 7.68 ( -0.02%)
Amean 5 4.80 ( 0.00%) 5.79 * -20.64%* 4.80 ( 0.00%) 5.79 * -20.73%* 4.81 * -0.24%* 4.80 * -0.18%* 4.81 * -0.24%* 4.81 * -0.24%*
Amean 7 3.59 ( 0.00%) 5.79 * -61.24%* 3.59 ( 0.00%) 5.79 * -61.20%* 3.59 ( -0.08%) 3.60 * -0.16%* 3.62 * -0.92%* 3.61 * -0.52%*
Amean 12 3.18 ( 0.00%) 5.79 * -81.79%* 3.19 ( -0.13%) 5.79 * -81.92%* 3.20 ( -0.49%) 3.19 * -0.27%* 3.27 * -2.78%* 3.41 * -7.04%*
Amean 16 3.19 ( 0.00%) 5.79 * -81.29%* 3.20 ( -0.13%) 5.79 * -81.38%* 3.24 ( -1.52%) 3.20 ( -0.31%) 3.27 * -2.51%* 3.40 * -6.40%*
Stddev 1 0.00 ( 0.00%) 0.01 ( -41.42%) 0.01 ( -82.57%) 0.01 (-194.39%) 0.01 ( -82.57%) 0.01 ( -41.42%) 0.01 (-158.20%) 0.01 (-269.68%)
Stddev 3 0.00 ( 0.00%) 0.01 ( -99.00%) 0.00 ( -99.00%) 0.00 ( -99.00%) 0.01 ( -99.00%) 0.00 ( -99.00%) 0.00 ( -99.00%) 0.00 ( -99.00%)
Stddev 5 0.01 ( 0.00%) 0.02 (-302.08%) 0.01 ( 0.00%) 0.02 (-258.24%) 0.00 ( 8.71%) 0.01 ( -0.00%) 0.01 ( -41.42%) 0.00 ( 8.71%)
Stddev 7 0.00 ( 0.00%) 0.02 ( -99.00%) 0.00 ( 0.00%) 0.02 ( -99.00%) 0.00 ( -99.00%) 0.01 ( -99.00%) 0.01 ( -99.00%) 0.01 ( -99.00%)
Stddev 12 0.01 ( 0.00%) 0.02 (-125.32%) 0.01 ( -54.42%) 0.02 (-103.81%) 0.03 (-321.54%) 0.01 ( -20.89%) 0.03 (-300.00%) 0.03 (-330.56%)
Stddev 16 0.01 ( 0.00%) 0.02 ( -3.28%) 0.01 ( 4.55%) 0.02 ( -44.53%) 0.10 (-591.05%) 0.02 ( -21.11%) 0.02 ( -44.53%) 0.04 (-145.85%)
v v VM VM VM VM VM VM
VM-HT VM-noHT HT noHT csc-HT csc_stallfix-HT csc_tim-HT csc_vruntime-HT
Amean 1 24.38 ( 0.00%) 22.57 * 7.41%* 22.56 * 7.45%* 22.56 * 7.44%* 22.56 * 7.44%* 22.94 * 5.91%* 22.56 * 7.44%* 22.57 * 7.43%*
Amean 3 8.12 ( 0.00%) 7.70 * 5.21%* 7.68 * 5.45%* 7.68 * 5.40%* 7.69 * 5.36%* 7.72 * 4.99%* 7.68 * 5.43%* 7.68 * 5.40%*
Amean 5 4.96 ( 0.00%) 5.85 * -17.88%* 4.79 * 3.43%* 5.85 * -17.91%* 4.81 * 3.02%* 5.09 * -2.68%* 4.82 * 2.94%* 4.80 * 3.25%*
Amean 7 3.62 ( 0.00%) 5.85 * -61.72%* 3.59 * 0.91%* 5.87 * -62.04%* 3.69 ( -1.82%) 3.94 * -8.76%* 3.62 ( 0.12%) 3.60 * 0.59%*
Amean 12 3.18 ( 0.00%) 5.89 * -84.93%* 3.19 ( -0.09%) 5.83 * -83.18%* 3.26 * -2.51%* 3.40 * -6.64%* 3.28 * -3.10%* 3.31 * -3.86%*
Amean 16 3.18 ( 0.00%) 5.86 * -83.89%* 3.18 ( 0.04%) 5.85 * -83.85%* 3.28 * -3.10%* 3.45 * -8.30%* 3.29 * -3.28%* 3.31 * -3.90%*
Stddev 1 0.02 ( 0.00%) 0.01 ( 47.21%) 0.01 ( 66.12%) 0.01 ( 53.84%) 0.01 ( 55.65%) 0.19 (-1012.08%) 0.01 ( 53.84%) 0.01 ( 53.84%)
Stddev 3 0.00 ( 0.00%) 0.02 (-254.96%) 0.00 ( 100.00%) 0.01 ( -61.25%) 0.00 ( 0.00%) 0.06 (-1116.55%) 0.00 ( 22.54%) 0.01 ( -9.54%)
Stddev 5 0.00 ( 0.00%) 0.07 (-1637.81%) 0.00 ( -0.00%) 0.06 (-1587.21%) 0.01 (-255.90%) 0.18 (-4785.35%) 0.01 (-236.65%) 0.01 ( -52.75%)
Stddev 7 0.00 ( 0.00%) 0.10 (-20807867647333972.00%) 0.01 (-1575931584692200.00%) 0.08 (-16887732666966734.00%) 0.15 (-30988869061711636.00%) 0.12 (-24303774169159900.00%) 0.01 (-1640281598669292.00%) 0.01 (-2228703820443910.00%)
Stddev 12 0.01 ( 0.00%) 0.12 (-2069.49%) 0.01 ( -41.42%) 0.08 (-1441.64%) 0.08 (-1396.11%) 0.09 (-1516.07%) 0.03 (-458.27%) 0.03 (-403.32%)
Stddev 16 0.01 ( 0.00%) 0.07 (-1295.23%) 0.00 ( 8.71%) 0.07 (-1235.42%) 0.11 (-1867.23%) 0.26 (-4790.98%) 0.04 (-711.38%) 0.05 (-800.00%)
v v VM VM VM VM VM VM
VM-v4-HT VM-v4-noHT v4-HT v4-noHT v4-csc-HT v4-csc_stallfix-HT v4-csc_tim-HT v4-csc_vruntime-HT
Amean 1 24.37 ( 0.00%) 22.76 * 6.62%* 22.56 * 7.43%* 22.57 * 7.41%* 22.57 * 7.41%* 22.86 * 6.21%* 22.56 * 7.44%* 22.56 * 7.44%*
Amean 3 8.12 ( 0.00%) 7.73 * 4.82%* 7.68 * 5.40%* 7.68 * 5.40%* 7.69 * 5.33%* 7.70 * 5.24%* 7.68 * 5.42%* 7.68 * 5.38%*
Amean 5 6.11 ( 0.00%) 5.90 * 3.53%* 5.79 * 5.35%* 5.78 * 5.51%* 5.79 * 5.33%* 6.07 * 0.77%* 5.78 * 5.44%* 5.77 * 5.58%*
Amean 7 6.11 ( 0.00%) 5.92 * 3.11%* 5.79 * 5.33%* 5.77 * 5.59%* 5.80 * 5.16%* 6.07 * 0.70%* 5.78 * 5.42%* 5.77 * 5.54%*
Amean 8 6.11 ( 0.00%) 5.91 * 3.30%* 5.78 * 5.33%* 5.77 * 5.52%* 5.81 * 4.93%* 6.09 ( 0.28%) 5.78 * 5.45%* 5.77 * 5.52%*
Stddev 1 0.01 ( 0.00%) 0.01 ( 27.45%) 0.01 ( 27.45%) 0.01 ( 17.28%) 0.01 ( -57.28%) 0.08 (-765.72%) 0.01 ( 39.30%) 0.01 ( 27.45%)
Stddev 3 0.00 ( 0.00%) 0.03 (-616.47%) 0.00 ( -29.10%) 0.00 ( -29.10%) 0.01 (-138.05%) 0.01 (-108.17%) 0.00 ( 0.00%) 0.01 ( -41.42%)
Stddev 5 0.01 ( 0.00%) 0.03 (-336.77%) 0.02 (-128.71%) 0.01 ( -75.41%) 0.01 ( -70.97%) 0.02 (-128.71%) 0.01 ( -54.42%) 0.01 ( -59.33%)
Stddev 7 0.01 ( 0.00%) 0.04 (-214.79%) 0.02 ( -82.57%) 0.01 ( 3.08%) 0.01 ( -10.10%) 0.02 ( -52.75%) 0.01 ( 3.08%) 0.01 ( -1.50%)
Stddev 8 0.01 ( 0.00%) 0.04 (-329.84%) 0.02 (-122.54%) 0.01 ( -11.27%) 0.03 (-228.78%) 0.07 (-649.92%) 0.01 ( -38.01%) 0.01 ( -11.27%)
v v VMx2 VMx2 VMx2 VMx2 VMx2 VMx2
VMx2-HT VMx2-noHT HT noHT csc-HT csc_stallfix-HT csc_tim-HT csc_vruntime-HT
Amean 1 22.58 ( 0.00%) 23.41 * -3.67%* 25.22 * -11.70%* 23.36 * -3.48%* 31.57 * -39.84%* 29.30 * -29.78%* 31.82 * -40.93%* 25.60 * -13.38%*
Amean 3 7.69 ( 0.00%) 11.03 * -43.47%* 8.81 * -14.66%* 11.19 * -45.60%* 8.64 * -12.34%* 10.68 * -38.88%* 10.10 * -31.43%* 9.05 * -17.75%*
Amean 5 4.80 ( 0.00%) 11.04 *-130.24%* 5.66 * -17.96%* 11.35 *-136.61%* 6.66 * -38.81%* 7.92 * -65.18%* 6.27 * -30.65%* 6.61 * -37.83%*
Amean 7 3.58 ( 0.00%) 10.94 *-205.34%* 5.44 * -51.79%* 11.53 *-221.93%* 6.39 * -78.35%* 6.23 * -73.96%* 5.65 * -57.70%* 6.07 * -69.30%*
Amean 12 3.18 ( 0.00%) 10.95 *-243.74%* 5.52 * -73.44%* 11.31 *-255.09%* 7.19 *-125.93%* 9.43 *-196.05%* 5.62 * -76.54%* 6.09 * -91.16%*
Amean 16 3.18 ( 0.00%) 11.10 *-248.65%* 5.43 * -70.56%* 11.35 *-256.55%* 6.84 *-115.04%* 7.19 *-125.90%* 5.50 * -72.76%* 6.13 * -92.64%*
Stddev 1 0.01 ( 0.00%) 0.14 (-1403.85%) 0.16 (-1575.99%) 0.09 (-866.82%) 1.41 (-14685.15%) 1.84 (-19279.11%) 1.96 (-20463.33%) 0.38 (-3929.30%)
Stddev 3 0.00 ( 0.00%) 0.51 (-10380.94%) 0.14 (-2749.21%) 0.68 (-13757.63%) 0.30 (-6133.94%) 0.80 (-16343.24%) 0.12 (-2332.69%) 0.16 (-3086.85%)
Stddev 5 0.01 ( 0.00%) 0.30 (-5443.92%) 0.23 (-4280.45%) 0.58 (-10722.66%) 0.21 (-3888.73%) 2.41 (-45040.69%) 0.12 (-2113.22%) 0.16 (-2867.88%)
Stddev 7 0.00 ( 0.00%) 0.40 (-8009.13%) 0.08 (-1480.51%) 0.47 (-9518.94%) 1.04 (-21241.56%) 0.51 (-10409.80%) 0.22 (-4319.28%) 0.25 (-5017.81%)
Stddev 12 0.01 ( 0.00%) 0.53 (-9750.38%) 0.16 (-2879.09%) 0.66 (-12313.77%) 1.29 (-24020.36%) 3.97 (-74080.08%) 0.25 (-4570.12%) 0.32 (-5817.77%)
Stddev 16 0.00 ( 0.00%) 0.46 (-9428.17%) 0.25 (-5124.17%) 0.75 (-15173.77%) 0.78 (-15851.43%) 1.38 (-28142.45%) 0.35 (-7094.72%) 0.21 (-4230.36%)

This is even better than kernbench! :-)

Basically, both on baremetal and in VMs, noHT causes some -81.29%. With
core scheduling, the worst we get is -6.40%.

And that's in the not Virt-overcommitted case. In such case, noHT
brings us down to -248.65% (i.e., we are ~250% slower, it's not that
we're going back in time :-P). Core scheduling contains the damage to
either -72.76% (Tim's patches) or -92.64% (vruntime). While plain v3
is, again, worse than both.

Overhead looks similar to other cases already discussed.

--
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

Attachment: signature.asc
Description: This is a digitally signed message part