Re: [RFC PATCH v3 00/16] Core scheduling v3

From: Dario Faggioli
Date: Tue Oct 29 2019 - 05:18:11 EST

Next message: Greg KH: "Linux 4.4.198"
Previous message: Jarkko Sakkinen: "Re: [PATCH v2] tpm: Add major_version sysfs file"
In reply to: Dario Faggioli: "Re: [RFC PATCH v3 00/16] Core scheduling v3"
Next in thread: Dario Faggioli: "Re: [RFC PATCH v3 00/16] Core scheduling v3"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

On Tue, 2019-10-29 at 10:11 +0100, Dario Faggioli wrote:
> On Sun, 2019-09-15 at 22:14 +0800, Aaron Lu wrote:
> > I'm using the following branch as base which is v5.1.5 based:
> > https://github.com/digitalocean/linux-coresched coresched-v3-
> > v5.1.5-
> > test
> >
> > And I have pushed Tim's branch to:
> > https://github.com/aaronlu/linux coresched-v3-v5.1.5-test-tim
> >
> > Mine:
> > https://github.com/aaronlu/linux coresched-v3-v5.1.5-test-
> > core_vruntime
> >
> Hello,
>
> As anticipated, I've been trying to follow the development of this
> feature and, in the meantime, I have done some benchmarks.
>
> I actually have a lot of data (and am planning for more), so I am
> sending a few emails, each one with a subset of the numbers in it,
> instead than just one which would be beyond giant! :-)
>
SYSBENCHTHREAD
==============

http://xenbits.xen.org/people/dariof/benchmarks/results/linux/core-sched/mmtests/boxes/wayrath/coresched-email-5_sysbenchthread.txt

v v BM BM BM BM BM BM
BM-HT BM-noHT HT noHT csc-HT csc_stallfix-HT csc_tim-HT csc_vruntime-HT
Amean 1 1.12 ( 0.00%) 1.11 ( 0.38%) 1.14 * -2.43%* 1.15 * -2.81%* 8.49 *-659.72%* 8.43 *-654.99%* 8.42 *-653.96%* 8.38 *-649.74%*
Amean 3 1.11 ( 0.00%) 1.11 ( 0.00%) 1.14 * -2.70%* 1.14 * -2.96%* 8.35 *-651.29%* 8.42 *-657.20%* 8.34 *-650.77%* 8.41 *-656.56%*
Amean 5 1.11 ( 0.00%) 1.11 ( 0.00%) 1.14 * -2.70%* 1.14 * -2.82%* 8.34 *-649.81%* 8.41 *-655.46%* 8.32 *-647.75%* 8.34 *-649.42%*
Amean 7 1.12 ( 0.00%) 1.11 ( 0.38%) 1.14 * -2.43%* 1.15 * -2.69%* 8.31 *-643.48%* 8.40 *-652.05%* 8.41 *-653.20%* 8.33 *-645.65%*
Amean 12 1.12 ( 0.00%) 1.12 ( 0.00%) 1.14 * -2.43%* 1.14 * -2.56%* 8.38 *-651.34%* 8.41 *-654.16%* 8.32 *-645.71%* 8.38 *-651.09%*
Amean 16 1.11 ( 0.00%) 1.11 ( -0.13%) 1.15 * -3.08%* 1.14 * -2.57%* 8.42 *-656.61%* 8.36 *-651.60%* 8.31 *-646.73%* 8.39 *-654.30%*
Stddev 1 0.01 ( 0.00%) 0.01 ( 49.47%) 0.01 ( 64.27%) 0.01 ( 39.86%) 0.08 (-414.47%) 0.16 (-976.34%) 0.01 ( 36.42%) 0.09 (-525.69%)
Stddev 3 0.00 ( 0.00%) 0.00 ( 0.00%) 0.00 ( 0.00%) 0.01 (-108.17%) 0.13 (-3211.60%) 0.10 (-2531.86%) 0.06 (-1515.55%) 0.10 (-2467.10%)
Stddev 5 0.00 ( 0.00%) 0.00 ( 0.00%) 0.00 ( 0.00%) 0.01 ( -9.54%) 0.12 (-2271.92%) 0.10 (-1917.42%) 0.09 (-1700.00%) 0.09 (-1725.38%)
Stddev 7 0.01 ( 0.00%) 0.01 ( 20.53%) 0.01 ( 43.80%) 0.01 ( 0.00%) 0.13 (-1255.69%) 0.10 (-988.21%) 0.03 (-247.93%) 0.08 (-762.68%)
Stddev 12 0.01 ( 0.00%) 0.01 ( -24.03%) 0.00 ( 37.98%) 0.01 ( 0.00%) 0.13 (-1577.68%) 0.10 (-1210.31%) 0.10 (-1140.97%) 0.06 (-644.73%)
Stddev 16 0.00 ( 0.00%) 0.01 ( -9.54%) 0.01 ( -54.92%) 0.00 ( 22.54%) 0.15 (-3032.73%) 0.12 (-2401.20%) 0.11 (-2206.51%) 0.07 (-1304.28%)
v v VM VM VM VM VM VM
VM-HT VM-noHT HT noHT csc-HT csc_stallfix-HT csc_tim-HT csc_vruntime-HT
Amean 1 1.43 ( 0.00%) 1.15 * 19.56%* 1.15 * 19.36%* 1.19 * 16.67%* 1.16 * 19.16%* 1.17 * 18.46%* 1.16 * 18.66%* 1.16 * 18.86%*
Amean 3 1.43 ( 0.00%) 1.16 * 19.16%* 1.16 * 19.26%* 1.17 * 18.46%* 1.16 * 19.16%* 1.21 * 15.17%* 1.16 * 18.86%* 1.21 * 15.27%*
Amean 5 1.43 ( 0.00%) 1.15 * 19.28%* 1.18 * 17.78%* 1.17 * 18.08%* 1.16 * 19.18%* 1.17 * 18.18%* 1.16 * 19.08%* 1.18 * 17.58%*
Amean 7 1.43 ( 0.00%) 1.16 * 19.26%* 1.16 * 18.86%* 1.17 * 18.56%* 1.16 * 19.16%* 1.19 * 16.57%* 1.15 * 19.36%* 1.15 * 19.36%*
Amean 12 1.44 ( 0.00%) 1.16 * 19.44%* 1.16 * 19.44%* 1.17 * 18.75%* 1.16 * 19.25%* 1.18 * 18.06%* 1.16 * 19.15%* 1.16 * 19.44%*
Amean 16 1.46 ( 0.00%) 1.16 * 20.37%* 1.16 * 20.37%* 1.16 * 20.47%* 1.16 * 20.37%* 1.17 * 19.59%* 1.17 * 20.08%* 1.16 * 20.37%*
Stddev 1 0.00 ( 0.00%) 0.00 ( 0.00%) 0.01 ( -41.42%) 0.05 (-1314.21%) 0.00 ( -29.10%) 0.00 ( -29.10%) 0.01 (-236.65%) 0.02 (-316.33%)
Stddev 3 0.00 ( 0.00%) 0.00 ( -29.10%) 0.01 (-108.17%) 0.03 (-773.69%) 0.01 (-100.00%) 0.09 (-2324.18%) 0.01 (-138.05%) 0.06 (-1378.74%)
Stddev 5 0.00 ( 0.00%) 0.01 ( -99.00%) 0.05 ( -99.00%) 0.02 ( -99.00%) 0.01 ( -99.00%) 0.01 ( -99.00%) 0.01 ( -99.00%) 0.04 ( -99.00%)
Stddev 7 0.00 ( 0.00%) 0.01 ( -41.42%) 0.01 (-138.05%) 0.02 (-468.62%) 0.01 (-100.00%) 0.06 (-1600.00%) 0.01 (-108.17%) 0.01 ( -41.42%)
Stddev 12 0.00 ( 0.00%) 0.00 ( 100.00%) 0.00 ( 100.00%) 0.03 (-11031521092846084.00%) 0.00 (-2034518927425100.00%) 0.01 (-4169523056347680.00%) 0.01 (-2228703820443891.00%) 0.00 ( 100.00%)
Stddev 16 0.05 ( 0.00%) 0.00 ( 92.31%) 0.00 ( 92.31%) 0.00 ( 100.00%) 0.00 ( 92.31%) 0.01 ( 80.64%) 0.01 ( 89.12%) 0.00 ( 92.31%)
v v VM VM VM VM VM VM
VM-v4-HT VM-v4-noHT v4-HT v4-noHT v4-csc-HT v4-csc_stallfix-HT v4-csc_tim-HT v4-csc_vruntime-HT
Amean 1 1.43 ( 0.00%) 1.17 * 18.15%* 1.15 * 19.64%* 1.15 * 19.44%* 1.16 * 19.14%* 1.17 * 18.34%* 1.16 * 18.94%* 1.16 * 18.94%*
Amean 3 1.43 ( 0.00%) 1.17 * 18.33%* 1.17 * 18.73%* 1.15 * 19.52%* 1.15 * 19.52%* 1.19 * 16.93%* 1.16 * 19.12%* 1.16 * 19.32%*
Amean 5 1.43 ( 0.00%) 1.18 * 17.45%* 1.15 * 19.44%* 1.15 * 19.54%* 1.17 * 18.15%* 1.18 * 17.55%* 1.15 * 19.44%* 1.16 * 19.04%*
Amean 7 1.43 ( 0.00%) 1.18 * 17.53%* 1.16 * 19.32%* 1.15 * 19.72%* 1.17 * 18.33%* 1.17 * 18.13%* 1.16 * 18.92%* 1.15 * 19.62%*
Amean 8 1.43 ( 0.00%) 1.17 * 18.46%* 1.16 * 19.16%* 1.16 * 18.96%* 1.17 * 18.46%* 1.18 * 17.86%* 1.15 * 19.36%* 1.16 * 19.06%*
Stddev 1 0.00 ( 0.00%) 0.00 ( 0.00%) 0.00 ( 22.54%) 0.01 ( -9.54%) 0.01 (-119.09%) 0.01 ( -67.33%) 0.01 (-149.00%) 0.01 ( -84.39%)
Stddev 3 0.01 ( 0.00%) 0.01 ( 12.29%) 0.02 (-105.69%) 0.01 ( 0.00%) 0.01 ( 0.00%) 0.05 (-515.82%) 0.01 ( -27.10%) 0.01 ( -41.42%)
Stddev 5 0.00 ( 0.00%) 0.04 (-700.00%) 0.01 ( -61.25%) 0.01 ( -54.92%) 0.04 (-717.31%) 0.01 ( -84.39%) 0.01 ( -61.25%) 0.01 ( -18.32%)
Stddev 7 0.01 ( 0.00%) 0.04 (-390.68%) 0.01 ( 3.92%) 0.00 ( 51.96%) 0.04 (-366.58%) 0.01 ( 0.00%) 0.02 (-151.15%) 0.01 ( 3.92%)
Stddev 8 0.00 ( 0.00%) 0.00 ( -29.10%) 0.01 (-151.66%) 0.02 (-383.05%) 0.05 (-1100.00%) 0.01 (-108.17%) 0.01 ( -41.42%) 0.01 (-138.05%)
v v VMx2 VMx2 VMx2 VMx2 VMx2 VMx2
VMx2-HT VMx2-noHT HT noHT csc-HT csc_stallfix-HT csc_tim-HT csc_vruntime-HT
Amean 1 1.16 ( 0.00%) 1.22 * -5.80%* 1.54 * -32.84%* 1.22 * -5.31%* 2.83 (-144.32%) 1.35 * -16.42%* 1.69 * -46.30%* 1.35 * -16.91%*
Amean 3 1.16 ( 0.00%) 1.24 * -6.90%* 1.56 * -34.11%* 1.24 * -6.53%* 1.89 * -63.05%* 1.95 * -68.35%* 1.67 * -43.84%* 1.33 * -14.41%*
Amean 5 1.17 ( 0.00%) 1.21 * -3.79%* 1.52 * -30.48%* 1.23 * -5.75%* 1.43 * -22.89%* 1.85 ( -58.51%) 1.71 * -46.88%* 1.35 * -15.91%*
Amean 7 1.15 ( 0.00%) 1.23 * -6.81%* 1.54 * -33.29%* 1.24 * -7.05%* 1.68 * -45.79%* 1.39 * -20.79%* 2.09 * -80.69%* 1.30 * -12.62%*
Amean 12 1.16 ( 0.00%) 1.25 * -7.25%* 1.69 * -45.09%* 1.22 * -5.28%* 1.72 * -47.54%* 1.52 * -31.08%* 2.08 * -78.87%* 1.32 * -13.14%*
Amean 16 1.16 ( 0.00%) 1.23 * -6.14%* 1.68 * -44.23%* 1.22 * -4.67%* 1.80 * -54.91%* 1.37 * -17.44%* 2.18 * -87.10%* 1.28 * -9.95%*
Stddev 1 0.01 ( 0.00%) 0.05 (-569.58%) 0.14 (-1693.97%) 0.02 (-169.26%) 3.19 (-42121.82%) 0.25 (-3271.57%) 0.26 (-3362.06%) 0.09 (-1072.60%)
Stddev 3 0.01 ( 0.00%) 0.06 (-664.85%) 0.06 (-630.95%) 0.03 (-259.56%) 0.84 (-10177.51%) 0.98 (-11852.56%) 0.45 (-5404.28%) 0.08 (-845.29%)
Stddev 5 0.00 ( 0.00%) 0.02 (-393.96%) 0.17 (-3371.31%) 0.07 (-1423.81%) 0.32 (-6361.11%) 1.04 (-21129.08%) 0.22 (-4383.53%) 0.06 (-1033.14%)
Stddev 7 0.01 ( 0.00%) 0.07 (-1234.79%) 0.20 (-3727.10%) 0.08 (-1483.25%) 0.57 (-10594.70%) 0.19 (-3455.98%) 0.30 (-5454.88%) 0.08 (-1400.56%)
Stddev 12 0.00 ( 0.00%) 0.08 (-1641.84%) 0.16 (-3078.68%) 0.03 (-524.50%) 0.50 (-10161.87%) 0.08 (-1634.36%) 0.32 (-6423.80%) 0.07 (-1232.67%)
Stddev 16 0.00 ( 0.00%) 0.05 (-996.36%) 0.20 (-3958.82%) 0.06 (-1045.43%) 0.60 (-12116.06%) 0.23 (-4642.99%) 0.23 (-4594.04%) 0.05 (-842.34%)

This is all quite bizarre. Judging from these numbers, the benchmark
seems to be no sensitive to HT at all (while, e.g., sysbenchcpu was),
when on baremetal. In VMs, at least in most cases, things are
significantly *faster* with HT off. Which I know is something that can
happen, I just was not expecting it from this workload.

Also, core scheduling is a total disaster on baremetal. And its
behavior in VM is an anti-pattern too.

So, I guess I'll go trying to see if I did something wrong when
configuring or running this particular benchmark. If that is not the
case, then core scheduling has a serious issue when dealing with
threads instead than with processes, I would say.

--
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

Attachment: signature.asc
Description: This is a digitally signed message part

Next message: Greg KH: "Linux 4.4.198"
Previous message: Jarkko Sakkinen: "Re: [PATCH v2] tpm: Add major_version sysfs file"
In reply to: Dario Faggioli: "Re: [RFC PATCH v3 00/16] Core scheduling v3"
Next in thread: Dario Faggioli: "Re: [RFC PATCH v3 00/16] Core scheduling v3"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]