Re: [PATCH v10 00/12] sched, steal_governor: Introduce preferred CPUs and steal-driven vCPU backoff

[Yury Norov]

> ========================================
> Performance Numbers (powerpc, x86, s390)
> ========================================
> 
> PowerPC:
> ===================
> VM1: 60VP/30EC and VM2: 30VP/20EC
> Shared physical CPU pool size: 50 Cores. Each core is SMT8.
> (VP - Virtual Core, EC - Entitles Core) -  PowerVM terminologies of SPLPAR[2]
> 
> Default parameter values: 1000ms, 200 low threshold, 500 high threshold
> Both the VMs are running the same workload. Total throughput/time of VM1+VM2
> is being mentioned in all cases.
> 
> Hackbench
>               baseline    steal_governor        steal_governor
>                              disabled               enabled
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> 
> 10 groups        5.20   |    5.40 (-3.85%)  |     4.65 (+10.58%)
> 20 groups       11.39   |   12.01 (-5.44%)  |     7.09 (+37.75%)
> 40 groups       20.32   |   19.80 (+2.56%)  |    11.31 (+44.34%)
> 10 groups(-p)    2.37   |    2.26 (+4.64%)  |     2.06 (+13.08%)
> 20 groups(-p)    3.34   |    3.28 (+1.80%)  |     3.20 (+4.19%)
> 40 groups(-p)    4.46   |    4.83 (-8.30%)  |     4.26 (+4.48%)
> Remarks: Net improvement with steal_governor specially high load points.
> 
> schbench ( -L -n 0 -r 30 -s 0)
>               baseline    steal_governor          steal_governor
>                              disabled                 enabled
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> -m 1 -t 128     2475162 |    2621246 (+5.90%)  |      2527299 (+2.11%)
> -m 1 -t 256     1467350 |    1470032 (+0.18%)  |      1492372 (+1.71%)
> -m 1 -t 512     1408813 |    1454687 (+3.26%)  |      1437605 (+2.04%)
> Remarks: Effectively means no-improvements or regressions
> 
> kernbench	baseline    steal_governor     steal_governor
> (elapsed time)	               disabled            enabled
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> -j nr_cpus	231      |      235 (-1.7%) |    199 (+14%)
> Remarks: Net improvement in elapsed time.
> 
> Daytrader - A real life work which is a proxy for trading based
> on db2[3]
>               baseline      steal_governor   steal_governor
>                               disabled          enabled
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Load@30%	1x	|	0.96x	|	 1.53x			
> Load@60%	1x	|	0.94x	|	 1.41x
> Remarks: Good improvement seen at different load points.
> 
> When there is no steal time (such as dedicated LPAR, or only VM2
> is running) throughput was same with steal_governor enabled/disabled
> which indicates minimal overhead of steal_governor. 
> 
> I have run v10 also on a smaller powerpc LPAR system and it shows
> good improvements.
> 
> =======================================================================
> 
> Data from x86,s390 KVM which Ilya Leoshkevich carried out during OSPM26
> time. *This was based on v2*. Idea is still the name, numbers are
> expected to be better in v10 as some of the overhead has been removed.
> Note: Other variations of the benchmark shows no observable
> difference.
> 
> x86:
> ====
> cascade-lake: 32 threads = 16 cores
> Benchmark      #VMs    #CPUs/VM  ΔRPS     (%std)
> ===============================================
> hackbench         8          16  90.73% ± 9.97%
> hackbench         4          24  52.67% ± 7.43%
> hackbench         4          16  37.96% ± 11.19%
> hackbench         4          32  37.82% ± 4.38%
> hackbench        12           8  36.90% ± 4.74%
> hackbench         8           8  35.30% ± 3.61%
> pgbench          16           4  31.77% ± 2.44%
> hackbench         2          24  25.85% ± 8.63%
> hackbench        16           8  24.87% ± 3.46%
> pgbench          16           8  21.83% ± 2.20%
> pgbench          12           8  21.35% ± 2.15%
> pgbench           8           8  18.46% ± 1.01%
> hackbench         2          32  15.56% ± 4.53%
> pgbench          12           4  14.28% ± 2.04%
> hackbench        16           4  14.07% ± 2.90%
> hackbench        12           4  9.60% ± 3.49%
> [...]
> pgbench           4           8  -1.16% ± 3.60%
> hackbench         4           4  -1.80% ± 9.55%
> sysbench         12           4  -2.19% ± 0.78%
> pgbench           4          24  -2.43% ± 4.38%
> pgbench           4          32  -3.21% ± 0.79%
> sysbench         16           4  -3.22% ± 1.09%
> 
> S390:
> =====
> z16: 16 threads = 8 cores (SMT-2)
> Benchmark      #VMs    #CPUs/VM  ΔRPS    (std%)
> ===============================================
> pgbench           2           8  73.50% ± 35.91%
> pgbench          16           4  61.30% ± 4.09%
> hackbench        16           4  54.11% ± 4.38%
> hackbench        12           4  36.34% ± 4.63%
> pgbench          12           4  34.83% ± 2.57%
> hackbench         8           4  29.75% ± 5.86%
> hackbench         8           8  25.98% ± 5.09%
> pgbench           2           4  23.31% ± 33.44%
> pgbench           2          16  19.95% ± 17.12%
> hackbench         4           8  19.43% ± 9.33%
> pgbench           8           4  19.32% ± 4.50%
> [...]
> schbench          8           8  -0.79% ± 0.33%
> sysbench          8           8  -0.81% ± 0.39%
> hackbench         4          16  -1.11% ± 5.82%
> sysbench          8           4  -1.62% ± 0.49%
> sysbench         16           4  -2.70% ± 0.58%
> schbench         16           4  -2.73% ± 0.91%
> sysbench         12           4  -2.91% ± 0.61%
> hackbench         2          24  -4.99% ± 3.31%
> 
> Summary:
> - Many improvement across archs specially with real life workloads.
> - No major regressions observed.
> - Overhead of steal_governor looks minimal when there is no steal time.
> - Overhead when STEAL_GOVERNOR=n is negligible.

OK, I gave it some testing on my laptop.

The results are pretty consistent: the steal ratio is converged to a
number withing the threshold, but the overall performance is 3-5% worse
comparing to baseline. I tried 2-5% and 1.5-15% boundaries.

It's 4 VMs, each running 8 vCPUs on 8 pCPU machine, the payload is
running for 2 minutes.

  Steal governor off:
  VM         THROUGHPUT       AVG STEAL%
  -------- ------------ ----------------
  0               13802            74.59
  1               11929            76.35
  2               13470            74.61
  3               10259            78.67
  -------- ------------ ----------------
  TOTAL           49460                -
  
  Steal governor on:
  VM         THROUGHPUT       AVG STEAL%
  -------- ------------ ----------------
  0               12921             5.72
  1               12762             6.11
  2               10160             5.99
  3               12098             6.00
  -------- ------------ ----------------
  TOTAL           47941                -


The test is attached below. The results are quite differ from the numbers
above, so maybe I misconfigured something? I didn't use hackbench or
similar benchmarks, just a basic math.

Shrikanth, can you please check my test and results? Is there something 
that I have missed?

I think this series should include some testing. The scripts below look
bulky and they depend on virtme, but they allow to build the proper
kernel and run tests with a single command.

Thanks,
Yury