Re: sysbench throughput degradation in 4.13+

[Matt Fleming]

On Wed, 04 Oct, at 06:18:50PM, Peter Zijlstra wrote:
> On Tue, Oct 03, 2017 at 10:39:32AM +0200, Peter Zijlstra wrote:
> > So I was waiting for Rik, who promised to run a bunch of NUMA workloads
> > over the weekend.
> > 
> > The trivial thing regresses a wee bit on the overloaded case, I've not
> > yet tried to fix it.
> 
> WA_IDLE is my 'old' patch and what you all tested, WA_WEIGHT is the
> addition -- based on the old scheme -- that I've tried in order to lift
> the overloaded case (including hackbench).

My results (2 nodes, 12 cores/node, 2 threads/core) show that you've
pretty much restored hackbench performance to v4.12. However, it's a
regression against v4.13 for hackbench-process-pipes (I'm guessing the
v4.13 improvement is due to Rik's original patches).

The last two result columns are your latest patch with NO_WA_WEIGHT
and then with WA_WEIGHT enabled.

(I hope you've all got wide terminals)

hackbench-process-pipes
                              4.12.0                 4.13.0                 4.13.0                 4.13.0                 4.13.0                 4.13.0
                             vanilla                vanilla             peterz-fix                rik-fixpeterz-fix-v2-no-wa-weightpeterz-fix-v2-wa-weight
Amean     1        1.1600 (   0.00%)      1.6037 ( -38.25%)      1.0727 (   7.53%)      1.0200 (  12.07%)      1.0837 (   6.58%)      1.1110 (   4.22%)
Amean     4        2.4207 (   0.00%)      2.2300 (   7.88%)      2.0520 (  15.23%)      1.9483 (  19.51%)      2.0623 (  14.80%)      2.2807 (   5.78%)
Amean     7        5.4140 (   0.00%)      3.2027 (  40.84%)      3.6100 (  33.32%)      3.5620 (  34.21%)      3.5590 (  34.26%)      4.6573 (  13.98%)
Amean     12       9.7130 (   0.00%)      4.7170 (  51.44%)      6.5280 (  32.79%)      6.2063 (  36.10%)      6.5670 (  32.39%)     10.5440 (  -8.56%)
Amean     21      11.6687 (   0.00%)      8.8073 (  24.52%)     14.4997 ( -24.26%)     10.2700 (  11.99%)     14.3187 ( -22.71%)     11.5417 (   1.09%)
Amean     30      14.6410 (   0.00%)     11.7003 (  20.09%)     23.7660 ( -62.32%)     13.9847 (   4.48%)     21.8957 ( -49.55%)     14.4847 (   1.07%)
Amean     48      19.8780 (   0.00%)     17.0317 (  14.32%)     37.6397 ( -89.35%)     19.7577 (   0.61%)     39.2110 ( -97.26%)     20.3293 (  -2.27%)
Amean     79      46.4200 (   0.00%)     27.1180 (  41.58%)     58.4037 ( -25.82%)     35.5537 (  23.41%)     60.8957 ( -31.18%)     49.7470 (  -7.17%)
Amean     110     57.7550 (   0.00%)     42.7013 (  26.06%)     73.0483 ( -26.48%)     48.8880 (  15.35%)     77.8597 ( -34.81%)     61.9353 (  -7.24%)
Amean     141     61.0490 (   0.00%)     48.0857 (  21.23%)     98.5567 ( -61.44%)     63.2187 (  -3.55%)     90.4857 ( -48.22%)     68.3100 ( -11.89%)
Amean     172     70.5180 (   0.00%)     59.3620 (  15.82%)    122.5423 ( -73.77%)     76.0197 (  -7.80%)    127.4023 ( -80.67%)     75.8233 (  -7.52%)
Amean     192     76.1643 (   0.00%)     65.1613 (  14.45%)    142.1393 ( -86.62%)     91.4923 ( -20.12%)    145.0663 ( -90.46%)     80.5867 (  -5.81%)

But things look pretty good for hackbench-process-sockets:

hackbench-process-sockets
                              4.12.0                 4.13.0                 4.13.0                 4.13.0                 4.13.0                 4.13.0
                             vanilla                vanilla             peterz-fix                rik-fixpeterz-fix-v2-no-wa-weightpeterz-fix-v2-wa-weight
Amean     1        0.9657 (   0.00%)      1.0850 ( -12.36%)      1.3737 ( -42.25%)      1.3093 ( -35.59%)      1.3220 ( -36.90%)      1.3937 ( -44.32%)
Amean     4        2.3040 (   0.00%)      3.3840 ( -46.88%)      2.1807 (   5.35%)      2.3010 (   0.13%)      2.2070 (   4.21%)      2.1770 (   5.51%)
Amean     7        4.5467 (   0.00%)      4.0787 (  10.29%)      5.0530 ( -11.14%)      3.7427 (  17.68%)      4.5517 (  -0.11%)      3.8560 (  15.19%)
Amean     12       5.7707 (   0.00%)      5.4440 (   5.66%)     10.5680 ( -83.13%)      7.7240 ( -33.85%)     10.5990 ( -83.67%)      5.9123 (  -2.45%)
Amean     21       8.9387 (   0.00%)      9.5850 (  -7.23%)     18.3103 (-104.84%)     10.9253 ( -22.23%)     18.1540 (-103.10%)      9.2627 (  -3.62%)
Amean     30      13.1243 (   0.00%)     14.0773 (  -7.26%)     25.6563 ( -95.49%)     15.7590 ( -20.07%)     25.6920 ( -95.76%)     14.6523 ( -11.64%)
Amean     48      25.1030 (   0.00%)     22.5233 (  10.28%)     40.5937 ( -61.71%)     24.6727 (   1.71%)     40.6357 ( -61.88%)     22.1807 (  11.64%)
Amean     79      39.9150 (   0.00%)     33.4220 (  16.27%)     66.3343 ( -66.19%)     40.2713 (  -0.89%)     65.8543 ( -64.99%)     35.3360 (  11.47%)
Amean     110     49.1700 (   0.00%)     46.1173 (   6.21%)     92.3153 ( -87.75%)     55.6567 ( -13.19%)     92.0567 ( -87.22%)     46.7280 (   4.97%)
Amean     141     59.3157 (   0.00%)     57.2670 (   3.45%)    118.5863 ( -99.92%)     70.4800 ( -18.82%)    118.6013 ( -99.95%)     57.8247 (   2.51%)
Amean     172     69.8163 (   0.00%)     68.2817 (   2.20%)    145.7583 (-108.77%)     83.0167 ( -18.91%)    144.4477 (-106.90%)     68.4457 (   1.96%)
Amean     192     75.9913 (   0.00%)     76.0503 (  -0.08%)    159.8487 (-110.35%)     91.0133 ( -19.77%)    159.6793 (-110.13%)     76.2690 (  -0.37%)

It's a similar story for hackbench-threads-{pipes,sockets}, i.e. pipes
regress but performance is restored for sockets.

Of course, like a dope, I forgot to re-run netperf with your WA_WEIGHT
patch. So I've queued that up now and it should be done by tomorrow.