Re: [PATCH v3 0/7] sched: Flatten the pick

[Szabina Korbai]

Hello Peter,

We ran the same benchmarks (schbench, sysbench, hackbench) as Shubhang
has on s390 on an LPAR running fedora 43 with 32 vCPUs.

We ran the benchmarks for each of the cgroup modes, and for the
baseline, we chose the commit prior to the patches (f666241e6bd5 -
sched/fair: Unify cfs_rq throttling via account_cfs_rq_runtime() ).

We have also tried running stress-ng in parallel with the benchmarks
(set to generate 50% or 90% utilization for each vCPU).

Compared to simply running the benchmarks on their own, this has
revealed some performance trade-offs that the move to a single runqueue
can introduce.

=======================================================================
  HACKBENCH (via phoronix-test-suite pts/hackbench)
=======================================================================

The hackbench results were quite consistent, CV was under 2.5% in most
cases, a '*' marks the cases where they weren't.

What we've found is that if there is no other workload running, the
results were generally favorable, especially for higher number of
threads/processes.

However, with stress-ng also running in parallel, while the high
thread/process count cases showed even greater improvement, the lower-
count cases actually started to regress, which got worse at higher CPU
utilization.

To compound this problem, the stress-ng results also showed regression
(the stress-ng figures were recorded over the whole hackbench stress-ng
run for a specific mode, so currently there is no higher granularity
data for the 32 process case for example).

(lower = better)

Hackbench % diff from baseline by mode:

[s-00] Hackbench — % diff from baseline by mode
+-------------------+---------+---------+---------+---------+---------+
| Hackbench arg     | concur  | max     | smp     | tasks   | up      |
+-------------------+---------+---------+---------+---------+---------+
| 1 thread          | -0.40%  | +0.81%  | +0.40%  | +0.37%  | -2.82%  |
| 2 thread          | -0.50%  | -0.96%  | -0.65%  | -1.68%  | -1.55%  |
| 4 thread          | -3.67%  | -3.95%  | -4.27%  | -3.67%  | -5.38%  |
| 8 thread          | -6.21%  | -6.62%  | -6.86%  | -6.30%  | -9.55%  |
| 16 thread         | -8.90%  | -9.09%  | -9.21%  | -8.58%  | -8.39%  |
| 32 thread         | -6.67%  | -6.57%  | -6.84%  | -6.25%  | -5.64%  |
| 1 process         | +1.41%  | +0.84%  | +0.19%  | +2.36%  | -3.50%  |
| 2 process         | +0.23%  | +0.91%  | +0.00%  | +1.49%  | -0.75%  |
| 4 process         | -3.21%  | -2.98%  | -3.54%  | -3.31%  | -5.57%  |
| 8 process         | -5.55%  | -5.40%  | -5.53%  | -5.56%  | -8.47%  |
| 16 process        | -7.77%  | -7.75%  | -8.33%  | -7.47%  | -7.89%  |
| 32 process        | -7.44%  | -6.92%  | -7.19%  | -4.76%  | -6.89%  |
+-------------------+---------+---------+---------+---------+---------+


[s-50] Hackbench — % diff from baseline by mode
+------------------+---------+---------+---------+---------+----------+
| Hackbench arg    | concur  | max     | smp     | tasks   | up       |
+------------------+---------+---------+---------+---------+----------+
| 1 thread         | +5.73%  | +0.98%  | +0.65%  | +0.77%  | -0.46%   |
| 2 thread         | +3.24%  | -1.82%  | -2.04%  | -2.11%  | -0.65%   |
| 4 thread         | -2.51%  | -4.50%  | -4.49%  | -4.18%  | -8.75%   |
| 8 thread         | -9.29%  | -18.48% | -18.05% | -18.33% | -21.06%  |
| 16 thread        | -14.77% | -35.28% | -34.42% | -35.24% | -28.20%* |
| 32 thread        | -5.79%  | -39.03% | -38.58% | -39.16% | -27.75%  |
| 1 process        | +6.35%  | +1.20%  | +1.01%  | +0.87%  | +0.34%   |
| 2 process        | +3.91%  | -1.87%  | -1.90%  | -2.26%  | -0.54%   |
| 4 process        | -1.71%  | -3.54%  | -3.95%  | -3.69%  | -7.42%   |
| 8 process        | -8.96%  | -17.83% | -17.35% | -17.82% | -21.43%  |
| 16 process       | -14.86% | -34.80% | -33.98% | -35.37% | -30.54%  |
| 32 process       | -6.72%  | -38.85% | -38.81% | -39.50% | -33.24%  |
+------------------+---------+---------+---------+---------+----------+


[s-90] Hackbench — % diff from baseline by mode
+-----------------+----------+---------+---------+---------+----------+
| Hackbench arg   | concur   | max     | smp     | tasks   | up       |
+-----------------+----------+---------+---------+---------+----------+
| 1 thread        | +17.81%  | +18.24% | +17.29% | +17.40% | +31.11%  |
| 2 thread        | +5.81%   | +5.43%  | +6.34%  | +5.59%  | +11.62%  |
| 4 thread        | -8.69%   | -8.65%  | -7.40%  | -8.80%  | -5.85%   |
| 8 thread        | -22.75%  | -22.69% | -21.47% | -22.88% | -23.04%  |
| 16 thread       | -37.81%  | -37.71% | -36.49% | -37.84% | -28.13%  |
| 32 thread       | -39.75%  | -39.97% | -39.45% | -40.06% | -32.54%  |
| 1 process       | +23.63%* | +22.55% | +22.17% | +21.17% | +34.84%  |
| 2 process       | +7.54%   | +7.44%  | +7.34%  | +7.47%  | +13.74%  |
| 4 process       | -6.61%   | -6.87%  | -5.53%  | -6.45%  | -4.03%   |
| 8 process       | -21.92%  | -21.70% | -20.40% | -21.79% | -21.90%  |
| 16 process      | -37.17%  | -36.92% | -35.98% | -37.31% | -30.01%* |
| 32 process      | -39.20%  | -38.32% | -39.03% | -39.19% | -25.16%* |
+-----------------+----------+---------+---------+---------+----------+

stress-ng bogo-ops/s statistics by stress level (Hackbench):

(higher = better)

[s-50] Hackbench — stress-ng vs baseline
+----------+------------+---------+-------+
| Variant  | Mean %diff | StdDev± | CV    |
+----------+------------+---------+-------+
| concur   | -3.35%     | ±0.15%  | 0.15% |
| max      | -29.48%    | ±0.75%  | 0.75% |
| smp      | -29.57%    | ±0.68%  | 0.68% |
| tasks    | -30.01%    | ±0.42%  | 0.42% |
| up       | -24.68%    | ±2.27%  | 2.26% |
+----------+------------+---------+-------+


[s-90] Hackbench — stress-ng vs baseline
+----------+------------+---------+-------+
| Variant  | Mean %diff | StdDev± | CV    |
+----------+------------+---------+-------+
| concur   | -13.55%    | ±0.23%  | 0.23% |
| max      | -20.19%    | ±0.44%  | 0.44% |
| smp      | -19.72%    | ±0.57%  | 0.57% |
| tasks    | -20.61%    | ±0.33%  | 0.33% |
| up       | -25.52%*   | ±5.86%  | 5.86% |
+----------+------------+---------+-------+


=======================================================================
  SYSBENCH (via phoronix-test-suite ciunas/sysbench) - throughput
=======================================================================

The sysbench results showed small improvements, but more interesting
were the stress-ng results, showing regressions across all modes at 50%
CPU utilization, but improvements at 90%.

RAM/Memory testcase

(higher = better)

Sysbench % diff from baseline by mode:

[s-00] Sysbench — % diff from baseline by mode
+-------------------+---------+---------+---------+---------+---------+
|                   | concur  | max     | smp     | tasks   | up      |
+-------------------+---------+---------+---------+---------+---------+
| memory run        | -3.91%  | +0.02%  | -4.05%  | +0.44%  | -4.11%  |
+-------------------+---------+---------+---------+---------+---------+


[s-50] Sysbench — % diff from baseline by mode
+-------------------+---------+---------+---------+---------+---------+
|                   | concur  | max     | smp     | tasks   | up      |
+-------------------+---------+---------+---------+---------+---------+
| memory run        | +2.84%  | +1.09%  | +0.73%  | +2.62%  | +1.91%  |
+-------------------+---------+---------+---------+---------+---------+


[s-90] Sysbench — % diff from baseline by mode
+-------------------+---------+---------+---------+---------+---------+
|                   | concur  | max     | smp     | tasks   | up      |
+-------------------+---------+---------+---------+---------+---------+
| memory run        | +1.83%  | +4.03%  | +2.64%  | +1.16%  | -12.52% |
+-------------------+---------+---------+---------+---------+---------+


stress-ng bogo-ops/s statistics by stress level (Sysbench):

(higher = better)

[s-50] Sysbench — stress-ng vs baseline
+----------+------------+---------+-------+
| Variant  | Mean %diff | StdDev± | CV    |
+----------+------------+---------+-------+
| baseline | 588.34     | ±1.72%  | 1.72% |
+----------+------------+---------+-------+
| concur   | -10.77%    | ±2.52%  | 2.52% |
| max      | -6.24%     | ±2.21%  | 2.21% |
| smp      | -6.16%     | ±1.93%  | 1.93% |
| tasks    | -8.51%     | ±3.21%  | 3.21% |
| up       | -10.88%    | ±3.25%  | 3.25% |
+----------+------------+---------+-------+


[s-90] Sysbench — stress-ng vs baseline
+----------+------------+---------+-------+
| Variant  | Mean %diff | StdDev± | CV    |
+----------+------------+---------+-------+
| baseline | 726.96     | ±2.00%  | 2.00% |
+----------+------------+---------+-------+
| concur   | +4.22%     | ±1.24%  | 1.24% |
| max      | +4.00%     | ±0.90%  | 0.90% |
| smp      | +3.10%     | ±1.73%  | 1.73% |
| tasks    | +4.28%     | ±1.83%  | 1.83% |
| up       | +15.61%    | ±5.10%  | 5.10% |
+----------+------------+---------+-------+

=======================================================================
  SCHBENCH
=======================================================================

All testruns were done with footprint set to 128kb.

There is some variability between the different test environments
(especially between the 32 threads-locking-yes case and the others),
but there is a general pattern.

Without stress-ng running in parallel, over the different test cases,
we did generally see a reduction in tail latency while other metrics
remained largely unchanged.

At 50% cpu utilization, tail latency is still improved (or close to the
noise floor), but average latency shows a more significant regression.

Then, at 90% cpu utilization, all latency components show regression,
tail latency the most of all.

RPS is not impacted as strongly.

Throughout this the stress-ng benchmark results show improvement at 90%
cpu utilization, while at 50% the changes are quite close to the noise
floor.


=======================================================================
  SCHBENCH 16T -- LOCKING: NO
=======================================================================

Schbench % diff from baseline by mode:

(Request latency percentiles: lower = better; RPS: higher = better)

[s-00] Schbench 16t -- Locking: No
+-------------------+---------+---------+---------+---------+---------+
| Metric            | concur  | max     | smp     | tasks   | up      |
+-------------------+---------+---------+---------+---------+---------+
| Req p50  (us)     | +0.00%  | +0.00%  | +0.00%  | +0.00%  | +0.00%  |
| Req p90  (us)     | -0.23%  | +0.00%  | +0.00%  | +0.00%  | -0.23%  |
| Req p99.9(us)     | -7.23%  | -5.16%  | -3.10%  | -5.85%  | -5.51%  |
| RPS p50  (req)    | +0.00%  | +0.00%  | +0.00%  | +0.00%  | +0.35%  |
+-------------------+---------+---------+---------+---------+---------+


[s-50] Schbench 16t -- Locking: No
+-------------------+---------+---------+---------+---------+---------+
| Metric            | concur  | max     | smp     | tasks   | up      |
+-------------------+---------+---------+---------+---------+---------+
| Req p50  (us)     | +13.51% | +15.57% | +15.27% | +13.80% | +14.68% |
| Req p90  (us)     | -1.48%  | +2.32%  | +2.53%  | -2.11%  | +1.48%  |
| Req p99.9(us)     | -8.99%  | -7.53%  | -5.35%  | -8.75%  | +4.86%  |
| RPS p50  (req)    | +3.65%  | +1.00%  | +1.00%  | +3.65%  | +0.66%  |
+-------------------+---------+---------+---------+---------+---------+


[s-90] Schbench 16t -- Locking: No
+-------------------+---------+---------+---------+---------+---------+
| Metric            | concur  | max     | smp     | tasks   | up      |
+-------------------+---------+---------+---------+---------+---------+
| Req p50  (us)     | +7.72%  | +7.72%  | +7.46%  | +7.72%  | +8.49%  |
| Req p90  (us)     | +4.76%  | +4.36%  | +4.76%  | +4.76%  | +19.35% |
| Req p99.9(us)     | +34.35% | +34.35% | +34.35% | +34.35% | +38.10% |
| RPS p50  (req)    | -1.09%  | +0.00%  | +0.00%  | -1.09%  | -7.56%  |
+-------------------+---------+---------+---------+---------+---------+


stress-ng bogo-ops/s statistics by stress level (schbench):

[s-50] Schbench 16t -- Locking: No — stress-ng vs baseline
+----------+------------+---------+-------+
| Variant  | Mean %diff | StdDev± | CV    |
+----------+------------+---------+-------+
| concur   | -2.17%     | ±1.50%  | 1.50% |
| max      | +1.08%     | ±1.61%  | 1.61% |
| smp      | +0.37%     | ±1.63%  | 1.63% |
| tasks    | -2.20%     | ±1.00%  | 1.00% |
| up       | +2.27%     | ±1.56%  | 1.56% |
+----------+------------+---------+-------+


[s-90] Schbench 16t -- Locking: No — stress-ng vs baseline
+----------+------------+---------+-------+
| Variant  | Mean %diff | StdDev± | CV    |
+----------+------------+---------+-------+
| concur   | +3.23%     | ±1.05%  | 1.05% |
| max      | -0.25%     | ±1.12%  | 1.12% |
| smp      | +0.93%     | ±2.70%  | 2.70% |
| tasks    | +1.33%     | ±1.62%  | 1.62% |
| up       | +8.09%     | ±5.19%  | 5.18% |
+----------+------------+---------+-------+



=======================================================================
  SCHBENCH 16T -- LOCKING: YES
=======================================================================

Schbench % diff from baseline by mode:

(Request latency percentiles: lower = better; RPS: higher = better)

[s-00] Schbench 16t -- Locking: Yes
+-------------------+---------+---------+---------+---------+---------+
| Metric            | concur  | max     | smp     | tasks   | up      |
+-------------------+---------+---------+---------+---------+---------+
| Req p50  (us)     | +0.00%  | +0.00%  | +0.00%  | +0.00%  | +0.00%  |
| Req p90  (us)     | +0.23%  | +0.23%  | +0.23%  | +0.23%  | +0.00%  |
| Req p99.9(us)     | -12.86% | -15.25% | -2.39%  | -11.66% | -8.67%  |
| RPS p50  (req)    | +0.00%  | +0.00%  | +0.00%  | +0.00%  | +0.35%  |
+-------------------+---------+---------+---------+---------+---------+


[s-50] Schbench 16t -- Locking: Yes
+-------------------+---------+---------+---------+---------+---------+
| Metric            | concur  | max     | smp     | tasks   | up      |
+-------------------+---------+---------+---------+---------+---------+
| Req p50  (us)     | +10.16% | +11.85% | +11.85% | +10.16% | +11.00% |
| Req p90  (us)     | -0.36%  | +4.63%  | +5.35%  | -0.36%  | +2.50%  |
| Req p99.9(us)     | +0.58%  | +0.87%  | +4.05%  | -1.45%  | +2.03%  |
| RPS p50  (req)    | +3.21%  | -0.36%  | +0.00%  | +2.85%  | +0.71%  |
+-------------------+---------+---------+---------+---------+---------+


[s-90] Schbench 16t -- Locking: Yes
+-------------------+---------+---------+---------+---------+---------+
| Metric            | concur  | max     | smp     | tasks   | up      |
+-------------------+---------+---------+---------+---------+---------+
| Req p50  (us)     | +7.70%  | +7.45%  | +7.45%  | +7.70%  | +9.24%  |
| Req p90  (us)     | +4.13%  | +1.50%  | +3.38%  | +3.75%  | +102.2… |
| Req p99.9(us)     | +18.21% | +13.94% | +16.22% | +13.66% | +306.8… |
| RPS p50  (req)    | -1.52%  | -0.38%  | -0.38%  | -1.52%  | -26.95% |
+-------------------+---------+---------+---------+---------+---------+

stress-ng bogo-ops/s statistics by stress level (schbench):

[s-50] Schbench 16t -- Locking: Yes — stress-ng vs baseline
+----------+------------+---------+-------+
| Variant  | Mean %diff | StdDev± | CV    |
+----------+------------+---------+-------+
| concur   | -1.29%     | ±1.16%  | 1.16% |
| max      | +1.26%     | ±1.18%  | 1.17% |
| smp      | +1.18%     | ±1.15%  | 1.15% |
| tasks    | +0.02%     | ±2.05%  | 2.04% |
| up       | +2.31%     | ±2.56%  | 2.56% |
+----------+------------+---------+-------+


[s-90] Schbench 16t -- Locking: Yes — stress-ng vs baseline
+----------+------------+---------+-------+
| Variant  | Mean %diff | StdDev± | CV    |
+----------+------------+---------+-------+
| concur   | +6.16%     | ±1.47%  | 1.47% |
| max      | +3.10%     | ±1.89%  | 1.89% |
| smp      | +3.91%     | ±1.83%  | 1.83% |
| tasks    | +5.68%     | ±1.87%  | 1.87% |
| up       | +16.60%    | ±8.60%  | 8.60% |
+----------+------------+---------+-------+


=======================================================================
  SCHBENCH 32T -- LOCKING: NO
=======================================================================

Schbench % diff from baseline by mode:

(Request latency percentiles: lower = better; RPS: higher = better)

[s-00] Schbench 32t -- Locking: No
+-------------------+---------+---------+---------+---------+---------+
| Metric            | concur  | max     | smp     | tasks   | up      |
+-------------------+---------+---------+---------+---------+---------+
| Req p50  (us)     | +0.00%  | +0.00%  | +0.00%  | +0.00%  | +0.00%  |
| Req p90  (us)     | +0.00%  | +0.00%  | -0.23%  | +0.00%  | -0.23%  |
| Req p99.9(us)     | -4.80%  | -3.77%  | -6.17%  | -7.55%  | -10.98% |
| RPS p50  (req)    | +0.35%  | +0.35%  | +0.35%  | +0.35%  | +0.35%  |
+-------------------+---------+---------+---------+---------+---------+


[s-50] Schbench 32t -- Locking: No
+-------------------+---------+---------+---------+---------+---------+
| Metric            | concur  | max     | smp     | tasks   | up      |
+-------------------+---------+---------+---------+---------+---------+
| Req p50  (us)     | +11.76% | +13.77% | +13.49% | +11.76% | +12.63% |
| Req p90  (us)     | -2.52%  | +2.31%  | +2.31%  | -2.73%  | -0.63%  |
| Req p99.9(us)     | -8.99%  | -6.80%  | -4.13%  | -9.96%  | +2.43%  |
| RPS p50  (req)    | +2.61%  | -0.65%  | -0.33%  | +2.61%  | +0.65%  |
+-------------------+---------+---------+---------+---------+---------+


[s-90] Schbench 32t -- Locking: No
+-------------------+---------+---------+---------+---------+---------+
| Metric            | concur  | max     | smp     | tasks   | up      |
+-------------------+---------+---------+---------+---------+---------+
| Req p50  (us)     | +5.55%  | +5.55%  | +5.55%  | +5.55%  | +7.06%  |
| Req p90  (us)     | +2.62%  | +2.62%  | +2.62%  | +2.62%  | +32.73% |
| Req p99.9(us)     | +31.57% | +31.11% | +31.57% | +31.57% | +34.33% |
| RPS p50  (req)    | +1.46%  | +2.55%  | +2.55%  | +1.46%  | -10.66% |
+-------------------+---------+---------+---------+---------+---------+


stress-ng bogo-ops/s statistics by stress level (schbench):

[s-50] Schbench 32t -- Locking: No — stress-ng vs baseline
+----------+------------+---------+-------+
| Variant  | Mean %diff | StdDev± | CV    |
+----------+------------+---------+-------+
| baseline | 478.27     | ±1.98%  | 1.99% |
+----------+------------+---------+-------+
| concur   | -2.85%     | ±1.45%  | 1.45% |
| max      | +0.12%     | ±1.95%  | 1.95% |
| smp      | -0.23%     | ±1.22%  | 1.22% |
| tasks    | -2.39%     | ±1.96%  | 1.96% |
| up       | -0.27%     | ±1.59%  | 1.59% |
+----------+------------+---------+-------+


[s-90] Schbench 32t -- Locking: No — stress-ng vs baseline
+----------+------------+---------+-------+
| Variant  | Mean %diff | StdDev± | CV    |
+----------+------------+---------+-------+
| baseline | 577.45     | ±0.69%  | 0.70% |
+----------+------------+---------+-------+
| concur   | +2.17%     | ±1.54%  | 1.54% |
| max      | +0.25%     | ±1.28%  | 1.28% |
| smp      | -0.33%     | ±1.42%  | 1.42% |
| tasks    | +1.07%     | ±1.17%  | 1.17% |
| up       | +10.70%    | ±5.47%  | 5.47% |
+----------+------------+---------+-------+


=======================================================================
  SCHBENCH 32T -- LOCKING: YES
=======================================================================

Schbench % diff from baseline by mode:

(Request latency percentiles: lower = better; RPS: higher = better)

[s-00] Schbench 32t -- Locking: Yes
+-------------------+---------+---------+---------+---------+---------+
| Metric            | concur  | max     | smp     | tasks   | up      |
+-------------------+---------+---------+---------+---------+---------+
| Req p50  (us)     | +0.00%  | +0.00%  | +0.00%  | +0.00%  | +0.00%  |
| Req p90  (us)     | +0.00%  | +0.00%  | +0.00%  | +0.00%  | -0.23%  |
| Req p99.9(us)     | +7.31%  | +10.05% | +13.70% | +10.35% | -16.74% |
| RPS p50  (req)    | +0.00%  | +0.00%  | +0.00%  | +0.35%  | +0.35%  |
+-------------------+---------+---------+---------+---------+---------+


[s-50] Schbench 32t -- Locking: Yes
+-------------------+---------+---------+---------+---------+---------+
| Metric            | concur  | max     | smp     | tasks   | up      |
+-------------------+---------+---------+---------+---------+---------+
| Req p50  (us)     | +8.30%  | +9.96%  | +9.96%  | +8.30%  | +9.41%  |
| Req p90  (us)     | -0.71%  | +3.54%  | +4.60%  | -0.71%  | +0.00%  |
| Req p99.9(us)     | -4.05%  | +3.76%  | +3.47%  | -2.89%  | -4.92%  |
| RPS p50  (req)    | +2.11%  | -1.05%  | -1.41%  | +2.11%  | +1.05%  |
+-------------------+---------+---------+---------+---------+---------+


[s-90] Schbench 32t -- Locking: Yes
+-------------------+---------+---------+---------+---------+---------+
| Metric            | concur  | max     | smp     | tasks   | up      |
+-------------------+---------+---------+---------+---------+---------+
| Req p50  (us)     | +5.55%  | +5.55%  | +5.55%  | +5.55%  | +6.81%  |
| Req p90  (us)     | -0.77%  | -1.16%  | -0.77%  | -0.77%  | +55.49% |
| Req p99.9(us)     | +11.00% | +11.28% | +14.67% | +19.18% | +92.67% |
| RPS p50  (req)    | +1.51%  | +2.64%  | +2.26%  | +1.51%  | -15.16% |
+-------------------+---------+---------+---------+---------+---------+


stress-ng bogo-ops/s statistics by stress level (schbench):

[s-50] Schbench 32t -- Locking: Yes — stress-ng vs baseline
+----------+------------+---------+-------+
| Variant  | Mean %diff | StdDev± | CV    |
+----------+------------+---------+-------+
| baseline | 506.91     | ±1.74%  | 1.74% |
+----------+------------+---------+-------+
| concur   | -6.03%     | ±1.28%  | 1.28% |
| max      | -3.98%     | ±1.68%  | 1.68% |
| smp      | -4.23%     | ±1.31%  | 1.31% |
| tasks    | -5.75%     | ±1.47%  | 1.47% |
| up       | -3.84%     | ±2.10%  | 2.10% |
+----------+------------+---------+-------+


[s-90] Schbench 32t -- Locking: Yes — stress-ng vs baseline
+----------+------------+---------+-------+
| Variant  | Mean %diff | StdDev± | CV    |
+----------+------------+---------+-------+
| baseline | 555.39     | ±1.03%  | 1.03% |
+----------+------------+---------+-------+
| concur   | +6.59%     | ±1.38%  | 1.38% |
| max      | +5.05%     | ±1.25%  | 1.25% |
| smp      | +7.28%     | ±1.90%  | 1.90% |
| tasks    | +7.07%     | ±1.46%  | 1.46% |
| up       | +18.88%    | ±7.68%  | 7.68% |
+----------+------------+---------+-------+

Regards,

-- 
Szabina Korbai

Linux on Z development
Software Labs Campus Unlimited Company
25 North Wall Quay,
Dublin 1,
D01 H104,
Ireland

szkorbai@xxxxxxxxxxxxx


IBM