Re: [PATCH] sched,numa: document and fix numa_preferred_nid setting
From: Srikar Dronamraju
Date: Fri Jun 19 2015 - 13:16:50 EST
>
> OK, so we are looking at two multi-threaded processes
> on a 4 node system, and waiting for them to converge?
>
> It may make sense to add my patch in with your patch
> 1/4 from last week, as well as the correct part of
> your patch 4/4, and see how they all work together.
>
Tested specjbb and autonumabenchmark on 4 kernels.
Plain 4.1.0-rc7-tip (i)
tip + only Rik's patch (ii)
tip + Rik's ++ (iii)
tip + Srikar's ++ (iv)
(i) = Plain 4.1.0-rc7-tip = tip = 4.1.0-rc7 (b7ca96b)
(ii) = tip + only Rik's patch = (i) + Rik's fix numa_preferred_nid setting
(iii) = tip + Rik's ++ (iii) = (ii) + Srikar's numa hotness + correct nid for evaluating task weight
(iv) = tip + Srikar's ++ (iv) = (i) + Srikar's numa hotness + correct nid for evaluating task weight +
numa_has_capacity fix + always update preferred node
Plain 4.1.0-rc7-tip (i)
Testcase: Min Max Avg StdDev
elapsed_numa01: 858.85 949.18 915.64 33.06
elapsed_numa02: 23.09 29.89 26.43 2.18
Testcase: Min Max Avg StdDev
system_numa01: 1516.72 1855.08 1686.24 113.95
system_numa02: 63.69 79.06 70.35 5.87
Testcase: Min Max Avg StdDev
user_numa01: 73284.76 80818.21 78060.88 2773.60
user_numa02: 1690.18 2071.07 1821.64 140.25
Testcase: Min Max Avg StdDev
total_numa01: 74801.50 82572.60 79747.12 2875.61
total_numa02: 1753.87 2142.77 1891.99 143.59
tip + only Rik's patch (ii)
Testcase: Min Max Avg StdDev %Change
elapsed_numa01: 665.26 877.47 776.77 79.23 15.83%
elapsed_numa02: 24.59 31.30 28.17 2.48 -5.56%
Testcase: Min Max Avg StdDev %Change
system_numa01: 659.57 1220.99 942.36 234.92 60.92%
system_numa02: 44.62 86.01 64.64 14.24 6.64%
Testcase: Min Max Avg StdDev %Change
user_numa01: 56280.95 75908.81 64993.57 7764.30 17.21%
user_numa02: 1790.35 2155.02 1916.12 132.57 -4.38%
Testcase: Min Max Avg StdDev %Change
total_numa01: 56940.50 77128.20 65935.92 7993.49 17.91%
total_numa02: 1834.97 2227.03 1980.76 136.51 -3.99%
tip + Rik's ++ (iii)
Testcase: Min Max Avg StdDev %Change
elapsed_numa01: 630.60 860.06 760.07 74.33 18.09%
elapsed_numa02: 21.92 34.42 27.72 4.49 -3.75%
Testcase: Min Max Avg StdDev %Change
system_numa01: 474.31 1379.49 870.12 296.35 59.16%
system_numa02: 63.74 120.25 86.69 20.69 -13.59%
Testcase: Min Max Avg StdDev %Change
user_numa01: 53004.03 68125.84 61697.01 5011.38 24.02%
user_numa02: 1650.82 2278.71 1941.26 224.59 -5.25%
Testcase: Min Max Avg StdDev %Change
total_numa01: 53478.30 69505.30 62567.12 5288.18 24.72%
total_numa02: 1714.56 2398.96 2027.95 238.08 -5.67%
tip + Srikar's ++ (iv)
Testcase: Min Max Avg StdDev %Change
elapsed_numa01: 690.74 919.49 782.67 78.51 14.46%
elapsed_numa02: 21.78 29.57 26.02 2.65 1.39%
Testcase: Min Max Avg StdDev %Change
system_numa01: 659.12 1041.19 870.15 143.13 78.38%
system_numa02: 52.20 78.73 64.18 11.28 7.84%
Testcase: Min Max Avg StdDev %Change
user_numa01: 56410.39 71492.31 62514.78 5444.90 21.75%
user_numa02: 1594.27 1934.40 1754.37 126.41 3.48%
Testcase: Min Max Avg StdDev %Change
total_numa01: 57069.50 72509.90 63384.94 5567.71 22.57%
total_numa02: 1647.85 2010.87 1818.55 136.88 3.65%
5 interations of Specjbb on 4 node, 24 core powerpc machine.
Ran 1 instance per system.
For specjbb (higher bops per JVM is better)
Plain 4.1.0-rc7-tip (i)
Metric: Min Max Avg StdDev
bopsperJVM: 265519.00 272466.00 269377.80 2391.04
tip + only Rik's patch (ii)
Metric: Min Max Avg StdDev %Change
bopsperJVM: 263393.00 269660.00 266920.20 2792.07 -0.91%
tip + Rik's ++ (iii)
Metric: Min Max Avg StdDev %Change
bopsperJVM: 264298.00 271236.00 266818.20 2579.62 -0.94%
tip + Srikar's ++ (iv)
Metric: Min Max Avg StdDev %Change
bopsperJVM: 266774.00 272434.00 269839.60 2083.19 +0.17%
So fix for numa_has_capacity and always setting preferred node based on
fault stats seems to help autonuma and specjbb.
--
Thanks and Regards
Srikar Dronamraju
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
Please read the FAQ at http://www.tux.org/lkml/