Re: [PATCH 2.5.58] new NUMA scheduler: fix

From: Michael Hohnbaum (hohnbaum@us.ibm.com)
Date: Tue Jan 14 2003 - 19:05:32 EST


On Tuesday 14 January 2003 16:55, Erich Focht wrote:
> Here's the new version of the NUMA scheduler built on top of the
> miniature scheduler of Martin. I incorporated Michael's ideas and
> Christoph's suggestions and rediffed for 2.5.58.
>
> The whole patch is really tiny: 9.5k. This time I attached the numa
> scheduler in form of two patches:

Ran tests on three different kernels:

stock58 - linux 2.5.58 with the cputime_stats patch
sched1-4-58 - stock58 with the first 4 NUMA scheduler patches
sched1-5-58 - stock58 with all 5 NUMA scheduler patches

Kernbench:
                        Elapsed User System CPU
             stock58 31.656s 305.85s 89.232s 1248.2%
         sched1-4-58 29.886s 287.506s 82.94s 1239%
         sched1-5-58 29.994s 288.796s 84.706s 1245%

Schedbench 4:
                        AvgUser Elapsed TotalUser TotalSys
             stock58 27.73 42.80 110.96 0.85
         sched1-4-58 32.86 46.41 131.47 0.85
         sched1-5-58 32.37 45.34 129.52 0.89

Schedbench 8:
                        AvgUser Elapsed TotalUser TotalSys
             stock58 45.97 61.87 367.81 2.11
         sched1-4-58 31.39 49.18 251.22 2.15
         sched1-5-58 37.52 61.32 300.22 2.06

Schedbench 16:
                        AvgUser Elapsed TotalUser TotalSys
             stock58 60.91 83.63 974.71 6.18
         sched1-4-58 54.31 62.11 869.11 3.84
         sched1-5-58 51.60 59.05 825.72 4.74

Schedbench 32:
                        AvgUser Elapsed TotalUser TotalSys
             stock58 84.26 195.16 2696.65 16.53
         sched1-4-58 61.49 140.51 1968.06 9.57
         sched1-5-58 55.23 117.32 1767.71 7.78

Schedbench 64:
                        AvgUser Elapsed TotalUser TotalSys
             stock58 123.27 511.77 7889.77 27.78
         sched1-4-58 63.39 266.40 4057.92 20.55
         sched1-5-58 59.57 250.25 3813.39 17.05

One anomaly noted was that the kernbench system time went up
about 5% with the 2.5.58 kernel from what it was on the last
version I tested with (2.5.55). This increase is in both stock
and with the NUMA scheduler, so is not caused by the NUMA scheduler.

Now that I've got baselines for these, I plan to next look at
tweaking various parameters within the scheduler and see how what
happens. Also, I owe Erich numbers running hackbench. Overall, I am
pleased with these results.

And just for grins, here are the detailed results for running
numa_test 32

sched1-4-58:

Executing 32 times: ./randupdt 1000000
Running 'hackbench 20' in the background: Time: 9.044
Job node00 node01 node02 node03 | iSched MSched | UserTime(s)
  1 0.0 100.0 0.0 0.0 | 1 1 | 55.04
  2 0.0 0.0 4.8 95.2 | 2 3 *| 70.38
  3 0.0 0.0 3.2 96.8 | 2 3 *| 72.72
  4 0.0 26.4 2.8 70.7 | 2 3 *| 72.99
  5 100.0 0.0 0.0 0.0 | 0 0 | 57.03
  6 100.0 0.0 0.0 0.0 | 0 0 | 55.06
  7 100.0 0.0 0.0 0.0 | 0 0 | 57.18
  8 0.0 100.0 0.0 0.0 | 1 1 | 55.38
  9 100.0 0.0 0.0 0.0 | 0 0 | 54.37
 10 0.0 100.0 0.0 0.0 | 1 1 | 56.06
 11 0.0 13.2 0.0 86.8 | 3 3 | 64.33
 12 0.0 0.0 0.0 100.0 | 3 3 | 62.35
 13 1.7 0.0 98.3 0.0 | 2 2 | 67.47
 14 100.0 0.0 0.0 0.0 | 0 0 | 55.94
 15 0.0 29.4 61.9 8.6 | 3 2 *| 78.76
 16 0.0 100.0 0.0 0.0 | 1 1 | 56.42
 17 18.9 0.0 74.9 6.2 | 3 2 *| 70.57
 18 0.0 0.0 100.0 0.0 | 2 2 | 63.01
 19 0.0 100.0 0.0 0.0 | 1 1 | 55.97
 20 0.0 0.0 92.7 7.3 | 3 2 *| 65.62
 21 0.0 0.0 100.0 0.0 | 2 2 | 62.70
 22 0.0 100.0 0.0 0.0 | 1 1 | 55.53
 23 0.0 1.5 0.0 98.5 | 3 3 | 56.95
 24 0.0 100.0 0.0 0.0 | 1 1 | 55.75
 25 0.0 30.0 2.3 67.7 | 2 3 *| 77.78
 26 0.0 0.0 0.0 100.0 | 3 3 | 57.71
 27 13.6 0.0 86.4 0.0 | 0 2 *| 66.55
 28 100.0 0.0 0.0 0.0 | 0 0 | 55.43
 29 0.0 100.0 0.0 0.0 | 1 1 | 56.12
 30 19.8 0.0 62.5 17.6 | 3 2 *| 66.92
 31 100.0 0.0 0.0 0.0 | 0 0 | 54.90
 32 100.0 0.0 0.0 0.0 | 0 0 | 54.70
AverageUserTime 61.49 seconds
ElapsedTime 140.51
TotalUserTime 1968.06
TotalSysTime 9.57

sched1-5-58:

Executing 32 times: ./randupdt 1000000
Running 'hackbench 20' in the background: Time: 9.145
Job node00 node01 node02 node03 | iSched MSched | UserTime(s)
  1 100.0 0.0 0.0 0.0 | 0 0 | 54.88
  2 0.0 100.0 0.0 0.0 | 1 1 | 54.08
  3 0.0 0.0 0.0 100.0 | 3 3 | 55.48
  4 0.0 0.0 0.0 100.0 | 3 3 | 55.47
  5 100.0 0.0 0.0 0.0 | 0 0 | 53.84
  6 100.0 0.0 0.0 0.0 | 0 0 | 53.37
  7 0.0 0.0 0.0 100.0 | 3 3 | 55.41
  8 90.9 9.1 0.0 0.0 | 1 0 *| 55.58
  9 0.0 100.0 0.0 0.0 | 1 1 | 55.61
 10 0.0 100.0 0.0 0.0 | 1 1 | 54.56
 11 0.0 0.0 98.1 1.9 | 2 2 | 56.25
 12 0.0 0.0 0.0 100.0 | 3 3 | 55.07
 13 0.0 0.0 0.0 100.0 | 3 3 | 54.92
 14 0.0 100.0 0.0 0.0 | 1 1 | 54.59
 15 100.0 0.0 0.0 0.0 | 0 0 | 55.10
 16 5.0 0.0 95.0 0.0 | 2 2 | 56.97
 17 0.0 0.0 100.0 0.0 | 2 2 | 55.51
 18 100.0 0.0 0.0 0.0 | 0 0 | 53.97
 19 0.0 4.7 95.3 0.0 | 2 2 | 57.21
 20 0.0 0.0 100.0 0.0 | 2 2 | 55.53
 21 0.0 0.0 100.0 0.0 | 2 2 | 56.46
 22 0.0 0.0 100.0 0.0 | 2 2 | 55.48
 23 0.0 0.0 0.0 100.0 | 3 3 | 55.99
 24 0.0 100.0 0.0 0.0 | 1 1 | 55.32
 25 0.0 6.2 93.8 0.0 | 2 2 | 57.66
 26 0.0 0.0 0.0 100.0 | 3 3 | 55.60
 27 0.0 100.0 0.0 0.0 | 1 1 | 54.65
 28 0.0 0.0 0.0 100.0 | 3 3 | 56.39
 29 0.0 100.0 0.0 0.0 | 1 1 | 54.91
 30 100.0 0.0 0.0 0.0 | 0 0 | 53.58
 31 100.0 0.0 0.0 0.0 | 0 0 | 53.53
 32 31.5 68.4 0.0 0.0 | 0 1 *| 54.41
AverageUserTime 55.23 seconds
ElapsedTime 117.32
TotalUserTime 1767.71
TotalSysTime 7.78

-- 

Michael Hohnbaum 503-578-5486 hohnbaum@us.ibm.com T/L 775-5486

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/



This archive was generated by hypermail 2b29 : Wed Jan 15 2003 - 22:00:52 EST