Re: [RFC PATCH v2 00/17] Core scheduling v2

From: Ingo Molnar
Date: Tue Apr 30 2019 - 00:43:19 EST



* Aubrey Li <aubrey.intel@xxxxxxxxx> wrote:

> On Tue, Apr 30, 2019 at 12:01 AM Ingo Molnar <mingo@xxxxxxxxxx> wrote:
> > * Li, Aubrey <aubrey.li@xxxxxxxxxxxxxxx> wrote:
> >
> > > > I.e. showing the approximate CPU thread-load figure column would be
> > > > very useful too, where '50%' shows half-loaded, '100%' fully-loaded,
> > > > '200%' over-saturated, etc. - for each row?
> > >
> > > See below, hope this helps.
> > > .--------------------------------------------------------------------------------------------------------------------------------------.
> > > |NA/AVX vanilla-SMT [std% / sem%] cpu% |coresched-SMT [std% / sem%] +/- cpu% | no-SMT [std% / sem%] +/- cpu% |
> > > |--------------------------------------------------------------------------------------------------------------------------------------|
> > > | 1/1 508.5 [ 0.2%/ 0.0%] 2.1% | 504.7 [ 1.1%/ 0.1%] -0.8% 2.1% | 509.0 [ 0.2%/ 0.0%] 0.1% 4.3% |
> > > | 2/2 1000.2 [ 1.4%/ 0.1%] 4.1% | 1004.1 [ 1.6%/ 0.2%] 0.4% 4.1% | 997.6 [ 1.2%/ 0.1%] -0.3% 8.1% |
> > > | 4/4 1912.1 [ 1.0%/ 0.1%] 7.9% | 1904.2 [ 1.1%/ 0.1%] -0.4% 7.9% | 1914.9 [ 1.3%/ 0.1%] 0.1% 15.1% |
> > > | 8/8 3753.5 [ 0.3%/ 0.0%] 14.9% | 3748.2 [ 0.3%/ 0.0%] -0.1% 14.9% | 3751.3 [ 0.4%/ 0.0%] -0.1% 30.5% |
> > > | 16/16 7139.3 [ 2.4%/ 0.2%] 30.3% | 7137.9 [ 1.8%/ 0.2%] -0.0% 30.3% | 7049.2 [ 2.4%/ 0.2%] -1.3% 60.4% |
> > > | 32/32 10899.0 [ 4.2%/ 0.4%] 60.3% | 10780.3 [ 4.4%/ 0.4%] -1.1% 55.9% | 10339.2 [ 9.6%/ 0.9%] -5.1% 97.7% |
> > > | 64/64 15086.1 [11.5%/ 1.2%] 97.7% | 14262.0 [ 8.2%/ 0.8%] -5.5% 82.0% | 11168.7 [22.2%/ 1.7%] -26.0% 100.0% |
> > > |128/128 15371.9 [22.0%/ 2.2%] 100.0% | 14675.8 [14.4%/ 1.4%] -4.5% 82.8% | 10963.9 [18.5%/ 1.4%] -28.7% 100.0% |
> > > |256/256 15990.8 [22.0%/ 2.2%] 100.0% | 12227.9 [10.3%/ 1.0%] -23.5% 73.2% | 10469.9 [19.6%/ 1.7%] -34.5% 100.0% |
> > > '--------------------------------------------------------------------------------------------------------------------------------------'
> >
> > Very nice, thank you!
> >
> > What's interesting is how in the over-saturated case (the last three
> > rows: 128, 256 and 512 total threads) coresched-SMT leaves 20-30% CPU
> > performance on the floor according to the load figures.
>
> Yeah, I found the next focus.
>
> > Is this true idle time (which shows up as 'id' during 'top'), or some
> > load average artifact?
>
> vmstat periodically reported intermediate CPU utilization in one
> second, it was running simultaneously when the benchmarks run. The cpu%
> is computed by the average of (100-idle) series.

Ok - so 'vmstat' uses /proc/stat, which uses cpustat[CPUTIME_IDLE] (or
its NOHZ work-alike), so this should be true idle time - to the extent
the HZ process clock's sampling is accurate.

So I guess the answer to my question is "yes". ;-)

BTW., for robustness sake you might want to add iowait to idle time (it's
the 'wa' field of vmstat) - it shouldn't matter for this particular
benchmark which doesn't do much IO, but it might for others.

Both CPUTIME_IDLE and CPUTIME_IOWAIT are idle states when a CPU is not
utilized.

[ Side note: we should really implement precise idle time accounting when
CONFIG_IRQ_TIME_ACCOUNTING=y is enabled. We pay all the costs of the
timestamps, but AFAICS we don't propagate that into the idle cputime
metrics. ]

Thanks,

Ingo