Re: [tip: sched/core] sched/fair: Only update stats for allowed CPUs when looking for dst group

[Erwan Velu]

Further investigations show the same impact on older generations like
the AMD Siena 8534 (64c).
So it's not only multi-socket systems being affected, but also any
machine with enough NUMA domains.
My Siena 8534 is configured, as my 9845 Turin, with NPS=2 and LLC=ON, gives :
  NUMA node0 CPU(s):         0-7,64-71
  NUMA node1 CPU(s):         8-15,72-79
  NUMA node2 CPU(s):         16-23,80-87
  NUMA node3 CPU(s):         24-31,88-95
  NUMA node4 CPU(s):         32-39,96-103
  NUMA node5 CPU(s):         40-47,104-111
  NUMA node6 CPU(s):         48-55,112-119
  NUMA node7 CPU(s):         56-63,120-127

I'm attaching graphs showing the performance impact from the baseline
(6.18.38) and the same machine with commit
82d6e01a0699800efd8b048eb584c907ccb47b7a on top of 6.18.38.

Le mar. 7 juil. 2026 à 10:03, Erwan Velu <erwanaliasr1@xxxxxxxxx> a écrit :
>
> Hi Adam, Peter,
>
> I've been chasing a performance bug on my systems and found that this
> commit was a
> real game-changer on my production server.
>
> I'm using dual AMD Turin 9845 (160 cores each), with the following
> NUMA topology:
> Socket 0:
>   NUMA node0 CPU(s):         0-15,320-335
>   NUMA node1 CPU(s):         16-31,336-351
>   NUMA node2 CPU(s):         32-47,352-367
>   NUMA node3 CPU(s):         48-63,368-383
>   NUMA node4 CPU(s):         64-79,384-399
>   NUMA node5 CPU(s):         80-95,400-415
>   NUMA node6 CPU(s):         96-111,416-431
>   NUMA node7 CPU(s):         112-127,432-447
>   NUMA node8 CPU(s):         128-143,448-463
>   NUMA node9 CPU(s):         144-159,464-479
>
> Socket 1:
>   NUMA node10 CPU(s):        160-175,480-495
>   NUMA node11 CPU(s):        176-191,496-511
>   NUMA node12 CPU(s):        192-207,512-527
>   NUMA node13 CPU(s):        208-223,528-543
>   NUMA node14 CPU(s):        224-239,544-559
>   NUMA node15 CPU(s):        240-255,560-575
>   NUMA node16 CPU(s):        256-271,576-591
>   NUMA node17 CPU(s):        272-287,592-607
>   NUMA node18 CPU(s):        288-303,608-623
>   NUMA node19 CPU(s):        304-319,624-639
>
> I was able to setup a reproducer by using the wonderful stress-ng tool
> like the following :
>     [root@host]$ stress-ng --taskset 0-175 --timeout 30 --metrics
> --cpu-method float128 --cpu 176
>
> On my 6.18.38 and 6.12.95, the latest LTS at the date, I have very
> weird performance like :
>     stress-ng: info:  [11329] dispatching hogs: 176 cpu
>     stress-ng: metrc: [11329] stressor       bogo ops real time  usr
> time  sys time   bogo ops/s     bogo ops/s CPU used per       RSS Max
>     stress-ng: metrc: [11329]                           (secs)
> (secs)    (secs)   (real time) (usr+sys time) instance (%)
> (KB)
>     stress-ng: metrc: [11329] cpu             5515290     29.97
> 1311.82      0.02    184034.19        4204.23        24.87
> 0
>     stress-ng: info:  [11329] skipped: 0
>
> /!\ Stress-ng reports 25% CPU usage, resulting in only 5M bogo-ops.
>
> When applying this patch on top of the two LTS releases, I get the
> following result:
>     stress-ng: info:  [11329] dispatching hogs: 176 cpu
>     stress-ng: metrc: [11329] stressor       bogo ops real time  usr
> time  sys time   bogo ops/s     bogo ops/s CPU used per       RSS Max
>     stress-ng: metrc: [11329]                           (secs)
> (secs)    (secs)   (real time) (usr+sys time) instance (%)
> (KB)
>     stress-ng: metrc: [11329] cpu            19306125     29.99
> 5236.45      0.06    643718.82        3686.83        99.20
> 0
>     stress-ng: info:  [11329] skipped: 0
>
> The CPU usage gets an expected 99% and 19M+ bogo-ops.
> That's a massive 4x performance impact on a CPU-bound application.
>
> This outcome is 100% reproducible and particularly visible with this
> stress-ng reproducer, but
> also exists in other conditions when choosing other NUMA nodes and CPUs.
>
> Considering the impact of this change on performance, can we request a
> backport to all stable kernels?
> Erwan,

Attachment: Siena_perf_scaling.svg
Description: image/svg

Attachment: Turin_perf_scaling.svg
Description: image/svg