Re: [PATCH v2 4/4] sched/rt: Split cpupri_vec->cpumask to per NUMA node to reduce contention

[K Prateek Nayak]

Hello Chenyu, Tim,

On 4/8/2026 9:22 PM, K Prateek Nayak wrote:
> Hello Chenyu,
> 
> On 4/8/2026 5:05 PM, Chen, Yu C wrote:
>> We haven't tried breaking it down further. One possible approach
>> is to partition it at L2 scope, the benefit of which may depend on
>> the workload.
> 
> I fear at that point we'll have too many cachelines and too much
> cache pollution when the CPU starts reading this at tick to schedule
> a newidle balance.
> 
> A 128 core system would bring in 128 * 64B = 8kB worth of data to
> traverse the mask and at that point it becomes a trade off between
> how fast you want reads vs writes and does it even speed up writes
> after a certain point?
> 
> Sorry I got distracted by some other stuff today but I'll share the
> results from my experiments tomorrow.

Here is some data from an experiments I ran on a 3rd Generation EPYC
system (2 socket x 64C/128T (8LLCs per socket)):

Experiment: Two threads pinned per-CPU on all CPUs yielding to each other
and are operating on some cpumask - one setting the current CPU on the
mask and other clearing the current CPU: Just an estimate of worst case
scenario is we have to do one modification per sched-switch.

I'm measuring total cycles taken for cpumask operations with following
variants:

                                            %cycles vs global mask operation

global mask                                     : 100.0000%  (var: 3.28%)
per-NUMA mask                                   :  32.9209%  (var: 7.77%)
per-LLC mask                                    :   1.2977%  (var: 4.85%)
per-LLC mask (u8 operation; no LOCK prefix)     :   0.4930%  (var: 0.83%)

per-NUMA split is 3X faster, per-LLC on this 16LLC machine is 77x faster
and since there is enough space in the cacheline we can use a u8 to set
and clear the CPu atomically without LOCK prefix and then do a >> 3 to
get the CPU index from set bit which is 202x faster.

If we use the u8 operations, we can only read 8CPUs per 8-byte load on
64-bit system but with per-LLC mask, we can scan all 16CPUs on the LLC
with one 8-byte read and and per-NUMA one requires two 8-byte reads to
scan the 128CPUs per socket.

I think per-LLC mask (or, as Tim suggested, 64CPUs per cacheline) is
a good tradeoff between the speedup vs amount of loads required to
piece together the full cpumask. Thoughts?

-- 
Thanks and Regards,
Prateek