Re: [RFC][PATCH 2/6] x86/topo: Add TOPO_NUMA_DOMAIN

[K Prateek Nayak]

Hello Peter,

On 3/2/2026 8:40 PM, Peter Zijlstra wrote:
> On Mon, Mar 02, 2026 at 09:46:57AM +0530, K Prateek Nayak wrote:
>> Hello Peter,
>>
>> On 2/27/2026 7:36 PM, Peter Zijlstra wrote:
>>>> Looking at the series, all we need is an equivalent of:
>>>>
>>>>   domain_weight(TOPO_NUMA_DOMAIN)
>>>
>>> Fair enough; but then lets replace patch 1 and 2 with something like
>>> that.
>>>
>>> But I must note that the nodemask API is crap; it has both node_set() and
>>> __node_set() be the atomic version :-(
>>>
>>> Let me go rework the other patches to fit on this.
>>
>> Boots fine with a s/domain_weight(TOPO_NUMA_DOMAIN)/num_phys_nodes()/
>> applied to Patch 3.
>>
>> Topology looks fine for NPS4 on my 3rd Generation EPYC with 2 sockets,
>> and I haven't triggered any warning even with "L3 as NUMA" turned on.
>> Feel free to include:
>>
>> Tested-by: K Prateek Nayak <kprateek.nayak@xxxxxxx>
> 
> Thanks!
> 
> I had a quick look at this NPS stuff, and that is more or less the same
> as the intel SNC thing. With two notable exceptions:
> 
>  - you've stuck to power-of-two numbers (good!)

Yeah but "L3 as NUMA" on a 6CCX machines doesn't follow that :-(
Is there any implicit dependency there?

P.S. All these configs are symmetric so those divisions should give the
correct results.

> 
>  - NPS0; I don't think Intel has anything like that (although I could be
>    mistaken).
> 
> Now, the __num_nodes_per_package is obviously not going to work for
> NPS0 (it bottoms out at 1).
> 
> Should we look at adding something for NPS0, or has that not been needed
> (yet) ?

Let me go boot into NPS0 to see what my machine thinks. But it shouldn't
do any harm right because of the DIV_ROUND_UP() right?

__num_nodes_per_package will be 1 (which is technically correct since
the whole package is indeed one node) and then we retain the PKG domain
so as far as those bits are concerned, it should be fine.

This is from my dual socket Zen3 booted into NPS0:

    CPU topo: Max. logical packages:   2
    CPU topo: Max. logical nodes:      1
    CPU topo: Num. nodes per package:  1
    CPU topo: Max. logical dies:       2
    CPU topo: Max. dies per package:   1
    CPU topo: Max. threads per core:   2
    CPU topo: Num. cores per package:    64
    CPU topo: Num. threads per package: 128
    CPU topo: Allowing 256 present CPUs plus 0 hotplug CPUs


CPU0's scheduler topology looks like:

    CPU0 attaching sched-domain(s):
     domain-0: span=0,128 level=SMT
      groups: 0:{ span=0 },
            128:{ span=128 }
      domain-1: span=0-7,128-135 level=MC
       groups: 0:{ span=0,128 cap=2048 },
               1:{ span=1,129 cap=2048 },
               2:{ span=2,130 cap=2048 },
               3:{ span=3,131 cap=2048 },
               4:{ span=4,132 cap=2048 },
               5:{ span=5,133 cap=2048 },
               6:{ span=6,134 cap=2048 },
               7:{ span=7,135 cap=2048 }
       domain-2: span=0-255 level=PKG
        groups:  0:{ span=0-7,128-135 cap=16384 },
                 8:{ span=8-15,136-143 cap=16384 },
                16:{ span=16-23,144-151 cap=16384 },
                24:{ span=24-31,152-159 cap=16384 },
                32:{ span=32-39,160-167 cap=16384 },
                40:{ span=40-47,168-175 cap=16384 },
                48:{ span=48-55,176-183 cap=16384 },
                56:{ span=56-63,184-191 cap=16384 },
                64:{ span=64-71,192-199 cap=16384 },
                72:{ span=72-79,200-207 cap=16384 },
                80:{ span=80-87,208-215 cap=16384 },
                88:{ span=88-95,216-223 cap=16384 },
                96:{ span=96-103,224-231 cap=16384 },
               104:{ span=104-111,232-239 cap=16384 },
               112:{ span=112-119,240-247 cap=16384 },
               120:{ span=120-127,248-255 cap=16384 }
    ...
    root domain span: 0-255


The PKG domain covers both the sockets since it uses the node mask which
covers the entire system.

-- 
Thanks and Regards,
Prateek