RE: [PATCH] sched/topology: fix the issue groups don't span domain->span for NUMA diameter > 2

[Valentin Schneider]

On 03/02/21 10:23, Song Bao Hua (Barry Song) wrote:
>> -----Original Message-----
>> From: Valentin Schneider [mailto:valentin.schneider@xxxxxxx]
>> Thoughts?
>
> I guess the original purpose of overlapping groups is creating as few groups
> as possible. If we totally remove overlapping groups, it seems we will create
> much more groups?
> For example, while node0 begins to build sched_domain for distance 20, it will
> add node2, since the distance between node2 and node3 is 15, so while node2 is
> added, node3 is also added as node2's lower domain has covered node3. So we need
> two groups only for node0's sched_domain of distance level 20.
> +-------+                  +--------+
>  |       |      15          |        |
>  |  node0+----------------+ | node1  |
>  |       |                  |        |
>  +----+--+                XXX--------+
>       |                 XXX
>       |                XX
> 20    |         15   XX
>       |            XXX
>       |       X XXX
>  +----+----XXX               +-------+
>  |         |     15          |  node3|
>  | node2   +-----------------+       |
>  |         |                 +-------+
>  +---------+
>
> If we remove overlapping group, we will add a group for node2, another
> group for node3. Then we get three groups.
>
> I am not sure if it is always positive for performance.
>

Neither am I! At the same time our strategy for generating groups is pretty
much flawed for anything with distance > 2, so I'd like to have a saner
setup that doesn't involve fixing groups "after the fact".

I have a sort-of-working hack, I'll make this into a patch and toss it out
for discussion.