[PATCH v3 2/2] sched/numa: Apply remote socket distance averaging for Hygon CPUs

[Chaohong Guo]

On Intel GNR/CWF platforms with SNC3 enabled,

  commit 4d6dd05d07d0 ("sched/topology: Fix sched domain build error for
  GNR, CWF in SNC-3 mode")

introduced the use of averaged distances to remote‑socket nodes when
constructing scheduling domains. This helped group remote‑socket NUMA
nodes into the same scheduling group, improving load‑balancing decisions.

Hygon model 7 can contain up to six dies per package. When NPS (NUMA
per socket) is enabled, each package exposes six NUMA nodes. We observe
the same issue as on GNR/CWF: the current domain construction incorrectly
merges nodes from different sockets at higher NUMA levels,leading to
suboptimal remote‑socket CPU selection during load balancing.

The following diagram illustrates the node topology (two packages, six
nodes each):

             package 0                        package 1

           ----------------------------------------------
           |            --------------------            |
           |            |                  |            |
        ------       -------             ------       ------
        |  0  |------|  3  |             | 6  |-------| 9  |
        ------       -------             -----        -----
           |            |                  |            |
        -------      -------             -----        ------
        |  5  |------|  2  |             | 11 |-------| 8  |
        -------      -------             -----        -----
           |            |                  |            |
        -------      -------             -----        ------
        |  4  |------|  1  |             | 10 |-------| 7  |
        ------       -------             -----        ------
           |            |                  |            |
           |            --------------------            |
           ----------------------------------------------

>From the perspective of CPUs in node 0, sched domains:
  numa-level 1:  [0]  [3]  [5]
  numa-level 2:  [0, 3, 5]  [2]  [4]
  numa-level 3:  [0, 2-5]  [1-5]
  numa-level 4:  [0-5]  [9]
  numa-level 5:  [0-5, 9]  [6]  [8]
  numa-level 6:  [0-6, 8, 9]   [7]  [11]
  numa-level 7:  [0-9]  [1-11]

NUMA level 5-7， nodes in different socket are grouped together,which
caused CPUs in remote socket are chosen while load balancing.

For better performace, we do the same as Intel is doing on intel GNR/CWF.
That reduces sched domains to 4-level NUMA and eliminates cross-socket
sched groups in each domain:
  numa-level 1:  [0]  [3]  [5]
  numa-level 2:  [0, 3, 5]  [2]  [4]
  numa-level 3:  [0, 2-5]  [1-5]
  numa-level 4:  [0-5]  [6-11]

Co-developed-by: Zhiquan Li <lizhiquan@xxxxxxxxxxxxxx>
Signed-off-by: Zhiquan Li <lizhiquan@xxxxxxxxxxxxxx>
Signed-off-by: Chaohong Guo <guochaohong@xxxxxxxxxxxxxx>
--
2.18.2
---
 arch/x86/kernel/smpboot.c | 21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c
index cb999feb66b0..14a5fb92317c 100644
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -675,17 +675,26 @@ int arch_sched_node_distance(int from, int to)
 {
 	int d = node_distance(from, to);
 
+	if (topology_max_packages() == 1)
+		return d;
+
+	/* Handle SNC-3/NPS-6 asymmetries. */
 	switch (boot_cpu_data.x86_vfm) {
 	case INTEL_GRANITERAPIDS_X:
 	case INTEL_ATOM_DARKMONT_X:
-		if (topology_max_packages() == 1 ||
-		    topology_num_nodes_per_package() < 3)
-			return d;
-
+		if (topology_num_nodes_per_package() >= 3)
+			d = slit_cluster_distance(from, to);
+		break;
+	case HYGON_F18_M07:
 		/*
-		 * Handle SNC-3 asymmetries.
+		 * Hygon model 7 CPUs have either 4 or 6 compute dies (aka CDD). All
+		 * 6‑die variants share the same interconnect topology. Apply remote
+		 * socket distance averaging if the CPU exposes 6 dies when NPS is
+		 * enabled.
 		 */
-		return slit_cluster_distance(from, to);
+		if (topology_num_nodes_per_package() >= 6)
+			d = slit_cluster_distance(from, to);
+		break;
 	}
 	return d;
 }
-- 
2.18.2