[PATCH] mm/percpu: allow embed allocator when total size fits in vmalloc

[Paul Sherman]

On large NUMA systems (e.g. 4-node 128GB), the physical span between
NUMA nodes may exceed 75% of vmalloc space even though the total percpu
allocation is tiny (e.g. ~6MB for 64 CPUs).

The 75% check was designed for pcpu_get_vm_areas() used by dynamic
percpu chunks, where vmalloc congruency is required. The embed
allocator uses the linear map directly for the first chunk -- physical
addresses are accessible as virtual addresses without vmalloc mapping.
The relevant constraint is total percpu size, not physical span.

Replace the unconditional fallback with a total-size check: if the
aggregate percpu allocation fits within 75% of vmalloc space, proceed
with embed regardless of physical NUMA span. Systems where total percpu
size genuinely exceeds the vmalloc bound retain the page allocator
fallback.

Tested on Sophgo SG2042 (64-hart, 4-NUMA, 128GB DDR4, RISC-V Sv39):
  percpu: embed: span 0x17def7a000 > vmalloc 75% but total 0x5c0000
          fits -- linear map used
  percpu: Embedded 23 pages/cpu s54168 r8192 d31848 u94208

Link: https://lkml.iu.edu/hypermail/linux/kernel/1707.3/00337.html
Cc: Tejun Heo <tj@xxxxxxxxxx>
Cc: Vlastimil Babka <vbabka@xxxxxxx>
Cc: Dennis Zhou <dennis@xxxxxxxxxx>
Signed-off-by: Paul Sherman <shermanpauldylan@xxxxxxxxx>
---

This fix is inert on systems where the original check never fired;
it only activates when physical span exceeds 75% of vmalloc but total
percpu size does not.

Architecture analysis:

  Architecture   | vmalloc      | phys span    | total percpu | result
  ---------------|--------------|--------------|--------------|--------
  RISC-V Sv39    | ~88 GB       | ~102 GB      | ~6 MB        | fixed
  RISC-V Sv48    | ~88 TB       | ~102 GB      | ~6 MB        | unaffected
  RISC-V Sv57    | ~44 PB       | ~102 GB      | ~6 MB        | unaffected
  ARM64 large    | ~248 TB      | varies       | tiny         | unaffected
  32-bit NUMA    | ~128 MB      | varies       | may exceed   | correct fallback

Tejun Heo noted in 2017 [Link] that the only constraint is vmalloc
size relative to NUMA node distances, and that making vmalloc bigger
would be the best fix. On RISC-V Sv39 with 88GB vmalloc and 102GB
physical NUMA span that is not an option -- but the embed allocator
does not need congruent vmalloc mapping for the first chunk anyway.

 mm/percpu.c | 29 ++++++++++++++++++++++++-----
 1 file changed, 24 insertions(+), 5 deletions(-)

diff --git a/mm/percpu.c b/mm/percpu.c
index b0676b8054ed..72695778872c 100644
--- a/mm/percpu.c
+++ b/mm/percpu.c
@@ -3069,13 +3069,32 @@ int __init pcpu_embed_first_chunk(size_t reserved_size, size_t dyn_size,
 
 	/* warn if maximum distance is further than 75% of vmalloc space */
 	if (max_distance > VMALLOC_TOTAL * 3 / 4) {
-		pr_warn("max_distance=0x%lx too large for vmalloc space 0x%lx\n",
+		pr_warn("percpu: embed: max_distance=0x%lx too large for vmalloc space 0x%lx\n",
 				max_distance, VMALLOC_TOTAL);
 #ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK
-		/* and fail if we have fallback */
-		rc = -EINVAL;
-		goto out_free_areas;
-#endif
+		/*
+		 * The embed allocator uses the linear map directly for the
+		 * first chunk -- physical addresses are accessible as virtual
+		 * addresses without vmalloc mapping. The 75% check was designed
+		 * for pcpu_get_vm_areas() (dynamic chunks) where vmalloc
+		 * congruency is required. On large NUMA systems, physical span
+		 * between nodes may exceed vmalloc bounds even though total
+		 * percpu size is tiny. Check total size, not physical span.
+		 */
+		{
+			unsigned long total_size = 0;
+			for (group = 0; group < ai->nr_groups; group++)
+				total_size += (unsigned long)ai->unit_size *
+					ai->groups[group].nr_units;
+			if (total_size > VMALLOC_TOTAL * 3 / 4) {
+				rc = -EINVAL;
+				goto out_free_areas;
+			}
+			pr_info("percpu: embed: span 0x%lx > vmalloc 75%%"
+				" but total 0x%lx fits -- linear map used\n",
+				max_distance, total_size);
+		}
+#endif /* CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK */
 	}
 
 	/*
-- 
2.53.0