Re: [PATCH v2 0/2] mm/mempolicy: stop copying state in the interleave paths

[Andrew Morton]

On Thu, 17 Sep 2026 20:12:01 -0400 Gregory Price <gourry@xxxxxxxxxx> wrote:

> The interleave node selectors and bulk allocators take copies of
> nodemasks and node weights (for weighted interleave) in the fault path.
> Both of these copies can be entirely eliminated.
> 
> For node weights, use SRCU to pin the weights in place.  This eliminates
> a copy and a kmalloc from the bulk allocator path.
> 
> For nodemasks, we can operate directly on pol->nodes as long as we bounds
> check the walk.  A concurrent rebind can shrink the mask, or tear the read
> of it so the mask appears empty.
> 
>  - The interleave node selectors fall back to numa_node_id() when that
>    happens, which is what they already did when a copy came back empty.
> 
>  - The bulk allocator simply returns what it managed to allocate.
> 
> The node count and weight totals are read separately from the nodemask
> walk that consumes them - creating a time-of-check / time-of-use race.
> Just clamp the walk to a single pass (number of nodes), and clamp each
> bulk allocation chunk to the space left in the request.
> 
> The cost is distribution accuracy during a rebind.  The copies never
> corrected for that either - they only kept the code from dividing by
> zero and overrunning the allocation request.

Thanks, I updated mm-unstable to this version.

> v2: Comments and commit message [David]

Here's how v2 altered mm.git:


 mm/mempolicy.c |   11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

--- a/mm/mempolicy.c~b
+++ a/mm/mempolicy.c
@@ -2231,8 +2231,13 @@ static unsigned int weighted_interleave_
 
 	/*
 	 * The target was calculated in a separate loop, and a concurrent
-	 * rebind can change the total number of nodes.  Clamp this loop to
-	 * a single pass (nnodes) to keep the walk bounded by node count.
+	 * rebind can change the contents of pol->nodes as we calculate.
+	 * Access is safe, in the worst case we suddenly perceive an empty
+	 * nodemask and return numa_node_id() below - otherwise we may
+	 * simply cause a skew in allocations.
+	 *
+	 * Clamp this loop to a single pass (nnodes) to keep the walk
+	 * bounded by node count.
 	 */
 	while (target && nnodes-- && nid < MAX_NUMNODES) {
 		/* detect system default usage */
@@ -2266,6 +2271,8 @@ static unsigned int interleave_nid(struc
 		return numa_node_id();
 	target = ilx % nnodes;
 	nid = first_node(pol->nodes);
+
+	/* A concurrent cpuset rebind may cause us to see an empty nodemask */
 	for (i = 0; i < target && nid < MAX_NUMNODES; i++)
 		nid = next_node_in(nid, pol->nodes);
 
_