[PATCH] mm/page_alloc: validate zonelist slots against concurrent rebuilds

[Gregory Price (Meta)]

Onlining the first block of a zone (e.g. via virtio-mem or a DIMM)
rebuilds every node's zonelists in place. An allocation in flight can
then be served from a zone it never asked for: a GFP_KERNEL allocation
can get a page from ZONE_MOVABLE.

The allocation succeeds, so the zonelist-retry logic never triggers,
and an unmovable page in ZONE_MOVABLE breaks offline of that block.

Recheck each zone the loop visits against highest_zoneidx and the
nodemask, using the zone pointer itself. This covers both cases and
is a no-op without a concurrent rebuild.

Reproduced with a kprobe harness holding a GFP_KERNEL|__GFP_THISNODE
allocation across the online.

Fixes: c33d6c06f60f7 ("mm, page_alloc: avoid looking up the first zone in a zonelist twice")
Cc: stable@xxxxxxxxxxxxxxx
Signed-off-by: Gregory Price (Meta) <gourry@xxxxxxxxxx>
---
 mm/page_alloc.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 6ed36ba4d055..a1254f2e1892 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -3782,6 +3782,19 @@ static inline unsigned int alloc_flags_cma(gfp_t gfp_mask)
 	return ALLOC_DEFAULT;
 }
 
+/*
+ * The zonelist can be rebuilt in place while it is walked, so the iterator's
+ * filters may have passed a slot that now holds a different zone.
+ * Recheck the zone itself before using it.
+ */
+static inline bool zonelist_zone_valid(struct zone *zone,
+				       const struct alloc_context *ac)
+{
+	if (zone_idx(zone) > ac->highest_zoneidx)
+		return false;
+	return !ac->nodemask || node_isset(zone_to_nid(zone), *ac->nodemask);
+}
+
 /*
  * get_page_from_freelist goes through the zonelist trying to allocate
  * a page.
@@ -3810,6 +3823,9 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int order, int alloc_flags,
 		struct page *page;
 		unsigned long mark;
 
+		if (unlikely(!zonelist_zone_valid(zone, ac)))
+			continue;
+
 		if (cpusets_enabled() &&
 			(alloc_flags & ALLOC_CPUSET) &&
 			!__cpuset_zone_allowed(zone, gfp_mask))
@@ -5234,6 +5250,9 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 	for_next_zone_zonelist_nodemask(zone, z, ac.highest_zoneidx, ac.nodemask) {
 		unsigned long mark;
 
+		if (unlikely(!zonelist_zone_valid(zone, &ac)))
+			continue;
+
 		if (cpusets_enabled() && (alloc_flags & ALLOC_CPUSET) &&
 		    !__cpuset_zone_allowed(zone, gfp)) {
 			continue;
-- 
2.56.0