[PATCH mm-hotfixes v2 0/2] mm/huge_memory: fix huge_zero_pfn race

[Lorenzo Stoakes (ARM)]

There is a subtle race in the reference-counted huge_zero_folio
implementation.

The fast path atomic logic fails to account for the fact that the
shrinker (which drops the final huge_zero_refcount pin) can overwrite
huge_zero_pfn with the ~0UL sentinel value in shrink_huge_zero_folio_scan()
after a racing get_huge_zero_folio() installed a valid value there.

This results in huge_zero_folio being correctly set but huge_zero_pfn being
set incorrectly and thus is_huge_zero_pfn() and consequently
is_huge_zero_pmd() will misidentify the huge zero folio as being an
ordinary THP folio.

This can result in the huge zero folio being split and otherwise treated
incorrectly.

The solution to this is very subtle as there is an atomic fast path, and
thus ordering in weakly ordered architectures has to be treated very
carefully.

The first commit fixes the issue by introducing a spinlock around
huge_zero_[pfn, folio, refcount] write, with careful consideration paid to
load/store ordering in the fast path. It is placed first and kept as small
as possible so that it can be backported on its own.

The second commit is a pure cleanup which reworks the
CONFIG_PERSISTENT_HUGE_ZERO_FOLIO logic to better separate the persistent
logic from the dynamically allocated one.

Andrew - I know you don't like a mix of fix/cleanup, but thought it'd be
easier to keep the 2 together as there's a dependency.

---
v2:
- Reordered the series so the fix now comes first with the
  CONFIG_PERSISTENT_HUGE_ZERO_FOLIO separation following it as a pure
  cleanup as per David.
- Fixed up wording as per David.
- Cleaned up Cc's since dependency of fix on cleanup no longer exists.

v1:
https://patch.msgid.link/20260728-fix-refcounted-huge-zero-v1-0-3f261f5447b4@xxxxxxxxxx

To: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
To: David Hildenbrand <david@xxxxxxxxxx>
To: Zi Yan <ziy@xxxxxxxxxx>
To: Baolin Wang <baolin.wang@xxxxxxxxxxxxxxxxx>
To: "Liam R. Howlett" <liam@xxxxxxxxxxxxx>
To: Nico Pache <npache@xxxxxxxxxx>
To: Ryan Roberts <ryan.roberts@xxxxxxx>
To: Dev Jain <dev.jain@xxxxxxx>
To: Barry Song <baohua@xxxxxxxxxx>
To: Lance Yang <lance.yang@xxxxxxxxx>
To: Usama Arif <usama.arif@xxxxxxxxx>
To: Pankaj Raghav <p.raghav@xxxxxxxxxxx>
To: Hannes Reinecke <hare@xxxxxxx>
To: Hugh Dickins <hughd@xxxxxxxxxx>
To: Yang Shi <shy828301@xxxxxxxxx>
To: Kiryl Shutsemau <kas@xxxxxxxxxx>
Cc: linux-mm@xxxxxxxxx
Cc: linux-kernel@xxxxxxxxxxxxxxx
Cc: Hengbin Zhang <uqbarz@xxxxxxxxx>
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@xxxxxxxxxx>

---
Lorenzo Stoakes (ARM) (2):
      mm/huge_memory: fix huge_zero_pfn race
      mm/huge_memory: separate out CONFIG_PERSISTENT_HUGE_ZERO_FOLIO logic

 mm/huge_memory.c | 189 ++++++++++++++++++++++++++++++++++---------------------
 1 file changed, 116 insertions(+), 73 deletions(-)
---
base-commit: 5db85e34ec49beee590676d55f13e8e2b14c742f
change-id: 20260728-fix-refcounted-huge-zero-21cb07b4fa3e

Cheers,
-- 
Lorenzo Stoakes (ARM) <ljs@xxxxxxxxxx>