[PATCH] mm: shmem: ignore sysfs configs for shmem forced collapse

[Baolin Wang]

According to Documentation/mm/transhuge.rst, MADV_COLLAPSE is expected to
ignore any THP or mTHP interface settings.

However, after commit 6beeab870e70 ("mm: shmem: move shmem_huge_global_enabled()
into shmem_allowable_huge_orders()"), performing MADV_COLLAPSE on shmem
will depend on /sys/.../hugepages-2048kB/shmem_enabled being set to "inherit"
(although that is the default), which can cause MADV_COLLAPSE to fail
unexpectedly.

Fix this by returning the result of shmem_huge_global_enabled() directly
when MADV_COLLAPSE is requested, allowing PMD-order collapse while ignoring
shmem THP/mTHP settings.

Fixes: 6beeab870e70 ("mm: shmem: move shmem_huge_global_enabled() into shmem_allowable_huge_orders()")
Reported-by: Kiryl Shutsemau (Meta) <kas@xxxxxxxxxx>
Closes: https://lore.kernel.org/all/20260908125105.1510704-7-kirill@xxxxxxxxxxxxx/
Signed-off-by: Baolin Wang <baolin.wang@xxxxxxxxxxxxxxxxx>
---
Hi Kiryl,

I did not update the selftests, as I hope this fix will be merged first.
After this fix has landed, I can send a patch to update the selftests.
---
 mm/shmem.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/mm/shmem.c b/mm/shmem.c
index c92ed17dbc4a..b572c60f2af8 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -2049,8 +2049,11 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode,
 
 	global_orders = shmem_huge_global_enabled(inode, index, write_end,
 						  shmem_huge_force, vma, vm_flags);
-	/* Tmpfs huge pages allocation */
-	if (!vma || !vma_is_anon_shmem(vma))
+	/*
+	 * Tmpfs huge pages allocation or forced collapse ignores
+	 * sysfs configs.
+	 */
+	if (!vma || !vma_is_anon_shmem(vma) || shmem_huge_force)
 		return global_orders;
 
 	/*
-- 
2.47.3