[PATCH v4 33/38] mm: eliminate VMA_SPECIAL_FLAGS usage when hugetlb explicitly tested

From: Lorenzo Stoakes (ARM)

Date: Sat Oct 03 2026 - 13:12:50 EST


It is now an invariant that VMA_IO_BIT is not set by mm-managed mappings,
so each existing VMA_SPECIAL_FLAGS test need only test for
VMA_DONTEXPAND_BIT, VMA_PFNMAP_BIT and VMA_MIXEDMAP_BIT.

This is precisely a test for a fixed or non-mm-managed mapping.

Update a number of callsites which already explicitly handle hugetlb
mappings.

vma_supports_mlock() and ksm_compatible() also explicitly bail on droppable
mappings - excluding non-mm-managed, fixed or droppable mappings is exactly
what vma_is_mm_backed() tests for, so in these cases use this predicate.

should_skip_vma() tests for locked, fixed or non-mm-managed memory (having
already excluded hugetlb mappings) so simply test for those there.

No functional change intended.

Reviewed-by: Zi Yan <ziy@xxxxxxxxxx>
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@xxxxxxxxxx>
---
mm/internal.h | 4 +---
mm/ksm.c | 4 +---
mm/vmscan.c | 3 ++-
3 files changed, 4 insertions(+), 7 deletions(-)

diff --git a/mm/internal.h b/mm/internal.h
index b5f7be8f65d8..e57c35e5f90b 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -1112,9 +1112,7 @@ static inline struct file *maybe_unlock_mmap_for_io(struct vm_fault *vmf,

static inline bool vma_supports_mlock(const struct vm_area_struct *vma)
{
- if (vma_test_any_mask(vma, VMA_SPECIAL_FLAGS))
- return false;
- if (vma_test_single_mask(vma, VMA_DROPPABLE))
+ if (!vma_is_mm_backed(vma))
return false;
if (vma_is_dax(vma) || vma_is_hugetlb(vma))
return false;
diff --git a/mm/ksm.c b/mm/ksm.c
index 624f37975e12..ed66a90e0031 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -747,9 +747,7 @@ static bool ksm_compatible(const struct file *file, vma_flags_t vma_flags)
if (vma_flags_test_any(&vma_flags, VMA_SHARED_BIT, VMA_MAYSHARE_BIT,
VMA_HUGETLB_BIT))
return false;
- if (vma_flags_test_single_mask(&vma_flags, VMA_DROPPABLE))
- return false;
- if (vma_flags_test_any_mask(&vma_flags, VMA_SPECIAL_FLAGS))
+ if (!vma_flags_is_mm_backed(&vma_flags))
return false;
if (file_is_dax(file))
return false;
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 12ed3388ee41..3b318efa2916 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -3477,7 +3477,8 @@ static int should_skip_vma(unsigned long start, unsigned long end, struct mm_wal
if (!vma_has_recency(vma))
return true;

- if (vma->vm_flags & (VM_LOCKED | VM_SPECIAL))
+ if (vma_test(vma, VMA_LOCKED_BIT) || !vma_is_mm_managed(vma) ||
+ vma_is_fixed_mapping(vma))
return true;

if (vma == get_gate_vma(vma->vm_mm))

--
2.55.0