Re: [PATCH v2] mm/khugepaged: clear MMF_VM_HUGEPAGE on mm_slot_alloc() failure

From: Baolin Wang

Date: Wed May 06 2026 - 01:17:49 EST




On 5/6/26 9:21 AM, Ye Liu wrote:
From: Ye Liu <liuye@xxxxxxxxxx>

__khugepaged_enter() sets MMF_VM_HUGEPAGE before allocating the
corresponding mm_slot. If mm_slot_alloc() fails, the function
returns with the flag set but without inserting the mm into the
khugepaged tracking structures.

This leaves the mm in an inconsistent state: it is marked as
registered (MMF_VM_HUGEPAGE set), but will never be scanned by
khugepaged. Future attempts to register the mm are skipped since
khugepaged_enter_vma() checks the flag and returns early.

Fix this by clearing MMF_VM_HUGEPAGE when mm_slot_alloc() fails,
restoring the ability to retry registration later.

Fixes: 16618670276a ("mm: khugepaged: avoid pointless allocation for struct mm_slot")
Signed-off-by: Ye Liu <liuye@xxxxxxxxxx>
---

Good catch. Feel free to add:
Reviewed-by: Baolin Wang <baolin.wang@xxxxxxxxxxxxxxxxx>