Re: [PATCH 2/3] mm/memory: do not report OOM if insert_pfn() races a huge fault

From: Zi Yan

Date: Tue Nov 17 2026 - 13:04:41 EST


On 15 Nov 2026, at 15:58, Lorenzo Stoakes (ARM) wrote:

> From: Eliav Farber <farbere@xxxxxxxxxx>
>
> A driver that implements both ->fault and ->huge_fault must anticipate
> concurrent use of both.
>
> This can happen, for example, if a user calls prctl(PR_SET_THP_DISABLE)
> concurrent with faulting, as that only holds the mmap write lock and thus
> does not exclude faulting threads stabilised by VMA lock.
>
> This is not currently safely handled by insert_pfn(), however, which
> obtains its PTE entry via get_locked_pte(), and treats a NULL return value
> as an allocation failure, returning VM_FAULT_OOM.
>
> The issue is that get_locked_pte() also returns NULL if the PMD entry is
> not a page table which can happen if a concurrent huge fault installs a
> huge PMD entry between __handle_mm_fault() observing the PMD entry to be
> empty and insert_pfn() being invoked.
>
> This can result in VM_FAULT_OOM being reported incorrectly.
>
> Address this by proactively checking whether a page table allocation
> actually failed, and returning VM_FAULT_NOPAGE if the NULL indicates that
> we were raced instead.
>
> The same assumption is made in insert_page(), however only DAX implements
> ->huge_fault and can reach it, and it serialises faults of different orders
> with its own mapping entry lock, so it does not require a change.
>
> Target the commit introducing the first driver able to do this.
>
> Link: https://lore.kernel.org/all/20261112051440.16140-3-farbere@xxxxxxxxxx/
> Fixes: f9e54c3a2f5b ("vfio/pci: implement huge_fault support")
> Cc: stable@xxxxxxxxxxxxxxx
> Suggested-by: Jose A. Perez de Azpillaga <azpijr@xxxxxxxxx>
> Signed-off-by: Eliav Farber <farbere@xxxxxxxxxx>
> [ ljs: rewrote commit message, trimmed comment ]
> Signed-off-by: Lorenzo Stoakes (ARM) <ljs@xxxxxxxxxx>
> ---
> mm/memory.c | 11 +++++++++--
> 1 file changed, 9 insertions(+), 2 deletions(-)
>
Makes sense.

Reivewed-by: Zi Yan <ziy@xxxxxxxxxx>


Related to get_locked_pte(), there are other callers, like __text_poke()
from arch/x86/kernel/alternative.c, assuming get_locked_pte() will never
encounter a PMD leaf (especially with the VM_BUG_ON() removed in patch 1).
I was worried about removing VM_BUG_ON() would also remove a check
for these callers. But LLM managed to convince me that the removed
VM_BUG_ON() is already weak and not present when DEBUG_VM is off
or THP is off. So all good.

Best Regards,
Yan, Zi