Re: [PATCH 01/13] mm/mmap: free gathered pages before MMF_OOM_SKIP

From: Michal Hocko

Date: Thu Nov 12 2026 - 06:53:28 EST


On Wed 11-11-26 18:13:15, Lorenzo Stoakes (ARM) wrote:
> The OOM killer can quickly trigger again after having selected a
> victim. That victim might not have freed its memory yet, but its mm will
> be marked MMF_OOM_SKIP by this point, so the OOM killer will ignore it.
>
> This can cause the OOM killer to kill an innocent process when the first
> might have freed enough memory to relieve the memory pressure.
>
> Fix the issue by freeing the memory before marking the mm MMF_OOM_SKIP.
>
> Note that the task which calls mmput() might not be the victim - if it
> took a reference on the mm (e.g. ptrace), it may hold the last reference
> and release it when it calls mmput().

I do not follow the reasoning here. MMF_OOM_SKIP is called after the
address space has been unmapped either by the last mmput or the
oom_reaper (or both when they are racing). Now all your patch does is to
call tlb_flush_mmu which I suspect is to release batched pages. But why
is that needed only for the oom victim and why there was no final flush
in unmap_vmas?

> diff --git a/mm/mmap.c b/mm/mmap.c
> index 7618e72766a5..4cb18b217048 100644
> --- a/mm/mmap.c
> +++ b/mm/mmap.c
> @@ -1313,11 +1313,18 @@ void exit_mmap(struct mm_struct *mm)
> /* update_hiwater_rss(mm) here? but nobody should be looking */
> /* Use ULONG_MAX here to ensure all VMAs in the mm are unmapped */
> unmap_vmas(&tlb, &unmap);
> +
> + /*
> + * Free the memory before marking MMF_OOM_SKIP so a re-triggered OOM
> + * killer will only skip this mm if it's already freed all it can.
> + */
> + if (mm_is_oom_victim(mm))
> + tlb_flush_mmu(&tlb);
> mmap_read_unlock(mm);
>
> /*
> * Set MMF_OOM_SKIP to hide this task from the oom killer/reaper
> - * because the memory has been already freed.
> + * because the memory has been already unmapped.
> */
> mm_flags_set(MMF_OOM_SKIP, mm);
> mmap_write_lock(mm);

--
Michal Hocko
SUSE Labs