Re: [PATCH v2 01/13] mm/mmap: free gathered pages before MMF_OOM_SKIP
From: Balbir Singh
Date: Sun Nov 15 2026 - 20:53:37 EST
On Fri, Nov 13, 2026 at 09:45:19PM +0000, Lorenzo Stoakes (ARM) wrote:
> The OOM killer can quickly trigger again after having selected a
> victim. That victim might not have freed its memory yet, but its mm will
> be marked MMF_OOM_SKIP by this point, so the OOM killer will ignore it.
>
> This can cause the OOM killer to kill an innocent process when the first
> might have freed enough memory to relieve the memory pressure.
>
> exit_mmap() marks the mm MMF_OOM_SKIP once unmap_vmas() returns.
> However unmap_vmas() does not free the last batch of pages it
> gathered. That is left to tlb_finish_mmu(), which runs after
> MMF_OOM_SKIP is set, and the batch can be large.
>
> Fix the issue by freeing the gathered pages of an OOM victim before
> marking the mm MMF_OOM_SKIP.
>
> Note that the task which calls mmput() might not be the victim - if it
> took a reference on the mm (e.g. ptrace), it may hold the last reference
> and release it when it calls mmput().
>
> Therefore we cannot rely on current to tell whether the mm belongs to an
> OOM victim. Instead, partially revert commit b3541d912a84 ("mm: delete
> unused MMF_OOM_VICTIM flag") to reinstate MMF_OOM_VICTIM, which
> mark_oom_victim() sets on the victim's mm, and check that.
>
> Reported-by: Chris Mason <mason@xxxxxxxxxx>
> Fixes: bf3980c85212 ("mm: drop oom code from exit_mmap")
> Cc: stable@xxxxxxxxxxxxxxx
> Reviewed-by: Gregory Price (Meta) <gourry@xxxxxxxxxx>
> Tested-by: Gregory Price (Meta) <gourry@xxxxxxxxxx>
> Acked-by: Michal Hocko <mhocko@xxxxxxxx>
> Reviewed-by: Pedro Falcato <pfalcato@xxxxxxx>
> Acked-by: Usama Arif <usama.arif@xxxxxxxxx>
> Signed-off-by: Lorenzo Stoakes (ARM) <ljs@xxxxxxxxxx>
> ---
> include/linux/mm_types.h | 2 ++
> include/linux/oom.h | 9 +++++++++
> mm/mmap.c | 9 ++++++++-
> mm/oom_kill.c | 4 +++-
> 4 files changed, 22 insertions(+), 2 deletions(-)
>
> diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
> index 1ed35d258d2e..09e63ff12b04 100644
> --- a/include/linux/mm_types.h
> +++ b/include/linux/mm_types.h
> @@ -2030,6 +2030,8 @@ enum {
> #define MMF_TOPDOWN 31 /* mm searches top down by default */
> #define MMF_TOPDOWN_MASK BIT(MMF_TOPDOWN)
>
> +#define MMF_OOM_VICTIM 32 /* mm is the oom victim */
> +
> #define MMF_INIT_LEGACY_MASK (MMF_DUMP_FILTER_MASK |\
> MMF_DISABLE_THP_MASK | MMF_HAS_MDWE_MASK |\
> MMF_VM_MERGE_ANY_MASK | MMF_TOPDOWN_MASK)
> diff --git a/include/linux/oom.h b/include/linux/oom.h
> index 00da05d227e6..a9390cb70541 100644
> --- a/include/linux/oom.h
> +++ b/include/linux/oom.h
> @@ -76,6 +76,15 @@ static inline bool tsk_is_oom_victim(struct task_struct * tsk)
> return tsk->signal->oom_mm;
> }
>
> +/*
> + * Use this helper if tsk->mm != mm and the victim mm needs a special
> + * handling. This is guaranteed to stay true after once set.
> + */
> +static inline bool mm_is_oom_victim(struct mm_struct *mm)
> +{
> + return mm_flags_test(MMF_OOM_VICTIM, mm);
> +}
> +
> /*
> * Checks whether a page fault on the given mm is still reliable.
> * This is no longer true if the oom reaper started to reap the
> diff --git a/mm/mmap.c b/mm/mmap.c
> index 7618e72766a5..4cb18b217048 100644
> --- a/mm/mmap.c
> +++ b/mm/mmap.c
> @@ -1313,11 +1313,18 @@ void exit_mmap(struct mm_struct *mm)
> /* update_hiwater_rss(mm) here? but nobody should be looking */
> /* Use ULONG_MAX here to ensure all VMAs in the mm are unmapped */
> unmap_vmas(&tlb, &unmap);
> +
> + /*
> + * Free the memory before marking MMF_OOM_SKIP so a re-triggered OOM
> + * killer will only skip this mm if it's already freed all it can.
> + */
> + if (mm_is_oom_victim(mm))
> + tlb_flush_mmu(&tlb);
> mmap_read_unlock(mm);
>
> /*
> * Set MMF_OOM_SKIP to hide this task from the oom killer/reaper
> - * because the memory has been already freed.
> + * because the memory has been already unmapped.
> */
> mm_flags_set(MMF_OOM_SKIP, mm);
> mmap_write_lock(mm);
> diff --git a/mm/oom_kill.c b/mm/oom_kill.c
> index 5d48bd862c27..9de57a23893c 100644
> --- a/mm/oom_kill.c
> +++ b/mm/oom_kill.c
> @@ -752,8 +752,10 @@ static void mark_oom_victim(struct task_struct *tsk)
> return;
>
> /* oom_mm is bound to the signal struct life time. */
> - if (!cmpxchg(&tsk->signal->oom_mm, NULL, mm))
> + if (!cmpxchg(&tsk->signal->oom_mm, NULL, mm)) {
> mmgrab(tsk->signal->oom_mm);
> + mm_flags_set(MMF_OOM_VICTIM, mm);
> + }
>
> /*
> * Make sure that the process is woken up from uninterruptible sleep
>
Acked-by: Balbir Singh <balbirs@xxxxxxxxxx>