Re: [PATCH 01/13] mm/mmap: free gathered pages before MMF_OOM_SKIP
From: Lorenzo Stoakes (ARM)
Date: Thu Nov 12 2026 - 07:44:32 EST
On Thu, Nov 12, 2026 at 12:43:57PM +0100, Michal Hocko wrote:
> On Wed 11-11-26 18:13:15, Lorenzo Stoakes (ARM) wrote:
> > The OOM killer can quickly trigger again after having selected a
> > victim. That victim might not have freed its memory yet, but its mm will
> > be marked MMF_OOM_SKIP by this point, so the OOM killer will ignore it.
> >
> > This can cause the OOM killer to kill an innocent process when the first
> > might have freed enough memory to relieve the memory pressure.
> >
> > Fix the issue by freeing the memory before marking the mm MMF_OOM_SKIP.
> >
> > Note that the task which calls mmput() might not be the victim - if it
> > took a reference on the mm (e.g. ptrace), it may hold the last reference
> > and release it when it calls mmput().
>
> I do not follow the reasoning here. MMF_OOM_SKIP is called after the
> address space has been unmapped either by the last mmput or the
> oom_reaper (or both when they are racing). Now all your patch does is to
> call tlb_flush_mmu which I suspect is to release batched pages. But why
> is that needed only for the oom victim and why there was no final flush
> in unmap_vmas?
Sorry maybe commit message isn't clear enough. I did have to pare these down
from really massive ones.
And yeah tlb_flush_mmu() is about freeing the batches.
unmap_vmas() doesn't have a final flush, it relies on the caller to figure that
out, and just batches up.
And currently tlb_finish_mmu() comes after MMF_OOM_SKIP is set (in
exit_mmap()), so at that point the memory is unmapped but whatever the last
batch is isn't freed yet, and the batch can be large (PMDs are single
entries if I recall correctly for e.g.).
This only matters for the OOM victim, hence reinstating MMF_OOM_VICTIM, which
made the patch _way_ smaller than the one proposed :)
>
> > diff --git a/mm/mmap.c b/mm/mmap.c
> > index 7618e72766a5..4cb18b217048 100644
> > --- a/mm/mmap.c
> > +++ b/mm/mmap.c
> > @@ -1313,11 +1313,18 @@ void exit_mmap(struct mm_struct *mm)
> > /* update_hiwater_rss(mm) here? but nobody should be looking */
> > /* Use ULONG_MAX here to ensure all VMAs in the mm are unmapped */
> > unmap_vmas(&tlb, &unmap);
> > +
> > + /*
> > + * Free the memory before marking MMF_OOM_SKIP so a re-triggered OOM
> > + * killer will only skip this mm if it's already freed all it can.
> > + */
> > + if (mm_is_oom_victim(mm))
> > + tlb_flush_mmu(&tlb);
> > mmap_read_unlock(mm);
> >
> > /*
> > * Set MMF_OOM_SKIP to hide this task from the oom killer/reaper
> > - * because the memory has been already freed.
> > + * because the memory has been already unmapped.
> > */
> > mm_flags_set(MMF_OOM_SKIP, mm);
> > mmap_write_lock(mm);
>
> --
> Michal Hocko
> SUSE Labs
--
Cheers, Lorenzo