Re: [PATCH] mm: nommu: set MMF_OOM_SKIP in exit_mmap()

[Alexis Bouzigues]

On Fri, Oct 16, 2026 at 08:54:56AM +0200, Michal Hocko wrote:
> Correct. The question is why killing the child, which has passed
> exit_mmap stage sucessfully didn't help.

There needs to be ongoing memory pressure otherwise the
parent's allocation succeeds and does not hit oom_kill.c or a livelock.

If there is memory pressure, then the allocation hits out_of_memory() and
then blocked in the __alloc_pages_slowpath/oom_evaluate_task retry loop.

> a) a corner case of a valid nommu application or this is
>    a theoretical problem

I think it is mainly a correctness fix (see below). I have only hit it
with benchmarks, including the /init provided earlier from which I obtained
the backtrace. I encountered this bug initially while running a
benchmark to test memory reductions.

> b) why this doesn't break otherwise very fragile nommu oom
>    handling

It does not break nommu oom handling because it places the flag only in
exit_mmap() right after the process has called the last delete_vma,
guaranteeing it has freed all of its private memory.

This flag is used similarly on mmu (mmap.c) and in previous versions of
nommu, until the Fixes tag 212925802454 ("mm: oom: let oom_reap_task
and exit_mmap run concurrently"). It took it away from the shared
__mmput() and only kept it in the mmu-only path (exit_mmap of
mm/mmap.c).

  void exit_mmap(struct mm_struct *mm)
  {
          ...
          unmap_vmas(&tlb, &unmap);
          mmap_read_unlock(mm);

          /*
           * Set MMF_OOM_SKIP to hide this task from the oom killer/reaper
           * because the memory has been already freed.
           */
          mm_flags_set(MMF_OOM_SKIP, mm);
          ...

This change places it in the same exit_mmap() of nommu.c.

The MMF_OOM_SKIP mm flag comes from your commit a while ago
862e3073b3ee ("mm, oom: get rid of signal_struct::oom_victims") and had
the same rationale for nommu to mark the pid's mm to hide the process from
oom killer:

    After "oom: keep mm of the killed task available" we can safely detect
    an oom victim by checking task->signal->oom_mm so we do not need the
    signal_struct counter anymore so let's get rid of it.

    This alone wouldn't be sufficient for nommu archs because
    exit_oom_victim doesn't hide the process from the oom killer anymore.
    We can, however, mark the mm with a MMF flag in __mmput.  We can reuse
    MMF_OOM_REAPED and rename it to a more generic MMF_OOM_SKIP.

Let me know if this answers your questions!