Re: [PATCH] mm/mglru: skip oom killing for high order allocs in kswapd

[Kairui Song]

On Tue, Oct 27, 2026 at 10:21 AM Bingfang Guo via B4 Relay
<devnull+bingfangguo.tencent.com@xxxxxxxxxx> wrote:
>
> From: Bingfang Guo <bingfangguo@xxxxxxxxxxx>
>
> We observed that when MGLRU min_ttl is set and the system is highly
> fragmented, high order allocations (e.g. order == 9) will fail and wake
> up kswapd. It can not make any progress, while pushing generations of
> memcgs so that they all end up younger than min_ttl_ms, repeatedly
> triggering the OOM killer in lru_gen_age_node(). Then the whole process
> repeats until all processes are killed and the kernel panics.
>
> Fix this by skipping OOM killer invocations in lru_gen_age_node() for
> sc->order > PAGE_ALLOC_COSTLY_ORDER, which is also what the normal
> allocation path will do, to fail high order allocations without OOM
> killing.
>
> ==== Testing ====
>
> Tested on a VM with 6 cores and 6GiB of memory, HZ=1000, MGLRU enabled,
> min_ttl_ms set to 1000, and a workload that fragments the memory then
> allocates 2M pages.
>
> Without the patch, kswapd kills all processes and panics the system even
> with plenty of memory available. With the patch, allocations fail but
> the system remains stable.
>
> Fixes: 30d77b7eef019 ("mm/mglru: fix ineffective protection calculation")
> Signed-off-by: Bingfang Guo <bingfangguo@xxxxxxxxxxx>
> ---
>  mm/vmscan.c | 6 ++++++
>  1 file changed, 6 insertions(+)

Thanks! Really nice catch.

> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index 9959ca158ce83..b32777468ead8 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -4407,6 +4407,12 @@ static void lru_gen_age_node(struct pglist_data *pgdat, struct scan_control *sc)
>                         reclaimable = lruvec_is_reclaimable(lruvec, sc, min_ttl);
>         } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL)));
>
> +       /*
> +        * OOM killing won't help high order allocations. Skip and let them fail.

A nit pick, OOM definately helps for high order allocations :-D, maybe
something like, the OOM kill is to prevent thrashing whtn all memcgs
are younger than min_ttl (and maybe merge with the comment & check
below?), so OOM for fragmentation is definitely unwanted.

> +        */
> +       if (sc->order > PAGE_ALLOC_COSTLY_ORDER)
> +               return;
> +
>         /*
>          * The main goal is to OOM kill if every generation from all memcgs is
>          * younger than min_ttl. However, another possibility is all memcgs are

This patch is good to me on its own with a nit fix above.

And checking commit 30d77b7eef019, you might need a Fixes: or maybe
even bring the check back which was removed in that commit?

But for long term, and as a follow up, the whole idea of OOM in kswapd
seems kind of wrong, maybe at least it need to check the water mark
here?