Re: [RFC PATCH] mm/vmalloc: fix vmalloc which may return null if called with __GFP_NOFAIL

From: Gao Xiang
Date: Wed May 08 2024 - 11:11:07 EST


Hi,

On 2024/5/8 22:43, Hailong Liu wrote:
On Wed, 08. May 21:41, Gao Xiang wrote:

+Cc Michal,

On 2024/5/8 20:58, hailong.liu@xxxxxxxx wrote:
From: "Hailong.Liu" <hailong.liu@xxxxxxxx>

Commit a421ef303008 ("mm: allow !GFP_KERNEL allocations for kvmalloc")
includes support for __GFP_NOFAIL, but it presents a conflict with
commit dd544141b9eb ("vmalloc: back off when the current task is
OOM-killed"). A possible scenario is as belows:

process-a
kvcalloc(n, m, GFP_KERNEL | __GFP_NOFAIL)
__vmalloc_node_range()
__vmalloc_area_node()
vm_area_alloc_pages()
--> oom-killer send SIGKILL to process-a
if (fatal_signal_pending(current)) break;
--> return NULL;

to fix this, do not check fatal_signal_pending() in vm_area_alloc_pages()
if __GFP_NOFAIL set.

Reported-by: Oven <liyangouwen1@xxxxxxxx>
Signed-off-by: Hailong.Liu <hailong.liu@xxxxxxxx>

Why taging this as RFC here? It seems a corner-case fix of
commit a421ef303008

Thanks,
Gao Xiang


Hi Gao Xiang:

RFC here to wait for a better way to handle this case :).
IMO, if vmalloc support __GFP_NOFAIL it should not return
null even system is deadlock on memory.

The starting point is that kmalloc doesn't support __GFP_NOFAIL
if order > 1 (even for very short temporary uses), see:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/mm/page_alloc.c?h=v6.8#n2896

but it is possible if we have such page pointer array (since two
(order-1) pages can only keep 1024 8-byte entries, it can happen
if compression ratios are high), and kvmalloc(__GFP_NOFAIL) has
already been supported for almost two years, it will fallback to
order-0 allocation as described in commit e9c3cda4d86e
("mm, vmalloc: fix high order __GFP_NOFAIL allocations").

With my limited understanding, I'm not sure why it can cause
deadlock here since it will fallback to order-0 allocation then,
and such allocation is just for short temporary uses again
because kmalloc doesn't support order > 1 short memory
allocation strictly.

Thanks,
Gao Xiang


--

Best Regards,
Hailong.