Re: [PATCH v2 13/13] percpu: queue the balance work on an unbound workqueue
From: Balbir Singh
Date: Sat Nov 14 2026 - 18:07:34 EST
On Fri, Nov 13, 2026 at 09:45:31PM +0000, Lorenzo Stoakes (ARM) wrote:
> From: Chris Mason <mason@xxxxxxxxxx>
>
> A burst of small atomic per-CPU allocations can fail with -ENOMEM while the
> machine has free memory and idle CPUs, as atomic per-CPU allocations are
> only served from pages that are already populated, and those can be used up
> before a worker gets to populate more.
>
> Consider:
>
> caller on CPU 1 kworker bound to CPU 1
> =============== ======================
> allocates 8 bytes, which leaves
> one empty page and 4088 bytes
> schedule_work() woken, waits for CPU 1
> allocates 8 bytes, 1023 times
> next allocation: "no space left"
> gives up CPU 1 populates pages
>
> The allocator keeps a few empty populated pages in reserve, and the work
> item pcpu_balance_work refills them.
>
> pcpu_alloc_noprof() queues the work item using schedule_work(), so only a
> worker bound to the CPU of the caller can run the work item.
>
> The issue is that the caller keeps that CPU - a task keeps it for the rest
> of its time slice, and a softirq keeps it until the softirq ends.
>
> In that time the caller can use up the reserve, resulting in an atomic
> per-CPU allocation failing.
>
> Fix the issue by queueing the work item on an unbound workqueue so that
> another CPU can populate pages while the caller keeps allocating.
>
> Gregory Price hammered atomic per-CPU allocations from one CPU with
> preemption disabled for about a second, with and without this series:
>
> Before After
> Successful allocs 1,188 131,072
> Failed allocs 4,097 0
> Balance worker runs 0 169
> Successful retries 0 4,096
>
> Fixes: 1a4d76076cda ("percpu: implement asynchronous chunk population")
> Signed-off-by: Chris Mason <mason@xxxxxxxxxx>
> [ ljs: tweaked commit message, comment ]
> Tested-by: Gregory Price (Meta) <gourry@xxxxxxxxxx>
> Reviewed-by: Gregory Price (Meta) <gourry@xxxxxxxxxx>
> Signed-off-by: Lorenzo Stoakes (ARM) <ljs@xxxxxxxxxx>
> ---
> mm/percpu.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/mm/percpu.c b/mm/percpu.c
> index f87813e6ad73..39cba1710383 100644
> --- a/mm/percpu.c
> +++ b/mm/percpu.c
> @@ -200,8 +200,9 @@ static bool pcpu_atomic_alloc_failed;
>
> static void pcpu_schedule_balance_work(void)
> {
> + /* Unbound, so another CPU can populate pgs if caller keeps this one. */
> if (pcpu_async_enabled)
> - schedule_work(&pcpu_balance_work);
> + queue_work(system_dfl_wq, &pcpu_balance_work);
> }
>
Can this run havoc on otherwise idle systems? Why not choose a workqueue
with WQ_POWER_EFFICIENT, which becomes unbound when nohz_full is
enabled? I guess it still leaves open the case when nohz_full is not
enabled and we get a burst of atomic allocations?
Balbir