[RFC PATCH 6/7] memcg: remove synchronous memory.high enforcement
From: Shakeel Butt
Date: Sat Nov 21 2026 - 21:48:44 EST
A task stuck in memcg limit enforcement in the charge path keeps its
locks, and tasks in other cgroups that need them wait for a limit that
has nothing to do with them.
memory.max is a hard limit, so the charge path has to enforce it.
memory.high doesn't: since commit b23afb93d317 ("memcg: punt high
overage reclaim to return-to-userland path"), the charge only records
the overage, and the task reclaims and is throttled on return to
userspace, where it holds no locks. Avoiding this priority inversion
was one of the reasons for that commit.
But return to userspace is too late when one system call charges a
lot, like mlock() of a large range. So commit c9afe31ec443 ("memcg:
synchronously enforce memory.high for large overcharges") made a
charge that can block handle the overage right away once more than a
batch is pending. That step reclaims and sleeps, up to 2s per call,
under whatever locks the caller holds: mmap_lock while mlock() faults
in a MADV_HUGEPAGE range, or invalidate_lock and locked folios in
readahead. It also misses THP charges in other ranges, which can't
block under the default defrag setting.
The previous patches handle the overage where the task holds no locks,
or only the locks of its file, in the operations that charge the most
in one call: populating memory, shmem fallocate, buffered writes,
reads into user memory and MADV_COLLAPSE. Remove the synchronous step.
Other paths now handle the overage only on return to userspace, and
memory.max still limits them. Many can get a hook later. For example,
reading /dev/urandom into an untouched buffer reaches 513M under
memory.high=64M. With THP on, it did before too, but with small pages
the synchronous step used to throttle it. Another is mprotect() making
an mlocked private range writable, which copies its pages with
mmap_lock held for writing.
Update the memory.high documentation to say when tasks are throttled.
Suggested-by: Tejun Heo <tj@xxxxxxxxxx>
Signed-off-by: Shakeel Butt <shakeel.butt@xxxxxxxxx>
---
Documentation/admin-guide/cgroup-v2.rst | 12 ++++++++++--
mm/memcontrol.c | 20 ++++++--------------
2 files changed, 16 insertions(+), 16 deletions(-)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
index 9e3c67b02ae2..215710a44a88 100644
--- a/Documentation/admin-guide/cgroup-v2.rst
+++ b/Documentation/admin-guide/cgroup-v2.rst
@@ -1386,6 +1386,14 @@ write "max"; to set it to zero explicitly, write "0".
over the high boundary, the processes of the cgroup are
throttled and put under heavy reclaim pressure.
+ A task is throttled when it returns to userspace after
+ charging, and at some points in long operations, such as
+ populating memory or large buffered reads and writes. A
+ system call that allocates a lot without reaching such a
+ point can take the usage over the high boundary by up to
+ the amount it allocates. Kernel threads are not throttled:
+ a worker reclaims for them instead.
+
Going over the high limit never invokes the OOM killer and
under extreme conditions the limit may be breached. The high
limit should be used in scenarios where an external process
@@ -1396,8 +1404,8 @@ write "max"; to set it to zero explicitly, write "0".
reclaim is bypassed. This is useful for admin processes that
need to dynamically adjust the job's memory limits without
expending their own CPU resources on memory reclamation. The
- job will trigger the reclaim and/or get throttled on its
- next charge request.
+ job will reclaim and/or get throttled after it charges
+ memory again.
Please note that with O_NONBLOCK, there is a chance that the
target memory cgroup may take indefinite amount of time to
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 6d147153bd06..4771730e017a 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -2615,8 +2615,9 @@ static unsigned long calculate_high_delay(unsigned int nr_pages,
}
/*
- * Handle pending high overage during a blocking charge or on return
- * to userspace.
+ * Handle pending high overage on return to userspace, or at a point in a
+ * long-running operation where the task can sleep (see
+ * mem_cgroup_over_high_throttle()).
*/
void __mem_cgroup_handle_over_high(gfp_t gfp_mask)
{
@@ -2725,9 +2726,9 @@ static void memcg_enforce_high(struct mem_cgroup *memcg, unsigned int nr_pages,
!current->mm || !mm_match_cgroup(current->mm, memcg);
/*
- * Check the charged cgroup and its parents. Local charges normally defer
- * reclaim until return to userspace so it can use GFP_KERNEL. Large
- * pending overages may be handled synchronously below. The handler looks
+ * Check the charged cgroup and its parents. Local charges defer reclaim
+ * until return to userspace, or until a long-running operation reaches
+ * a point where it can sleep and calls the handler. The handler looks
* up current->mm and rechecks the limits. If the task moved to another
* cgroup in the meantime, the handler works on the new cgroup instead.
* That is rare and costs little.
@@ -2765,15 +2766,6 @@ static void memcg_enforce_high(struct mem_cgroup *memcg, unsigned int nr_pages,
current->memcg_nr_pages_over_high += nr_pages;
set_notify_resume(current);
}
-
- /*
- * Earlier charges may have left high overage pending even if this charge
- * found no new overage. Handle it now if this charge can block, to limit
- * overrun while the task remains in the kernel.
- */
- if (current->memcg_nr_pages_over_high > MEMCG_CHARGE_BATCH &&
- gfpflags_allow_blocking(gfp_mask))
- __mem_cgroup_handle_over_high(gfp_mask);
}
static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask,
--
2.53.1-Meta