Re: [PATCH] mm: memcg: use ratelimited stats flush in obj_cgroup_may_zswap()

From: Joanne Koong

Date: Fri Nov 13 2026 - 14:15:31 EST


On Wed, Nov 11, 2026 at 12:12 AM Chengfeng Lin <lin2530632123@xxxxxxxxx> wrote:
>
> Hi Joanne, Yosry, and all,
>
> With existing rstat updates retained, hierarchical per-CPU counters reduced
> single-worker PAGEOUT CPU by 44.69% and whole-program CPU by 21.78% in this
> bare-metal test.
>
> One worker uses MADV_PAGEOUT on 32 MiB of compressible anonymous memory in
> a private cgroup, with THP disabled and memory.zswap.max=64 MiB. The limit
> is not reached. The host is an i7-12700KF with 20 online logical CPUs,
> full preemption, LZO and zsmalloc. Unlimited controls change only the
> zswap limit to max.
>
> A matched parent/commit/parent test of 0efc297a3c49
> ("cgroup/rstat: avoid disabling irqs for O(num_cpu)") showed:
>
> CPU time, ms / 32 MiB Parent A Commit Parent B
> PAGEOUT 17.180 24.464 17.316
> Whole test program 46.215 53.419 46.235
>
> PAGEOUT increased by 7.216 ms (41.84%) and whole-program CPU by 7.193 ms
> (15.56%). Both versions stored 8192 pages and performed 8192 limit-check
> flushes: a finite zswap limit forces a stats flush before each store.
>
> To limit interrupt-disabled time, the commit
> releases and reacquires the rstat lock after processing each possible CPU,
> with cond_resched() in between and cpu_relax() if it does not reschedule.
> Previously this was conditional on rescheduling or lock contention.
> Per-page flushing repeats these extra operations.
>
> On MM baseline 2967f3ccdb5f, I adapted Joanne's hierarchical per-CPU option
> [1]. The limit pre-check uses dedicated counters, retaining MEMCG_ZSWAP_B
> rstat updates and the IRQ-latency protection. Extra updates are included:
>
> CPU time, ms / 32 MiB Base A Counter Base B
> PAGEOUT 23.903 13.113 23.514
> Whole test program 49.247 38.569 49.363
>
> PAGEOUT CPU fell by 10.595 ms (44.69%) and whole-program CPU by 10.737 ms
> (21.78%). Stores were unchanged; forced flushes fell from 8192 to zero.
> These two comparisons use different baselines.
>
> Both comparisons used fresh A/B/A boots, two warmups and nine clean
> samples per condition per boot. PAGEOUT return drift was 0.80%/-1.63%,
> with maximum within-boot CV of 2.82%/3.55%. PAGEOUT measures worker CPU;
> whole-program CPU includes preparation, re-access, verification
> and cleanup, but not external cgroup setup or host-wide background work.
>
> A single-worker unlimited control changed PAGEOUT/whole-program CPU by
> +0.41%/+0.86%, with PAGEOUT CV up to 4.74%; there was no clear difference.
>
> A supplementary A/B/A test compared the MM baseline with the counter
> derivative, with eight workers pinned to different P cores, each processing
> 32 MiB (256 MiB total). Their shared cgroup had memory.max=2 GiB and
> memory.swap.max=1 GiB.
> The zswap limit remained 64 MiB; compressed usage was about 18.75 MiB,
> below the limit. Counter-kernel means follow, with baseline-relative changes:
>
> Eight-worker metric, per 256 MiB Finite limit Unlimited
> Summed PAGEOUT CPU 199.238 ms (-84.04%) 196.272 ms (+1.17%)
> Summed re-access/verify CPU 215.365 ms ( -1.36%) 214.950 ms (+0.09%)
> Whole-program CPU 663.404 ms (-61.39%) 660.535 ms (+0.24%)
> Whole-program elapsed time 94.618 ms (-58.26%) 95.183 ms (+0.99%)
>
> Both kernels stored 65,536 pages. Finite-limit forced flushes fell from
> 65,536 to zero; unlimited controls performed none. Counter-kernel PAGEOUT
> CVs were 4.51%/4.91%, with baseline return drift of +5.10%/+0.97%.
> All nine finite-limit counter samples were below both baseline groups,
> but baseline drift limits the precision of the gain estimate.
> The small unlimited increase does not
> establish a regression. Re-access and elapsed-time CVs were at most 2.81%.
>
> Elapsed time includes preparation and cleanup; it is not summed CPU time.
> The re-access metric includes reading and verifying the data; zswap
> decompression was not timed separately.
>
> Both kernels passed bounded leaf/ancestor-limit, dynamic-limit,
> cross-CPU-freeing and cgroup-offlining checks. Deep cgroup hierarchies, CPU
> hotplug, and 32-bit systems were not tested.
>
> I hope these results help assess the benefit and update cost of dedicated
> counters while retaining the existing statistics. The reproducer sources and
> the measured counter patch are attached.
>
> [1] https://github.com/joannekoong/linux/commit/9092e0057f038ffca43ae3f35faf58c33b1d7c9b
>

Hi Chengfeng,

Thank you for benchmarking this and sharing your results. Would you
mind using your setup to test how this compares to the hierarchical
per-CPU option?:

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 091401cecf39..d2b0db311a7d 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -6324,8 +6324,7 @@ bool obj_cgroup_may_zswap(struct obj_cgroup *objcg)
break;
}

- /* Force flush to get accurate stats for charging */
- __mem_cgroup_flush_stats(memcg, true);
+ mem_cgroup_flush_stats(memcg);
pages = memcg_page_state(memcg, MEMCG_ZSWAP_B) / PAGE_SIZE;
if (pages < max)
continue;

I ran a quick test on this back in August and saw these results:
"Quick benchmarks at depth=4 with zswap.max set at every level, 20
workers on 80 cpus: rstat forced = ~7,600 stores/s, rstat unforced =
~300k stores/s, and hierarchical per-cpu counters = ~440k stores/s.
The hierarchical per-cpu counters outperforms rstat unforced on the
stores given that it removes the flush, but we already see massive
improvements from using rstat unforced over rstat forced". I'm curious
if you see the same results on your more-well-thought-out benchmarking
setup.

I don't think the flush needs to be forced here for zswap. The
kerneldoc for obj_cgroup_may_zswap() describes it as an "optimistic
pre-check" and I think it currently already functions as an estimate
rather than an exact state of truth. imo, it'd be preferable to go
with non-forced flushing than hierarchical per-cpu counters, as it
incurs zero additional overhead on updates and is code-wise simpler.
For fixing the writeback problem, we're planning to go with this as
well instead of hierarchical per-CPU counters [2].

Thanks,
Joanne

[1] https://lore.kernel.org/linux-fsdevel/CAJnrk1YO5E6ETxn9RmM-LWRQSRxwF9iEakAPz6veMPjaKMAEOQ@xxxxxxxxxxxxxx/
[2] https://lore.kernel.org/linux-fsdevel/CAJnrk1b5asARJ_g176DsoNCvf+iFqzPbOtVveeuPxTPZDmnuKw@xxxxxxxxxxxxxx/