Re: [PATCH v3 2/2] mm/zswap: Support batch writeback in shrink_memcg()

[Johannes Weiner]

On Fri, Jul 31, 2026 at 03:19:00PM +0800, Hao Jia wrote:
> From: Hao Jia <jiahao1@xxxxxxxxxxx>
> 
> Currently, shrink_memcg() writes back at most one entry per-node during
> its traversal. This makes shrink_worker() inefficient, as it must
> repeatedly re-enter shrink_memcg() to make any substantial progress.
> Under high memory pressure, this can cause the writeback speed to be
> too slow to keep up with refaults, leading to zswap store failures and
> forcing pages to skip zswap and go directly to disk, which results in
> an LRU inversion.
> 
> To address this, extend shrink_memcg() and rewrite its LRU iteration logic,
> enabling batch writeback for both the shrink_worker() and zswap_store() paths.
> To prevent shrink unfairness across NUMA nodes caused by a shared global scan
> quota, limit scanning to up to SWAP_CLUSTER_MAX pages per node and write back
> any reclaimable entries found.
> 
> Test Setup:
> - Total memory: 32 GB, 1 NUMA node.
> - zswap settings: accept_threshold_percent=50, shrinker_enabled=N.
> 
> Test Case 1:
> Set max_pool_percent=1, allocate 512MB of anonymous pages, and fill them
> with random data (to avoid compression). Then, use cgroup memory.reclaim
> to force a large amount of anonymous pages into zswap. At an interval of
> 2ms, allocate a 4K anonymous page where the first 4 bytes are random numbers
> and the rest are zeros, and then trigger reclamation of this 4K page through
> cgroup memory.reclaim. When the pool threshold is reached, shrink_memcg()
> will be triggered.
> The test data after running for 120s is as follows:
>                                 Baseline      Patched
> shrink_worker wakeups              5,363          169
> shrink_memcg calls            11,373,201      350,703
> written_back pages                40,212       40,241
> zswap_store calls                161,190      163,753
>    store succeeded (ret=1)       102,743      117,183
>    store rejected (ret=0)         58,447       46,570
>    store reject rate                ~36%        ~28%
> pool_limit_hit delta              55,826       33,760
> pswpout                           98,659       86,811
> pswpin                                 2            0
> 
> Test Case 2:
> We evaluated the following two sub-configurations using stress-ng inside
> a cgroup capped at memory.max=1G for 120 seconds:
>   Test Case 2a (max_pool_percent=1): Continuously triggers the global
>   zswap pool limit, thereby waking up shrink_worker() to perform asynchronous
>   shrinking.
>   Test Case 2b (zswap.max=320M, max_pool_percent=50): Continuously triggers
>   the cgroup's zswap.max limit, thereby invoking synchronous shrinking.
> Command executed for both setups:
>    bash -c 'echo $$ > /sys/fs/cgroup/zswaptest/cgroup.procs ; \
>    exec stress-ng --vm 4 --vm-bytes 4G --vm-keep --vm-method rand-set -t \
> 120s -q'
> 
> Test Case 2a (max_pool_percent=1):
>                                 Baseline       Patched
> shrink_worker wakeups              5,640         1,308
> shrink_memcg calls             8,481,500     3,140,972
> written_back pages                   260       468,216
> zswap_store calls              2,742,756     2,011,269
>    store succeeded (ret=1)       934,640       947,988
>    store rejected (ret=0)      1,808,116     1,063,281
>    store reject rate                ~66%          ~52%
> pool_limit_hit delta           1,181,310       196,882
> pswpout                        1,808,376     1,531,497
> pswpin                         4,288,497     3,635,365
> Test Case 2b (zswap.max=320M, max_pool_percent=50):
>                                 Baseline       Patched
> shrink_worker wakeups                 0              0
> shrink_memcg calls              687,608         54,002
> written_back pages              639,176        846,663
> zswap_store calls             1,224,222      1,228,548
>    store succeeded (ret=1)      992,816      1,208,123
>    store rejected (ret=0)       231,431         20,425
>    store reject rate               ~19%            ~2%
> pool_limit_hit delta                  0              0
> pswpout                         870,745        867,360
> pswpin                        1,707,823      1,216,814
> 
> Under identical workloads and runtimes, batched zswap shrinking
> exhibits a significant reduction in both shrink_worker() wakeups
> and shrink_memcg() calls. Furthermore, the sharp drop in both pswpin
> and zswap_store() rejections demonstrates that batching zswap shrink
> operations effectively mitigates zswap_store() failures caused by
> hitting the pool limit. This significantly prevents pages from bypassing
> zswap and falling back directly to disk, thereby reducing LRU inversion.
> 
> Suggested-by: Yosry Ahmed <yosry@xxxxxxxxxx>
> Acked-by: Yosry Ahmed <yosry@xxxxxxxxxx>
> Acked-by: Nhat Pham <nphamcs@xxxxxxxxx>
> Signed-off-by: Hao Jia <jiahao1@xxxxxxxxxxx>
> ---
>  mm/zswap.c | 30 ++++++++++++++++++++++++++++--
>  1 file changed, 28 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/zswap.c b/mm/zswap.c
> index 48fc7b575e24..d406c14925d8 100644
> --- a/mm/zswap.c
> +++ b/mm/zswap.c
> @@ -1275,6 +1275,21 @@ static struct shrinker *zswap_alloc_shrinker(void)
>  	return shrinker;
>  }
>  
> +/*
> + * Scan up to SWAP_CLUSTER_MAX pages on each per-node zswap LRU of @memcg
> + * and write back the reclaimable ones.
> + *
> + * Since the second-chance algorithm rotates referenced entries to the
> + * LRU tail, the per-node scan is capped at the current LRU length so
> + * each entry is scanned at most once per call. It is up to the caller
> + * to handle retries, deciding whether to scan another memcg to complete
> + * the full iteration, or to rescan the current memcg to drain its zswap
> + * entries.
> + *
> + * Return: 0 if at least one entry was written back, -EAGAIN if entries
> + * were scanned but none could be written back, or -ENOENT if @memcg has
> + * writeback disabled, is a zombie cgroup, or has empty zswap LRUs.
> + */
>  static int shrink_memcg(struct mem_cgroup *memcg)
>  {
>  	int nid, shrunk = 0, scanned = 0;
> @@ -1290,13 +1305,24 @@ static int shrink_memcg(struct mem_cgroup *memcg)
>  		return -ENOENT;
>  
>  	for_each_node_state(nid, N_NORMAL_MEMORY) {
> -		unsigned long nr_to_walk = 1;
> +		unsigned long nr_to_walk, node_budget;
> +
> +		/*
> +		 * Cap the scan at the per-node LRU length so each entry is
> +		 * scanned at most once per call.
> +		 */
> +		node_budget = min(SWAP_CLUSTER_MAX,
> +			     list_lru_count_one(&zswap_list_lru, nid, memcg));

AFAICS you can just do unsigned long nr_to_walk = SWAP_CLUSTER_MAX.

__list_lru_walk_one() does a list_for_each_safe() that will exit the
same way whether you hit !nr_to_walk or run out of items.

> +		if (!node_budget)
> +			continue;
>  
> +		nr_to_walk = node_budget;
>  		shrunk += list_lru_walk_one(&zswap_list_lru, nid, memcg,
>  					    &shrink_memcg_cb, NULL, &nr_to_walk);
> -		scanned += 1 - nr_to_walk;
> +		scanned += node_budget - nr_to_walk;

scanned += SWAP_CLUSTER_MAX - nr_to_walk;