Re: [PATCH v4 0/8] Batched zswap_store() framework with compression batching

From: Kanchana P. Sridhar

Date: Fri Nov 20 2026 - 20:05:13 EST


On Fri, Nov 20, 2026 at 3:16 PM Nhat Pham <nphamcs@xxxxxxxxx> wrote:
>
> On Thu, Nov 19, 2026 at 8:19 PM Kanchana P. Sridhar
> <kanchanapsridhar2026@xxxxxxxxx> wrote:
> >
> >
> > v4: Batched zswap_store() framework with compression batching
> > =============================================================
> >
> > This series is a spin-off of "v14: zswap compression batching with optimized
> > iaa_crypto driver" [1], and focuses on the generic crypto_acomp and zswap
> > compression batching enablers. Resuming work on this series following a gap
> > since v14 (January 2026). Please note that my email address and affiliation have
> > changed from kanchana.p.sridhar@xxxxxxxxx to kanchanapsridhar2026@xxxxxxxxx. I
> > have retained the original Intel sign-offs on prior patches for historical
> > attribution while signing off v4 of this series with my current address.
> >
> > The main focus of v4 was to establish a performance measurement methodology to
> > quantify the benefits to high level metrics by using zswap compression
> > batching. This was done using an Intel Sapphire Rapids bare-metal instance with
> > 48 physical cores and by running "usemem30", i.e., 30 concurrent usemem
> > processes, each of which allocates 10G, in a 150G memory.high contrained cgroup,
> > for 10 iterations. The environment was set up to minimize performance jitter so
> > we can understand the system-level/workload-level advantages of zswap
> > compression batching when running workloads at scale.
> >
> > To summarize the main contribution of this patch series: Batching is introduced
> > in zswap_store() of large folios: we carve out a large folio into batches of
> > ZSWAP_MAX_BATCH_SIZE (4 pages) for software compressors, and store each
> > batch. Moving the fundamental operations that need to be happen to store a page
> > to now work at the granularity of a batch offers opportunities to optimize them
> > from these perspectives:
> >
> > 1) Utilize existing kernel API that minimize obtaining locks for bulk
> > operations, e.g. kmem_cache_alloc_bulk()/kmem_cache_free_bulk().
> >
> > 2) We keep the working set for each of the core store functionalities
> > (alloc/initialize zswap entries, compress pages, store pages in zsmalloc,
> > pool/objcg reference increments, objcg charging) to be focused on a
> > batch, thereby minimizing cache evictions and benefiting from cache
> > locality.
> >
> > 3) Along the lines of (1), the main zswap_compress() batching routine obtains
> > the comp_ctx mutex lock once per batch, and releases it after it has
> > processed all pages in that batch.
> >
> > 4) Any zswap store functionality that works on a batch can potentially be
> > optimized via bulk-processing/parallelized as a next step, to further improve
> > performance.
> >
> > Performance testing: usemem30 with 100% over-commit:
> > =====================================================
> > Validation was done with mm-unstable as of 11-17-2026 (Baseline: commit
> > 866f761fd414) and with this patch series.
> >
> > v4 was validated on an Intel Xeon Sapphire Rapids bare-metal instance with
> > 48 cores.
> >
> > Bare-metal instance specs:
> > --------------------------
> > - 48 physical cores
> > - 190G RAM
> > - 200G SWAP disk partition
> >
> > Workload: 30 concurrent usemem processes, each allocates 10G:
> > -------------------------------------------------------------
> > The vm-scalability "usemem" test (unmodified version) was run in a
> > cgroup with memory.high fixed at 150G. There is no swap limit set for the
> > cgroup.
> >
> > To quantify the performance benefit of zswap compress batching at scale,
> > 30 usemem processes were run ("usemem30"), each allocating and writing 10G of
> > memory, and sleeping for 10 sec before exiting.
> >
> > usemem -w -O -b 5 -s 10 -n 30 10g
> > echo 0 > /sys/module/zswap/parameters/shrinker_enabled
> >
> > Performance validation methodology:
> > -----------------------------------
> > These settings were used to minimize variation:
> >
> > - All cores were fixed to run at 2.4GHz with "performance" scaling governor.
> > - Turbo boost was disabled.
> > - CPU power-saving C-states restricted with:
> >
> > GRUB_CMDLINE_LINUX_DEFAULT="intel_idle.max_cstate=1 processor.max_cstate=1"
> >
> > The usemem30 experiments were run with the baseline mm-unstable as of
> > 11-17-2026 (commit 866f761fd414) and with this patch-series.
> >
> > After booting into each kernel:
> >
> > 1) Constrain the cgroup's memory.high to 150G.
> >
> > 2) Run 2 warm-up iterations of usemem30. Don't use this data for benchmarking
> > since sys time typically skews higher for initial runs.
> >
> > 3) Next, run 10 benchmarking iterations of usemem30:
> > - Log these high-level metrics: Sys time and individual usemem throughput.
> >
> > 4) Sys time and Throughput metrics were gathered for three mTHP settings:
> >
> > - 2048kB
> > - 1024kB
> > - 64kB
> >
> > 5) Benchmark summary data includes Sys time and Throughput components from the
> > 10 iterations of usemem30:
> >
> > - Total Samples
> > - Mean Value
> > - Std Deviation
> > - CoV (Variance): Coefficient of variation (relative standard deviation)
> > measures spread relative to the mean rather than statistical variance.
> >
> > An LLM was used to help generate the benchmarking and data processing scripts.
> >
> > usemem30 Benchmarking Summary:
> > ==============================
> >
> > ---------------------------------------------------
> > Improvement with v4 zswap compress batching
> > ---------------------------------------------------
> > mTHP enabled: 2048kB 1024kB 64kB
> > ---------------------------------------------------
> > Sys time: -13.87% -2.64% -1.72%
> > Throughput: +3.85% +0.42% +1.31%
> > ---------------------------------------------------
>
> Have you tried perf-ing the store path to see what was the overhead
> source (before and after this optimization)? I'm curious.
>
> Mostly LGTM - still taking a look...

Thanks Nhat, for reviewing the code and providing valuable feedback!
Thanks also for your time.

Forgot to mention: ZSTD is the zswap compressor used. I will also
reply to your comment in the other email, along with more setup
details.
All zswap parameters (max pool percent, shrinker) are defaults.

I did analyze the performance of the store path using bpftrace to
understand what was the overhead source before and after this
optimization. My experiments used PMD folios to maximize the zswap
store/compress batching samples so the traces are helpful in
understanding how the optimizations are eliminating overhead.
Although, this was done with v3 and my earlier, memory constrained
QEMU setup running on Chromebook/Crostini.

These are the bpftrace p50/p95/p99 latencies. The labels mean:

"zswap_store_higher_order_folio":
zswap_store() of PMD folios.

"zswap_store_order0_folio":
zswap_store() of order-0 folios.

"zswap_batch_compress_per_page":
zswap_compress() per-page latency of store batches generated from PMD
folios.

"zswap_single_compress_per_page":
zswap_compress() per-page latency of order-0 folios.

=============================================================================
usemem PMD THP BPFTRACE
=============================================================================
Baseline zswap zswap Change Change
mm-unstable store compress with with
10-25-2026 batching batching compress compress
(patch 6-7) (patch 8) batching batching
(ns) %
=============================================================================
zswap_store_order0_folio (ns):
=============================================================================
p50 39,631 37,538 37,765 -1,866 -4.71%
p95 53,088 49,690 49,349 -3,739 -7.04%
p99 65,065 63,529 57,404 -7,661 -11.77%
=============================================================================
zswap_store_higher_order_folio (ns):
=============================================================================
p50 21,460,366 21,201,411 20,383,233 -1,077,133 -5.02%
p95 25,975,341 27,558,253 25,045,350 -929,991 -3.58%
p99 29,538,493 61,926,648 27,184,835 -2,353,658 -7.97%
=============================================================================
zswap_single_compress_per_page (ns):
=============================================================================
p50 37,545 39,320 37,218 -327 -0.87%
p95 51,549 66,503 51,481 -68 -0.13%
p99 89,866 134,854 87,091 -2,775 -3.09%
=============================================================================
zswap_batch_compress_per_page (ns):
=============================================================================
p50 40,493 2,948 7.85%
p95 56,944 5,395 10.47%
p99 76,242 -13,624 -15.16%
=============================================================================

Analysis:
=========

The batching of zswap_store() phases, and compress batching
optimizations benefit zswap_store() and zswap_compress_per_page()
median and tail latencies
for PMD folios and base 4K folios. The benefits are attributable
firstly to the store batching framework added in patches 6-7, that is
fully utilized to improve
zswap_compress() latency by giving it a batch of pages to be compressed.

Gains seen from store batching alone (patches 6-7) are due to:

1) Bulk allocating the zswap entries for the batch,
2) Minimizing computes by getting the wb_enabled once per PMD folio,
rather than per-page,
3) Fewer function calls: zswap_store_pages() called once every 4
pages, rather than per-page.

Furthermore, the batch compression added in patch 8 improves latency
significantly, even more than that obtained by store batching alone,
for these reasons:

The overall approach for making the zswap_store() code more efficient, has been:

a) Minimize computes, especially conditionals. For e.g. crypto API
that set things up for a batch, rather than for a page.
b) The working set is kept focused on batch-centric data pertaining to
a handful of operations on the batch, to minimize cache evictions and
benefit from cache locality.
c) Each block of compute in zswap_compress() is doing checks/state
changes for a batch, which I believe simplifies the architecture and
allows us to build further optimizations.

Specifically:

1) Per-CPU sg_table with ZSWAP_MAX_BATCH_SIZE scatterlists that are
statically mapped to compression dst buffers for that CPU, thereby
minimizing local SG lists initialization, and using compact new
crypto_acomp API introduced in patches 1-4 that set up an acomp_req's
src/dst to be SG lists.

2) Patch 6 allocates ZSWAP_MAX_BATCH_SIZE destination buffers in the
per-CPU acomp_ctx, for an added memory overhead of ~12 KB per-CPU.
However, it allows us to not have to immediately call zs_store() with
the compressed buffer, but instead continue to compress the next page.
I haven't quantified this with perf event counters, but my guess is
this minimizes cache evictions because we are focused on running a
focused set of computes on a specific data set.

3) A one-shot zswap_update_entries() does all the zsmalloc handle
allocation and copying for the batch.

4) Bulk accounting for the batch instead of per-page.

5) Single point error-handling in zswap_store_pages() to avoid
conditionals, so as to not wash away batching gains.

6) By making zswap_compress() more efficient, we are able to lock the
comp mutex per batch of ZSWAP_MAX_BATCH_SIZE (4) pages instead of
per-page.

7) Optimize most-likely-to-succeed fast paths, move conditionals and
error processing to less-likely code (which is also consolidated to a
single error handling label).

8) Don't proceed to next stage of batch processing until this stage is
successful for its specific functionality scope. This minimizes
computes by avoiding if-then-else checks per-page.

The bpftraces highlight exceptions in p50/p95 batch compress latencies
(normalized per-page), even though the p99 batch compress latency
improves by 15% over mm-unstable. Batch compress latency can be
optimized further in subsequent patches. The batching framework
established in this series allows us to make locking and
bulk-processing improvements in each stage of zswap_store().

Please feel free to share any additional insights you may have as you
review the code!

Thanks,
Kanchana