[PATCH v3 0/3] mm: zswap: free cold writeback folios promptly

[Alexandre Ghiti]

When zswap writes an entry back, it allocates an order-0 swap cache folio,
decompresses into it, and issues the write. The folio is cold by
construction, yet today it is left on the LRU for page reclaim to find and
free later. That wastes a reclaim scan and keeps cold memory resident
longer than necessary.

Rather than implement this in zswap, extend the existing dropbehind
mechanism to swap cache folios and have zswap opt into it (Yosry). A
PG_dropbehind folio is already dropped from its cache once writeback
completes instead of being left for reclaim; for a swap cache folio that
"drop" is removing it from the swap cache.

  Patch 1 - move LRU insertion out of the swap cache allocator into its
            callers, so a folio that will be dropped on writeback completion
            (zswap writeback) can be allocated off the LRU without a flag.

  Patch 2 - drop dropbehind swap cache folios on writeback completion. The
            drop needs the folio and swap cluster locks and may sleep, but
            writeback can complete in interrupt context, so it is deferred
            to a worker via a per-CPU llist, which frees them in batches.

  Patch 3 - zswap allocates its writeback folio off the LRU and marks it
            dropbehind, opting into the mechanism above.

v1: https://lore.kernel.org/linux-mm/20260718093723.153324-1-alex@xxxxxxxx/
v2: https://lore.kernel.org/linux-mm/20260727143618.1582318-1-alex@xxxxxxxx/

Changes in v3:
- Drop the synchronous-IO special case in zswap writeback: those folios are
  now marked dropbehind like any other and dropped from folio_end_writeback()
  instead of inline. It was only an optimisation for zswap on a sync backend,
  which is not worth the extra code (Yosry, Nhat). This also removes
  swap_dropbehind_free_one_folio(), which had no other caller.
- Queue the deferred drop on a dedicated WQ_MEM_RECLAIM workqueue instead of
  system_wq: the work item runs on the reclaim path and the folios it frees
  stay off the LRU until it does, so it needs a rescuer to guarantee forward
  progress under memory pressure.
- Use mem_cgroup_tryget()/mem_cgroup_put() rather than css_tryget()/css_put():
  struct mem_cgroup is only defined under CONFIG_MEMCG, so the latter failed
  to build with CONFIG_MEMCG=n.
- Use raw_cpu_ptr() for the per-CPU llist: now that synchronous-IO swap also
  marks its folios dropbehind, writeback can complete in task context and the
  enqueue can run preemptible.

Changes in v2:
- Make swap dropbehind a generic core-mm mechanism that zswap opts into,
  rather than a zswap-specific implementation (Yosry).
- Allocate off the LRU by moving folio_add_lru() out of the swap cache
  allocator into its callers, instead of a skip_lru flag; rename it to
  __swap_cache_alloc_folio() (Kairui).
- Batch the freed folios in the deferred worker (folios_put) instead of a
  put per folio (Nhat).
- Skip the folio in the free path if it is still under writeback
  (defensive) (Nhat).

Results
-------
Paired baseline vs series on async swap (NVMe, Patch 2 deferred path). Each
workload runs confined to a memory cgroup (memory.max) small enough to force
zswap shrinker writeback.

Kernel build (defconfig, make -j4; memory.max = 600M):

  metric            baseline       series      delta
  pgrotated           441028         2521     -99.4%
  pgsteal_direct     3524343      2869004     -18.6%
  pgscan_direct      8393791      7765008      -7.5%
  zswpwb              705129       699019      -0.9%
  build time (s)        1155         1114      -3.6%

Of the 699019 folios written back, 698990 (99.996%) were freed promptly on
writeback completion; only 28 fell back to reclaim.

MySQL/OLTP (sysbench, 10 tables x 1M rows, 512M buffer pool, 8 threads, 300s;
memory.max = 256M):

  metric            baseline       series      delta
  transactions/s      153.87       163.30      +6.1%
  p95 latency (ms)    157.42       145.82      -7.4%
  avg latency (ms)     52.09        49.05      -5.9%
  pgrotated           743738        22460     -97.0%
  pgsteal_direct     6886490      5445278     -20.9%
  pgscan_direct     13462510     10820730     -19.6%

Future work
-----------
Barry suggested extending this to MADV_PAGEOUT and general reclaim, since on
asynchronous swap devices the memory is not freed until a later reclaim scan
even after the folio has been written out. I prototyped dropbehind for all
reclaimed swap folios (not included here) and it regressed sysbench OLTP
throughput by ~15% on NVMe swap. This matches Kairui's point that for common
reclaim the swap cache is worth keeping to catch fast refaults, unlike
synchronous-IO or zswap writeback where the reclaimer already waits.

So this series stays scoped to zswap cold writeback. I still need time to work
out whether general reclaim / MADV_PAGEOUT dropbehind is worth pursuing for
everything or only for known-cold cases, and how; I will follow up.

Alexandre Ghiti (3):
  mm: swap: move LRU insertion out of the swap cache allocator
  mm: swap: drop dropbehind swap cache folios on writeback completion
  mm: zswap: drop cold writeback folios via swap dropbehind

 fs/splice.c          |   2 +-
 include/linux/swap.h |   6 ++-
 mm/filemap.c         |  19 ++++++++
 mm/swap.h            |   6 +--
 mm/swap_state.c      | 114 ++++++++++++++++++++++++++++++++++++++++---
 mm/truncate.c        |   2 +-
 mm/vmscan.c          |  13 +++--
 mm/zswap.c           |   7 ++-
 8 files changed, 148 insertions(+), 21 deletions(-)


base-commit: 626acb37cd445144f321f1b64cac9a93760fa716
-- 
2.53.0-Meta