[PATCH v2 0/1] mm, swap: only prefer nonfull clusters on synchronous devices

[Alexandre Ghiti]

7.3 regressed swap I/O: for the same swap volume, a workload that
rewrites swapped out memory sends many more requests to the device.
mysql/sysbench in a 256M memcg swapping to an NVMe partition, 7.3
compared to the previous version:

  device read requests per transaction        +59%
  device write requests per transaction       +27%
  throughput, iocost off                      no change
  throughput, iocost on                       -19%

Two commits interact:

- Commit 9a42aed48421 ("mm, swap: prefer nonfull over free clusters"),
  in 6.18, makes allocations fill the nonfull clusters of their order
  before taking a free one.

- Commit 3cedb745d354 ("mm: clarify the folio_free_swap() for
  do_swap_page()"), in 7.3, frees the swap slot on an exclusive write
  fault. 7.0 to 7.2 kept the slot, but only as a side effect of commit
  4b34f1d82c65 ("mm, swap: free the swap cache after folio is
  mapped").

The freed slots leave holes in full clusters, which then go back on
the nonfull list. Reclaim allocates slots one folio at a time and
merges the writes afterwards, so a batch of LRU-adjacent folios gets
scattered over those holes and is written with many small requests.
Readahead later finds those pages scattered as well, and needs more
read requests to bring them back.

The RFC [1] listed three fixes, which perform similarly:

A. Revert 3cedb745d354. This keeps the slot on write fault, but undoes
   the intent of that commit and keeps swap space pinned under dirty
   pages.

B. Below 50% swap usage, make order-0 allocations take a free cluster
   before a nonfull one. This is what the allocator already does, at
   any usage, on swap devices with discard enabled (swapon
   --discard=pages). The threshold is arbitrary and it stops working
   as swap fills up.

C. The RFC patch. Reclaim tells the allocator how many pages of each
   small order the batch is about to swap out, and those allocations
   then follow contiguous free slots.

In the RFC thread, Kairui suggested another option, which he prefers
[2]: keep nonfull-first only on synchronous devices like zram, which
don't merge I/O and where filling holes first keeps the free clusters
for large folios, and go back to preferring free clusters on every
other device, as discard devices already do. This is what v2 does. It
is a few lines, needs no information from reclaim, and performs as
well as C.

Same workload as above, each fix relative to its base kernel:

                                              RFC (C)      v2
  device read requests per transaction        -12%         -14%
  device write requests per transaction       -36%         -48%
  throughput, iocost off                      no change    no change
  throughput, iocost on                       +41%         +33%

Changes since RFC:
- Replace the batch-aware slot allocation with keeping nonfull-first
  for synchronous devices only, as suggested by Kairui.

[1] https://lore.kernel.org/linux-mm/20261022145615.2184343-1-alex@xxxxxxxx/
[2] https://lore.kernel.org/linux-mm/CAMgjq7BFbXEKEt9faj_rg5AStfLkpJ0j57EQOOJ+L3-w2xRvtg@xxxxxxxxxxxxxx/

Alexandre Ghiti (1):
  mm, swap: only prefer nonfull clusters on synchronous devices

 mm/swapfile.c | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

base-commit: afb4eea28037fa55c4560313ecbd8a1e1da8369a