[RFC PATCH v4 3/3] mm/swap: allow swap faults to retry under the VMA lock

[Hongru Zhang]

From: Hongru Zhang <zhanghongru@xxxxxxxxxx>

do_swap_page() currently falls back to mmap_lock for swap-in retries.
When VMA-lock retry is available, keep the retry under the per-VMA lock
instead. This helps swap-in handling stay efficient when mmap_lock is
heavily contended.

Using a dedicated benchmark [1], we tested this on a 20-core Intel
i7-12700 desktop with a 2GB swapfile. The benchmark uses one pressure
thread under memcg limits to keep a 128MB non-zero anonymous mapping
under swap pressure, 12 reader threads to fault it back in, and optional
mmap writer threads to amplify mmap_lock read-write contention. Each
test ran for 60 seconds and reports completed reader rounds per second
under swap pressure.

Throughput at the same mmap writer pressure (higher is better):
+--------------+-------------+-------------+-------------+
| mmap writers |   Vanilla   |   Patched   | Improvement |
+--------------+-------------+-------------+-------------+
|      0       | 17303.09 /s | 18394.51 /s |    +6.3%    |
+--------------+-------------+-------------+-------------+
|      2       | 16728.04 /s | 18591.68 /s |   +11.1%    |
+--------------+-------------+-------------+-------------+
|      4       | 12596.23 /s | 18534.00 /s |   +47.1%    |
+--------------+-------------+-------------+-------------+

Throughput under increasing mmap writer pressure (higher is better):
+---------+----------------+---------------------+----------------------+
|  Kernel | 0 mmap writers |    2 mmap writers   |    4 mmap writers    |
+---------+----------------+---------------------+----------------------+
| Vanilla |   17303.09 /s  | 16728.04 /s (-3.3%) | 12596.23 /s (-27.2%) |
+---------+----------------+---------------------+----------------------+
| Patched |   18394.51 /s  | 18591.68 /s (+1.1%) | 18534.00 /s  (+0.8%)  |
+---------+----------------+---------------------+----------------------+

As mmap writer pressure increased, the baseline throughput dropped sharply
while the patched kernel remained stable.

[1] https://gist.github.com/zhr250/991d0bed1d1c25ab72a4361517e93d6b

Signed-off-by: Hongru Zhang <zhanghongru@xxxxxxxxxx>
Suggested-by: Barry Song <baohua@xxxxxxxxxx>
Suggested-by: Suren Baghdasaryan <surenb@xxxxxxxxxx>
---
 mm/memory.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/mm/memory.c b/mm/memory.c
index 428eb555ecb7..5e21d11b6086 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -4988,8 +4988,10 @@ vm_fault_t do_swap_page(struct vm_fault *vmf)
 
 	swapcache = folio;
 	ret |= folio_lock_or_retry(folio, vmf);
-	if (ret & VM_FAULT_RETRY)
+	if (ret & VM_FAULT_RETRY) {
+		ret |= VM_FAULT_MAY_USE_VMA_LOCK;
 		goto out_release;
+	}
 
 	page = folio_file_page(folio, swp_offset(entry));
 	/*
-- 
2.43.0