Re: [RFC PATCH 0/3] mm/vmscan: adaptive multi-threaded kswapd for NUMA-aware reclaim

[Zi Yan]

On Tue Sep 8, 2026 at 6:10 PM EDT, Ehab Ababneh wrote:
> This series revives Buddy Lumpkin's earlier multi-kswapd proposal:
> https://lkml.iu.edu/hypermail/linux/kernel/1804.0/00342.html
+others

Please Cc everyone in the cover letter, so people can know the
motivation of the patchset.

>
> The motivation is stronger now than when the patch was first discussed.
> Many current systems have hundreds of cores per NUMA node, not the
> single-digit or low-tens core counts that were more common at the time.
> When reclaim does not keep up, direct reclaim can still push allocation
> latency into application paths and leave substantial CPU capacity waiting
> for memory to be freed.
>
> This patchset adds adaptive multi-threaded kswapd. The wakeup policy uses
> node load to decide how many kswapd workers to run, so reclaim can scale
> when it helps and stay conservative on already busy nodes.
>
> Series summary:
>
> 1. Allow multiple kswapd threads per node and add control plumbing.
> 2. Wake an appropriate number of kswapd threads from per-node
>    runnable load.
>
> Concerns from the original discussion and how this series addresses some
> of them:
>
> - Concern: Direct reclaim is intended to slow a memory-hogging thread.
>   Response: That can be acceptable on lower-core systems. On high-core
>   systems, idling many cores while reclaim catches up can cost more than
>   allowing reclaim parallelism to scale. It can also block higher-priority
>   tasks in direct reclaim while they perform reclaim work on behalf of
>   lower-priority memory-hogging tasks.
>
> - Concern: More kswapd threads may hide deeper reclaim issues.
>   Response: This series is additive to ongoing reclaim improvements. In
>   our testing, multi-threaded kswapd was able to improve performance on
>   top of what multi-gen LRU already provides.
>
> - Concern: Existing knobs (such as swappiness and watermarks) should be
>   preferred.
>   Response: In our testing, those knobs alone did not reliably hit
>   performance targets and could increase CPU cost for the same workload
>   objective.
>
> - Concern: Need evidence from real workloads.
>   Response: This cover letter includes Cassandra results showing higher
>   throughput and lower response latency.
>
> - Concern: More reclaim threads may increase pressure on well-behaved
>   tasks.
>   Response: Adaptive wakeup addresses this by choosing thread count from
>   node load.
>
> - Concern: Additional configuration can increase operational complexity.
>   Response: The user-facing interface is intentionally minimal:
>   max_kswapds_per_node.
>
> - Concern: Lock contention may serialize workers.
>   Response: The Cassandra runs below still show net gains, indicating
>   contention did not erase the benefit for this workload. The wakeup path
>   now uses wake_up_nr() against the existing kswapd_wait queue, avoiding
>   pgdat->kswapd_lock (a sleeping mutex) entirely on the allocation hot
>   path.
>
> Real-world workload results (Cassandra):
>
> Tests were performed on 7.0.0-rc1.
>
> - max_kswapds_per_node=1
>   - throughput sample: 171146
>   - reference latency value: 6.375
>   - op rates: 43256, 42731, 42341, 42818 ops/s
>   - p99 latency: 6.3, 6.4, 6.4, 6.4 ms
>
> - max_kswapds_per_node=8
>   - throughput sample: 183639
>   - reference latency value: 6.0
>   - op rates: 45791, 45253, 46534, 46061 ops/s
>   - p99 latency: 6.0, 6.1, 5.9, 6.0 ms
>
> Observed improvement in these runs was about +7.3% throughput and about
> -5.9% response latency, which shows practical benefit for production-style
> database workloads.

But the result suggested we should not have multi-threaded kswapd,
because 7 extra threads give only 7.3% throughput. That is not a good
use of CPU cores. I suspect these threads are competing for LRU locks.
We might want to partition each LRU list into N first, before N kswapds
can work truely in parallel.

>
> In our runs, performance numbers were essentially unchanged with and
> without the adaptive multi-threaded kswapd wakeup policy. In both cases,
> they outperformed the single-kswapd-thread baseline. This indicates the
> adaptive method preserved the multi-threaded performance improvement.
>
> Addendum: alternative approaches evaluated
>
> - PSI per NUMA node.
>   I prototyped PSI-based node pressure ranges to drive wakeup count.
>   This became cumbersome because robust PSI-to-thread mappings were not
>   straightforward across workload types.
>
> - CPU mask snapshot policy.
>   I also tested a simple CPU mask snapshot approach.
>   While functional, it reflects a moment-in-time view and does not capture
>   pressure trends over a broader sampling window.
>
> Buddy Lumpkin (1):
>   vmscan: Support multiple kswapd threads per node
>
> Ehab Ababneh (2):
>   mm/vmscan: handle racing max_seq advancement
>   mm/vmscan: make kswapd wakeups NUMA load-aware
>
>  include/linux/mmzone.h        |   5 +-
>  include/trace/events/vmscan.h |  28 +++
>  mm/compaction.c               |   8 +-
>  mm/internal.h                 |   3 +
>  mm/page_alloc.c               |  26 +++
>  mm/vmscan.c                   | 419 +++++++++++++++++++++++++++++++++++++++---
>  6 files changed, 465 insertions(+), 24 deletions(-)




-- 
Best Regards,
Yan, Zi