[PATCH 0/4] mm/ksm: use per-VMA locking for find_mergeable_vma()

[xu.xin16]

From: Xu Xin (ZTE) <xu.xin@xxxxxxxxx>

KSM scans VM_MERGEABLE VMAs and currently protects each scan with
mmap_read_lock(). Per-VMA locking allows KSM to read-lock only the VMA
it is actually interested in, so that unrelated mmap()/munmap() activity
in the same mm no longer blocks ksmd.

This series is organized as follows:

  Patch 1 removes an unused 'vma' member from struct folio_walk.  It has
  never been used since its introduction and is pure cleanup.

  Patch 2 adds a 'walk_lock' member to struct folio_walk and extends
  folio_walk_start() to assert the required locking mode.  Existing
  callers are converted to pass PGWALK_RDLOCK, so there is no functional
  change.  This prepares folio_walk_start() for callers that hold a
  per-VMA read lock instead of mmap_read_lock(), which is needed by the
  Patch 4. No functional change.

  Patch 3 tranforms the boolean 'lock_vma' into the enum 'page_walk_lock'
  without any behavior changed, which is prepared for the Patch 4 to use
  per-VMA locking. No functional change.

  Patch 4 introduces find_mergeable_vma_locked(), which uses the
  universal per-VMA locking helper vma_start_read_unlocked() to look up
  and read-lock a VM_MERGEABLE VMA without taking mmap_read_lock().  All
  KSM call sites that previously used find_mergeable_vma() under
  mmap_read_lock() are converted to the new helper, and the locking in
  get_mergeable_page() is switched to PGWALK_VMA_RDLOCK_VERIFY so that
  folio_walk_start() can verify the per-VMA lock is held.

A microbenchmark was run to measure the time KSM takes to merge a
victim region under mmap_lock contention. Under interference from 4 churner
threads, the merge time of the per-VMA KSM-optimized kernel is
significantly reduced by 50%.

Xu Xin (4):
  mm/pagewalk: delete the unused member
  mm: make folio_walk_start()'s locking asserts scalable
  mm/ksm: make break_ksm() more scalable
  mm/ksm: add find_mergeable_vma_locked() to use per-VMA locking

 arch/s390/mm/fault.c     |  4 +-
 include/linux/pagewalk.h |  2 +-
 kernel/events/uprobes.c  |  4 +-
 mm/huge_memory.c         |  4 +-
 mm/ksm.c                 | 85 ++++++++++++++++++++++++----------------
 mm/migrate.c             |  8 +++-
 mm/pagewalk.c            | 10 ++++-
 mm/rmap.c                |  4 +-
 8 files changed, 79 insertions(+), 42 deletions(-)

-- 
2.25.1