[PATCH v4 0/7] sched/numa: stop VMA scan filters from gating promotion

[Gregory Price]

NUMA balancing uses hinting faults for both task placement and
memory-tier promotion. Several filters designed to avoid unproductive
socket-placement faults also prevent promotion.

In a production deployment using kernel.numa_balancing=2 on a host
with 768 GB of DRAM and 256 GB of CXL memory running two ~430 GB
database workloads with large shmem VMAs - we discovered poor NUMA
balancing behavior:

Before patch series:
  - 150-200 GB/s DRAM bandwidth
  - 40-45 GB/s CXL bandwidth, saturating the device
  - request latency above 5 ms with longer tails

After patch series:
  - 250+ GB/s sustained DRAM bandwidth
  - approximately 5-10 GB/s sustained CXL bandwidth
  - request latency between 800 us and 2 ms

The core issue is that the global balancing mode says which mechanisms
are enabled, but it cannot describe the intent of an individual
VMA/PTE walk (placement vs promotion).

This plumbs the scan reasoning into the prot_none injection while
retaining the placement-scan filtering.

Patches 1-5 form the backportable fix set:

  1. Add promotion-only NUMA protection walks and derive private VMA
     state in the folio eligibility check.
  2. Permit eligible shared folios to be promoted to a fast tier and
     rename the promotion predicate to folio_numab_promotable().
  3. Scan mappings covered by the legacy file placement filter using
     promotion-only scans.
  4. Separate VMA placement eligibility from partial-scan continuation.
  5. Scan PID-inactive VMAs for promotion and account for placement scans
     separately.

Patches 4 and 5 are technically part of the same fix, but are separated
to make it easier to review - they must be backported together.

Patches 6-7 are independently requested cleanups.

  6. Use BIT() for the change_protection() flags.
  7. Use the VMA flag API in the touched NUMA-balancing code.

Changes in v4:
  - 1/7: commit message fix (David).
         Hoist balancing mode check into variables.
  - 2/7: Rename folio_in_lowtier() to folio_numab_promotable()
         Typo fix.
  - 3/7: Compute placement_scan once per VMA.
         Avoid complex boolean logic (David)
         Fix indentation (David).
  - 4/7: Avoid complex boolean logic (David)
         Store the in-progress state as numab_state->promo_only.
  - 5/7: Avoid complex boolean logic (David).
         Fix "sticky bit" issue where resumed scans would cause
         valid VMAs to drop out of the scanning-eligible set.

v3: https://lore.kernel.org/r/20260922182928.2199090-1-gourry@xxxxxxxxxx

Gregory Price (Meta) (7):
  mm: support promotion-only NUMA hinting scans
  mm: allow shared folios to be promoted to a fast tier
  sched/numa: scan read-only file mappings in tiering mode
  sched/numa: separate VMA placement from scan continuation
  sched/numa: scan PID-inactive VMAs for promotion
  mm: use BIT() for change_protection() flags
  mm: use VMA flag helpers in NUMA balancing

 include/linux/mm.h       |  23 +++++----
 include/linux/mm_types.h |  13 +++++
 kernel/sched/fair.c      | 100 ++++++++++++++++++++++++++++-----------
 mm/huge_memory.c         |   3 +-
 mm/internal.h            |   6 +--
 mm/memory-tiers.c        |   9 ++--
 mm/memory.c              |   2 +-
 mm/mempolicy.c           |  37 ++++++++-------
 mm/migrate.c             |  13 +++--
 mm/mprotect.c            |   7 +--
 10 files changed, 140 insertions(+), 73 deletions(-)

-- 
2.55.0