[PATCH 2/2] sched/numa: scan read-only file mappings in tiering mode

[Gregory Price]

task_numa_work() refuses to scan any read-only file-backed mapping:

  if (vma->vm_file &&
      (vma->vm_flags & (VM_READ|VM_WRITE)) == (VM_READ)) {
          trace_sched_skip_vma_numa(mm, vma, NUMAB_SKIP_SHARED_RO);
          continue;
  }

commit 4591ce4f2d22 ("sched/numa: Do not trap hinting faults for shared
libraries") added this because such pages are expected to be cache
replicated and bounce between accessor nodes, so:

  If we are never going to migrate the page, it is overhead for no gain

Memory tiering broke that premise.  A slow tier folio is promoted on
hint fault based on hotness, so these can migrate now - they just never
get the chance, because the VMA is never scanned.

Observed on a 768G DRAM + 256G CXL host running a ~430G database service:

  91% of its main binary (169M of 185M) accumulated on CXL.

After this patch the binary's tier residency tracked runtime load.

Scan these VMAs with the slow_only restriction. Only non-toptier folios
are made hint-faultable, so the scan never marks a toptier folio in a
read-only file mapping in any mode.  A page already on the fast tier costs
nothing, and a page on the slow tier costs one hint fault, which is what
buys the promotion.

That fault is not free of placement effect: a successful promotion reports
its toptier destination node to task_numa_fault(), so it lands in
numa_faults[] like any other promotion.  Only the set of VMAs that can
produce one changes.

Fixes: c574bbe91703 ("NUMA balancing: optimize page placement for memory tiering system")
Signed-off-by: Gregory Price (Meta) <gourry@xxxxxxxxxx>
Assisted-by: Claude:claude-opus-5
---
 kernel/sched/fair.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 990af0e45d69..5c5c80b17649 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -4075,6 +4075,11 @@ static void reset_ptenuma_scan(struct task_struct *p)
 	p->mm->numa_scan_offset = 0;
 }
 
+static bool vma_is_ro_file(struct vm_area_struct *vma)
+{
+	return vma->vm_file && (vma->vm_flags & (VM_READ | VM_WRITE)) == VM_READ;
+}
+
 static bool vma_is_accessed(struct mm_struct *mm, struct vm_area_struct *vma)
 {
 	unsigned long pids;
@@ -4211,6 +4216,8 @@ static void task_numa_work(struct callback_head *work)
 	}
 
 	for (; vma; vma = vma_next(&vmi)) {
+		bool ro_file;
+
 		if (!vma_migratable(vma) || !vma_policy_mof(vma) ||
 			is_vm_hugetlb_page(vma) || (vma->vm_flags & VM_MIXEDMAP)) {
 			trace_sched_skip_vma_numa(mm, vma, NUMAB_SKIP_UNSUITABLE);
@@ -4222,9 +4229,16 @@ static void task_numa_work(struct callback_head *work)
 		 * migrated as it is expected they are cache replicated. Avoid
 		 * hinting faults in read-only file-backed mappings or the vDSO
 		 * as migrating the pages will be of marginal benefit.
+		 *
+		 * Tiering still wants them.  A read-only mapping strands just
+		 * as much on the slow tier as any other, so scan it for slow
+		 * tier folios alone.  Nothing on the top tier is marked here,
+		 * in any mode.
 		 */
+		ro_file = vma_is_ro_file(vma);
 		if (!vma->vm_mm ||
-		    (vma->vm_file && (vma->vm_flags & (VM_READ|VM_WRITE)) == (VM_READ))) {
+		    (ro_file && !(sysctl_numa_balancing_mode &
+				  NUMA_BALANCING_MEMORY_TIERING))) {
 			trace_sched_skip_vma_numa(mm, vma, NUMAB_SKIP_SHARED_RO);
 			continue;
 		}
@@ -4302,7 +4316,7 @@ static void task_numa_work(struct callback_head *work)
 		 * VMA can cause others to become "permanently unaccessed" if a scan
 		 * cycle is consumed entirely by the large VMA.
 		 */
-		vma->numab_state->slow_only = false;
+		vma->numab_state->slow_only = ro_file;
 		if (!vma_pids_forced && !vma_is_accessed(mm, vma)) {
 			if (!(sysctl_numa_balancing_mode &
 			      NUMA_BALANCING_MEMORY_TIERING)) {
-- 
2.55.0