[RFC PATCH v3 0/4] mm/damon: Introduce node_eligible_mem_bp and node_ineligible_mem_bp Quota Goal Metrics

[Ravi Jonnalagadda]

MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This series introduces two new DAMON quota goal metrics for controlling
memory migration in heterogeneous memory systems (e.g., DRAM and CXL
memory tiering) using physical address (PA) mode monitoring.

v2: https://lore.kernel.org/linux-mm/20260129215814.1618-1-ravis.opensrc@xxxxxxxxx/

Changes since v2:
=================

- Split single metric into two complementary metrics:
  * node_eligible_mem_bp: hot memory present ON the specified node
  * node_ineligible_mem_bp: hot memory NOT on the specified node.
  This enables both PUSH and PULL schemes to work together.

- Added PA-mode detection lag compensation cache (see dedicated section
  below for design details).

- Added fix for esz=0 quota bypass that allowed unlimited migration when
  goal was achieved.

- Added fix for goal_tuner sysfs setting being ignored due to
  damon_new_scheme() always defaulting to CONSIST.

- Rebased on SJ's damon/next branch which includes the TEMPORAL goal
  tuner required for these metrics.

Background and Motivation
=========================

In heterogeneous memory systems, controlling hot memory distribution
across NUMA nodes is essential for performance optimization. This series
enables system wide hot page distribution with target-state goals like
"maintain 30% of hot memory on CXL" using PA-mode DAMON schemes.

Two-Scheme Setup for Hot Page Distribution
==========================================

For maintaining 30% of hot memory on CXL (node 1):

    PUSH scheme (DRAM->CXL): migrate_hot from node 0 -> node 1
      goal: node_eligible_mem_bp, nid=1, target=3000
      Activates when node 1 has less than 30% hot memory

    PULL scheme (CXL->DRAM): migrate_hot from node 1 -> node 0
      goal: node_ineligible_mem_bp, nid=1, target=7000
      Activates when node 1 has more than 30% hot memory

Both schemes use the TEMPORAL goal tuner which sets esz to maximum when
under goal and zero when achieved. Together they converge to equilibrium
at the target distribution.

What These Metrics Do
=====================

node_eligible_mem_bp measures:
    effective_hot_bytes_on_node / total_hot_bytes * 10000

node_ineligible_mem_bp measures:
    (total_hot_bytes - effective_hot_bytes_on_node) / total_hot_bytes * 10000

The metrics are complementary: eligible_bp + ineligible_bp = 10000 bp.

PA-Mode Detection Lag and Cache Design
======================================

In PA-mode, when pages are migrated:
1. Source node detection drops immediately (pages are gone)
2. Target node detection increases slowly (new addresses need sampling)

This asymmetry causes temporary underestimation of hot memory on the
target node. Without compensation, the system keeps migrating even after
reaching the goal.

The cache addresses this by remembering how much was recently migrated.
When calculating effective hot memory:
- Source node: reduce detected amount by recent migrations out
- Target node: boost detected amount by recent migrations in

The cache uses a rolling window to track migrations over time, and
expires after a configurable timeout (default 10s) when no migration
activity occurs. It also detects when its baseline becomes stale due
to new hot memory appearing in the workload.

Dependencies
============

This series is based on SJ's damon/next branch which includes:

- mm/damon/core: introduce damos_quota_goal_tuner [1]
- mm/damon/core: set quota-score histogram with core filters [2]
- mm/damon: always respect min_nr_regions from the beginning [3]
- mm/damon/core: disallow non-power of two min_region_sz [4]

[1] https://lore.kernel.org/linux-mm/20260212062314.69961-1-sj@xxxxxxxxxx/
[2] https://lore.kernel.org/linux-mm/20260131194145.66286-1-sj@xxxxxxxxxx/
[3] https://lore.kernel.org/linux-mm/20260217000400.69056-1-sj@xxxxxxxxxx/
[4] https://lore.kernel.org/linux-mm/20260214214124.87689-1-sj@xxxxxxxxxx/

Patch Organization
==================

1. mm/damon/sysfs: set goal_tuner after scheme creation
    - Fixes goal_tuner initialization order in sysfs scheme creation

2. mm/damon: fix esz=0 quota bypass allowing unlimited migration
    - Ensures esz=0 stops migration rather than bypassing quota entirely

3. mm/damon: add node_eligible_mem_bp and node_ineligible_mem_bp goal metrics
    - Adds the two complementary metrics for hot memory distribution control

4. mm/damon: add PA-mode cache for eligible memory detection lag
    - Implements rolling window cache to compensate for PA-mode detection lag
    - Adds configurable cache timeout via sysfs

Testing Status
==============

Functionally tested on a two-node heterogeneous memory system (DRAM + CXL)
with PUSH+PULL scheme configuration.

This is an RFC and feedback on the design is appreciated.

Ravi Jonnalagadda (4):
  mm/damon/sysfs: set goal_tuner after scheme creation
  mm/damon: fix esz=0 quota bypass allowing unlimited migration
  mm/damon: add node_eligible_mem_bp and node_ineligible_mem_bp goal
    metrics
  mm/damon: add PA-mode cache for eligible memory detection lag

 include/linux/damon.h    |  51 ++++
 mm/damon/core.c          | 496 +++++++++++++++++++++++++++++++++++++--
 mm/damon/sysfs-schemes.c |  43 ++++
 3 files changed, 576 insertions(+), 14 deletions(-)

-- 
2.43.0