[RFC v2 0/3] mm/mglru: proactive aging via memory.aging

[Zicheng Wang]

MGLRU inverts the reclaim order when anonymous memory is faulted in
bulk: anonymous pages sit in the young generations while file pages
sit in the oldest two, so reclaim evicts hot file pages before cold
anonymous pages.  swappiness cannot correct it.  This series adds a
per-cgroup proactive-aging operation, memory.aging, with the
observability to drive it as a closed loop, so userspace can rebalance
a cgroup's generations before reclaim runs.

The problem
-----------

MGLRU places pages by access path, not hotness.  On the current tree
the entry rules in lru_gen_folio_seq() give:

  - faulted-in pages (anonymous pages; the fault path sets the active
    mark) land at the 2nd youngest generation (max_seq - 1);
  - file pages from read()/fadvise() land in the oldest two
    generations: non-workingset at min_seq, workingset at min_seq + 1.

Pictorially, for a full four-generation window::

       youngest                          oldest
       max_seq   max_seq-1   min_seq+1   min_seq
       -------------------------------------------
 anon    .          X           .           .     (faulted in)
 file    .          .           X           X     (read/fadvise)
                                            ^
                              reclaim scans from here -> evicts file first

Reclaim scans from min_seq, so file pages are evicted before the
anonymous pages two generations younger, a hot/cold inversion.

swappiness does not fix it: it only selects which type to scan, not
the ordering within a type, so reclaim can still evict hot pages.
swappiness=201 (SWAPPINESS_ANON_ONLY) just flips the inversion to the
other side: no value recovers a correct hot/cold order, because the
order comes from the access path, not access recency [1].  This is not
Android-specific; the same file over-reclaim is shown on servers and
worked around there by forcing an age iteration before reclaim.

Why userspace-driven
--------------------

The benefit is workload-dependent: file-cache-bound servers gain from
aging, anon-bound servers do not, so no kernel default is correct for
all.  The kernel also cannot know when to age: on Android the right
moment is the foreground-to-background transition, when the app's pages
are cold but their PTE accessed bits are still accurate from foreground
execution, a framework concept.  The kernel therefore provides the
mechanism; userspace runs the loop.

Design: observation and control
-------------------------------

The series gives userspace one control primitive and the observation to
use it.  All three pieces are per-cgroup and gated by CONFIG_LRU_GEN,
and together they form a closed loop:

  control   memory.aging          write nr_gens to advance max_seq without
                                  reclaim; the aging counterpart of
                                  memory.reclaim
  observe   nr_oldest_anon/file   pages in the oldest generation of each
                                  type: what the next reclaim can evict
            aging                 number of aging passes, paired with
                                  workingset_refault to spot over-aging

A policy reads nr_oldest_* to see whether anon is sheltered, ages with
memory.aging, then checks the aging counter and workingset_refault.
A typical flow ages to rebalance, then reclaims::

    echo 2 > /sys/fs/cgroup/foo/memory.aging
    echo "100M" > /sys/fs/cgroup/foo/memory.reclaim

Why cgroup v2
-------------

memory.reclaim already lives in cgroup v2, but it is broken when
anonymous pages are sheltered in the young generations: reclaim cannot
reach anon until aging advances it.  A useful memory.reclaim therefore
requires aging first.  Aging is not a new MGLRU detail leaked into
cgroup v2; it is the other half of the proactive-reclaim operation
cgroup v2 already exposes.  Putting it anywhere else (sysfs) cuts one
operation across two ABIs and two addressing models.

memory.aging is gated by CONFIG_LRU_GEN and exposes one primitive
(advancing the generation), not internal data structures; if MGLRU is
removed, the file goes with it.

Alternatives considered
-----------------------

1) sysfs (/sys/kernel/mm/lru_gen/) fights sysfs's one-value-per-file,
stateless model: the command is multi-argument and per-(memcg,node),
the read is parameterized per memcg, and css-id addressing is hostile
to cgroup paths.  It also splits aging from memory.reclaim.

2) in-kernel automatic aging is workload-dependent (memcached regresses)
and needs a trigger the kernel cannot know: the Android
foreground-to-background moment is a framework concept.  MGLRU had this
once: `should_run_aging' fired on a generation-balance heuristic::

    if (young * MIN_NR_GENS > total)
        return true;
    if (old * (MIN_NR_GENS + 2) < total)
        return true;


3) extending memory.reclaim to run the debugfs aging command
(`+ memcg_id node_id seq`) overloads a reclaim file with a non-reclaim
operation and pulls MGLRU's debugfs command syntax into a generic
cgroup v2 ABI. But Aging is reclaim's counterpart, not one of its modes.

Results
-------
N39 (Device-X1 8 GB/SM7750, Device-X2 12 GB/SM8750).  Production telemetry,
~300 users, 30K+ device-hours:
    keep-alive          +15.4% (8 GB) / +10.2% (12 GB)
    IME dismiss latency -47.2%
    launch/exit jank >50ms   eliminated
    one aging pass      342 us median, never on a UX thread

AN90 (a new shipping product, separate from N39):
    scroll jank >50 ms  -22%, severe jank -26%
    cold-launch stalls >= 3 s  eliminated
    workingset_refault (anon + file) and direct reclaim both down

Server (Intel i7-14700F, 20C/28T, 32 GB DDR5-5600, 1 TB NVMe,
Ubuntu 24.04; same root cause, aging triggered on the
reclaim path when anon in the oldest gen ratio < 10%):
    fio random-read     +31.8%
    ripgrep             +15.8%
    memcached           -1.2%   (anon-bound; expected no gain)

N39 and AN90 are different products.

Why RFC
-------

v1 [2] put the control in procfs/debugfs and was NAKed for the
cgroup-v2 venue.  This version keeps aging in cgroup v2, next to
memory.reclaim, and argues why above.  Discussed at LSF/MM 2026 [3].

[1] swappiness=0/201 hot/cold inversion, and why the straightforward
    fix is deferred:
    https://lore.kernel.org/linux-mm/7829b070df1b405dbc97dd6a028d8c8a@xxxxxxxxx/
[2] v1: https://lore.kernel.org/linux-mm/20251128025315.3520689-1-wangzicheng@xxxxxxxxx/
[3] LSF/MM 2026 slides:
    https://docs.google.com/presentation/d/1hUogz6InyLn13c8CjHuvEIzE4rT7saVRUV6xpWZoNfQ/

Changes since v1
----------------

- Dropped the debugfs -> procfs move; aging is now a cgroup v2 file.
- Added the AGING counter and nr_oldest_anon/file observability.
- Corrected the generation-placement description to the current tree
  (faults at the 2nd youngest, file in the oldest two).

Zicheng Wang (3):
  mm/lru_gen: add AGING counter and proactive aging helper
  mm: memcontrol: add memory.aging cgroup v2 file
  mm/lru_gen: expose oldest-generation page counts in memory.stat

 Documentation/admin-guide/cgroup-v2.rst       |  53 +++++++++++
 Documentation/admin-guide/mm/multigen_lru.rst |  13 +++
 include/linux/mmzone.h                        |   3 +
 mm/internal.h                                 |  19 +++++
 mm/memcontrol.c                               |  63 +++++++++++
 mm/vmscan.c                                   | 109 +++++++++++++++++++++
 mm/vmstat.c                                   |   3 +
 7 files changed, 263 insertions(+)

--
2.25.1