[RFC PATCH v3 0/3] mm: use non-temporal stores for demotion
From: Huan Nguyen
Date: Tue Nov 03 2026 - 17:34:03 EST
This series continues the non-temporal demotion work originally posted
by Yiannis Nikolakopoulos [1].
Motivation
==========
Memory tier demotion moves cold folios from local DRAM to slower memory
tiers, such as CXL or remote NUMA nodes. Standard store instructions
force the CPU to allocate cache lines for cold data that is unlikely to
be referenced in the near term.
Using non-temporal stores and prefetching on x86 bypasses CPU cache
allocation, protecting active workload working sets and improving
demotion throughput.
Changes in v3
=============
- Unified MCE-safe NT copy (addressing v2 Open #1):
Implement copy_mc_page_nt() in x86 assembly to provide machine-check
safe copying using non-temporal stores (movnti) and prefetchnta.
- Architecture abstraction (addressing v2 Open #2):
v2 relied on memcpy_flushcache(), which lacks non-temporal stores on
architectures like arm64. RFC v3 replaces this with a dedicated
__HAVE_ARCH_COPY_PAGE_NT interface and generic fallback, enabling
architectures to plug in native non-temporal routines independently.
- Memory ordering:
Add a wmb() barrier at the end of folio_mc_copy_nt() to ensure weakly
ordered non-temporal stores are drained before page tables are updated.
Discussions
===========
1. Migration mode vs. migration control refactoring:
Currently, this series introduces MIGRATE_ASYNC_NONTEMPORAL into
enum migrate_mode. In parallel, Shivank Garg's migration refactoring
series [2] consolidates migration arguments into migrate_control.
Once that lands, demotion can simply rely on the migration reason
instead of adding a dedicated migration mode and helper.
2. Large folio copy optimization:
Multi-page folios (such as 2MiB THP) are supported by iterating over
constituent 4KiB pages via copy_mc_page_nt(). Once the batch page
copy series [3] lands, copy_mc_page_nt() can be extended to copy an
entire multi-page folio in a single invocation, eliminating per-4KiB
loop overhead and keeping the prefetchnta stream uninterrupted.
Thanks to Yiannis Nikolakopoulos, Alirad Malek, and Dimitrios
Palyvos-Giannas for their original work, feedback, and collaboration.
References
==========
[1] https://lore.kernel.org/linux-mm/20260730-rfc-nt-demote-v2-0-452dbe3b5073@xxxxxxxxxxx/
[2] https://lore.kernel.org/linux-mm/20260902-migrate-refactor-shivank-v1-9-9dcca87669c4@xxxxxxx/
[3] https://lore.kernel.org/linux-mm/20261019-batpagecopy-v2-1-d75e7d0dfaae@xxxxxxx/
Signed-off-by: Huan Nguyen <nhuhuan@xxxxxxxxxx>
Huan Nguyen (3):
mm, x86: support copying a folio using non-temporal stores
mm/migrate: introduce async non-temporal migration mode
mm/vmscan: demote folios using non-temporal stores
arch/x86/include/asm/page_64.h | 2 ++
arch/x86/lib/copy_page_64.S | 65 ++++++++++++++++++++++++++++++++++
fs/nfs/write.c | 2 +-
include/linux/highmem.h | 21 +++++++++++
include/linux/migrate.h | 5 +++
include/linux/migrate_mode.h | 1 +
include/linux/mm.h | 1 +
include/trace/events/migrate.h | 1 +
mm/compaction.c | 18 +++++-----
mm/migrate.c | 29 +++++++++------
mm/util.c | 29 +++++++++++++++
mm/vmscan.c | 2 +-
12 files changed, 154 insertions(+), 22 deletions(-)
base-commit: 05441d697f7f9a927264f4058cbbc6a03dfcac9d
--
2.56.0.398.g882cbad275-goog