[PATCH RFC v2 0/3] Demote to lower tier using non-temporal stores

[Yiannis Nikolakopoulos]

In most memory tiering scenarios, the memory to be demoted is expected
to be cold and most likely out of the node's last-level cache (as well
as target pages in the target node). Using non-temporal stores instead
of a standard memcpy path can reduce the cache pollution in the local
node and the bandwidth overhead to the target node. Furthermore, for
certain types of CXL devices that support in-line memory compression,
the last-level cache eviction patterns can negatively affect the
bandwidth of the device. Non-temporal stores can mitigate this.

This patch-set introduces a new migrate_mode flag for using non-temporal
stores that is used only in the demotion path. Patch 1 adds some helpers in
x86 and mm to bring non-temporal stores support to a respective folio_copy
function. Patch 2 adds the new flag and necessary changes for compatibility
with the existing behavior. Patch 3 uses the new flag for demotions.

Experimental data: in a CXL system with 1 memory expander, a microbenchmark
that allocates N=64 GB memory in the local node and then triggers demotion
using memory.reclaim, shows a practically complete elimination of read
traffic on the device, i.e. write traffic is N GB with and without the
patch, while read traffic drops from N to almost 0 with the patch.

Opens:
1. There is some "duplication" in the x86 tree and a bit in mm. Can we do
   something better there? As it is now in copy_mc_to_kernel_nt we
duplicate the machine check functionality, which if available will override
the non-temporal. We were not sure how to prioritize these two and what's
the best approach here. Can we completely skip the machine checked for this
path? Huan Nguyen has some ideas here that we will align for the next
version.
2. I am not sure how this should be structured so that it is easily
adopted in other architecture trees (e.g. aarch64). We rely on
memcpy_flushcache for x86_64 but this does not use non temporal stores
in ARM. ARM support is currently out of our scope but any input is
appreciated.

Signed-off-by: Yiannis Nikolakopoulos <yiannis.nikolakop@xxxxxxxxx>
---
Changes in v2:
- Remove the KConfig option that was guarding this feature (patch 3) and
  use it always in x86_64 and memcpy_flushcache (non-temporal stores
should be there).
- Link to v1: https://patch.msgid.link/20260526-rfc-nt-demote-v1-0-eb9c9422daef@xxxxxxxxxxx

To: Thomas Gleixner <tglx@xxxxxxxxxx>
To: Ingo Molnar <mingo@xxxxxxxxxx>
To: Borislav Petkov <bp@xxxxxxxxx>
To: Dave Hansen <dave.hansen@xxxxxxxxxxxxxxx>
To: x86@xxxxxxxxxx
To: "H. Peter Anvin" <hpa@xxxxxxxxx>
To: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
To: David Hildenbrand <david@xxxxxxxxxx>
To: Lorenzo Stoakes <lorenzo.stoakes@xxxxxxxxxx>
To: "Liam R. Howlett" <Liam.Howlett@xxxxxxxxxx>
To: Vlastimil Babka <vbabka@xxxxxxxxxx>
To: Mike Rapoport <rppt@xxxxxxxxxx>
To: Suren Baghdasaryan <surenb@xxxxxxxxxx>
To: Michal Hocko <mhocko@xxxxxxxx>
To: Trond Myklebust <trondmy@xxxxxxxxxx>
To: Anna Schumaker <anna@xxxxxxxxxx>
To: Zi Yan <ziy@xxxxxxxxxx>
To: Matthew Brost <matthew.brost@xxxxxxxxx>
To: Joshua Hahn <joshua.hahnjy@xxxxxxxxx>
To: Rakie Kim <rakie.kim@xxxxxx>
To: Byungchul Park <byungchul@xxxxxx>
To: Gregory Price <gourry@xxxxxxxxxx>
To: Ying Huang <ying.huang@xxxxxxxxxxxxxxxxx>
To: Alistair Popple <apopple@xxxxxxxxxx>
To: Steven Rostedt <rostedt@xxxxxxxxxxx>
To: Masami Hiramatsu <mhiramat@xxxxxxxxxx>
To: Mathieu Desnoyers <mathieu.desnoyers@xxxxxxxxxxxx>
To: Brendan Jackman <jackmanb@xxxxxxxxxx>
To: Johannes Weiner <hannes@xxxxxxxxxxx>
Cc: linux-kernel@xxxxxxxxxxxxxxx
Cc: linux-mm@xxxxxxxxx
Cc: linux-nfs@xxxxxxxxxxxxxxx
Cc: linux-trace-kernel@xxxxxxxxxxxxxxx

---
Alirad Malek (3):
      mm, x86: support copying a folio using non-temporal stores
      mm: new migrate_mode flag for async using non-temporal stores
      mm: use non-temporal stores for demotion

 arch/x86/include/asm/uaccess.h |  4 ++++
 arch/x86/lib/copy_mc.c         | 26 ++++++++++++++++++++++++++
 fs/nfs/write.c                 |  2 +-
 include/linux/highmem.h        | 32 ++++++++++++++++++++++++++++++++
 include/linux/migrate_mode.h   |  9 +++++++++
 include/linux/mm.h             |  1 +
 include/trace/events/migrate.h |  1 +
 mm/compaction.c                | 18 +++++++++---------
 mm/migrate.c                   | 22 +++++++++++++++-------
 mm/util.c                      | 17 +++++++++++++++++
 10 files changed, 115 insertions(+), 17 deletions(-)
---
base-commit: 1f318b96cc84d7c2ab792fcc0bfd42a7ca890681
change-id: 20260526-rfc-nt-demote-0fafadfdd006

Best regards,
--  
Yiannis Nikolakopoulos <yiannis.nikolakop@xxxxxxxxx>