Re: [PATCH RFC 3/3] mm: use non-temporal stores for demotion

[Yiannis Nikolakopoulos]

On Tue, May 26, 2026 at 6:25 PM Gregory Price <gourry@xxxxxxxxxx> wrote:
>
> On Tue, May 26, 2026 at 01:37:04PM +0200, Yiannis Nikolakopoulos wrote:
> > From: Alirad Malek <alirad.malek@xxxxxxxxxxx>
> >
> > Memory demoted to a lower tier is assumed to be cold and most likely out of
> > the CPU's last level cache. Additionally, in certain demotion targets (e.g.
> > CXL devices with compressed memory) the bandwidth can be negatively
> > impacted by the eviction patterns of the last level cache when standard
> > memcpy is used. When the feature is enabled, use the
> > MIGRATE_ASYNC_NON_TEMPORAL_STORES flag in demotions to trigger the folio
> > copy path using non-temporal stores.
> >
> > Signed-off-by: Alirad Malek <alirad.malek@xxxxxxxxxxx>
> > Co-developed-by: Yiannis Nikolakopoulos <yiannis.nikolakop@xxxxxxxxx>
> > Signed-off-by: Yiannis Nikolakopoulos <yiannis.nikolakop@xxxxxxxxx>
> > ---
> >  mm/Kconfig   | 8 ++++++++
> >  mm/migrate.c | 9 ++++++++-
> >  2 files changed, 16 insertions(+), 1 deletion(-)
> >
> > diff --git a/mm/Kconfig b/mm/Kconfig
> > index ebd8ea353687..4b7a75b57f6e 100644
> > --- a/mm/Kconfig
> > +++ b/mm/Kconfig
> > @@ -645,6 +645,14 @@ config MIGRATION
> >         pages as migration can relocate pages to satisfy a huge page
> >         allocation instead of reclaiming.
> >
> > +config DEMOTION_WITH_NON_TEMPORAL_STORES
> > +     bool "Use non-temporal stores for demotion"
> > +     default n
> > +     depends on MIGRATION
> > +     help
> > +       Enable non-temporal stores when migrating pages due to demotion.
> > +       If disabled, demotion uses regular migration copy paths.
> > +
>
> Do we actually need this config flag or should we just default to this
> (if the arch supports NT stores)?
As discussed in the sync call, I was just being defensive :).
In fact the non-temporal instructions should be in all x86_64 afaict.
So I've removed this in the next version (landing soon).

/Yiannis
>
> >  config DEVICE_MIGRATION
> >       def_bool MIGRATION && ZONE_DEVICE
> >
> > diff --git a/mm/migrate.c b/mm/migrate.c
> > index ff6cf50e7b0b..368d40dc8772 100644
> > --- a/mm/migrate.c
> > +++ b/mm/migrate.c
> > @@ -862,7 +862,10 @@ static int __migrate_folio(struct address_space *mapping, struct folio *dst,
> >       if (folio_ref_count(src) != expected_count)
> >               return -EAGAIN;
> >
> > -     rc = folio_mc_copy(dst, src);
> > +     if (mode == MIGRATE_ASYNC_NON_TEMPORAL_STORES)
> > +             rc = folio_mc_copy_nt(dst, src);
> > +     else
> > +             rc = folio_mc_copy(dst, src);
> >       if (unlikely(rc))
> >               return rc;
> >
> > @@ -2081,6 +2084,10 @@ int migrate_pages(struct list_head *from, new_folio_t get_new_folio,
> >       LIST_HEAD(split_folios);
> >       struct migrate_pages_stats stats;
> >
> > +     if (IS_ENABLED(CONFIG_DEMOTION_WITH_NON_TEMPORAL_STORES) &&
> > +             reason == MR_DEMOTION && mode == MIGRATE_ASYNC)
> > +             mode = MIGRATE_ASYNC_NON_TEMPORAL_STORES;
> > +
> >       trace_mm_migrate_pages_start(mode, reason);
> >
> >       memset(&stats, 0, sizeof(stats));
> >
> > --
> > 2.43.0
> >