Re: [PATCH v4 07/17] mm/mglru, gup: mark folios referenced via a fast helper

From: Kairui Song

Date: Mon Nov 09 2026 - 04:33:41 EST


On Mon, Nov 9, 2026 at 5:04 AM Barry Song <baohua@xxxxxxxxxx> wrote:
>
> On Fri, Oct 16, 2026 at 2:15 AM Kairui Song via B4 Relay
> <devnull+kasong.tencent.com@xxxxxxxxxx> wrote:
> >
> > From: Kairui Song <kasong@xxxxxxxxxxx>
> >
> > Introduce folio_inc_lru_refs_fast(): raise refs from 0 to
> > LRU_REFS_REFERENCED via cmpxchg, or just set PG_referenced for the
> > classical LRU as before. No counter update, promotion, and guarteens
> > no other complex operations in following commits.
> >
> > Convert the gup fast-path sites. No behavior change for classical LRU.
> >
> > Tested-by: zhaozhengzhuo <zhaozhengzhuo@xxxxxxxxxxxxx>
> > Signed-off-by: Kairui Song <kasong@xxxxxxxxxxx>
> > ---
> > include/linux/mm_inline.h | 25 +++++++++++++++++++++++++
> > mm/gup.c | 6 +++---
> > 2 files changed, 28 insertions(+), 3 deletions(-)
> >
> > diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h
> > index 65415eeac802..effa13c63632 100644
> > --- a/include/linux/mm_inline.h
> > +++ b/include/linux/mm_inline.h
> > @@ -454,6 +454,31 @@ static inline bool lru_gen_del_folio(struct lruvec *lruvec, struct folio *folio,
> > }
> > #endif /* CONFIG_LRU_GEN */
> >
> > +/**
> > + * folio_inc_lru_refs_fast - Bump folio refs count without promotion.
> > + * @folio: the folio
> > + *
> > + * Raise refs from 0 to LRU_REFS_REFERENCED and leave hotter folios
> > + * untouched. For the classical LRU, a plain PG_referenced set.
> > + */
> > +static __always_inline void folio_inc_lru_refs_fast(struct folio *folio)
> > +{
> > + unsigned long new_flags, old_flags;
> > +
> > + if (!lru_gen_enabled()) {
> > + folio_set_referenced(folio);
> > + return;
> > + }
> > +
> > + old_flags = READ_ONCE(*folio_flags(folio, 0));
> > + do {
> > + if (lru_get_refs_flags(old_flags))
> > + break;
> > + new_flags = old_flags;
> > + lru_set_refs_flags(&new_flags, LRU_REFS_REFERENCED);
> > + } while (!try_cmpxchg(folio_flags(folio, 0), &old_flags, new_flags));
> > +}
>
> What is the actual purpose of this patch? Is it to skip
> cmpxchg() when the folio already has non-zero refs for
> better performance?

Right, as the name suggests, it's a faster helper, sacrificing some
recency accuracy (limited to LRU_REFS_REFERENCED, so it never touch
any counter). But the low bar for this helper is still equal to
mainline so I think it's fine at this point. I can add more commit
message or comment on this.