Re: [PATCH v4 15/17] mm/mglru: make folio_inc_lru_refs lruvec lockless

[Barry Song]

On Fri, Oct 16, 2026 at 2:16 AM Kairui Song via B4 Relay
<devnull+kasong.tencent.com@xxxxxxxxxx> wrote:
>
> From: Kairui Song <kasong@xxxxxxxxxxx>
>
> The lruvec spinlock in folio_inc_lru_refs was only taken to keep
> concurrent aging from corrupting the size counters.  This is no
> longer needed: the per-generation counters are atomic, and the
> active/inactive ABI counters are updated per folio and linearized by
> the CAS on the folio flags, so concurrent aging can no longer corrupt
> them.  The same argument already allowed folio_reset_lru_refs() to
> drop the lock.
>
> Use folio_lruvec_live_get()/folio_lruvec_live_put() to hold the RCU
> read lock around the lruvec lookup, and drop the spinlock entirely.
>

Yes, this is cool. We have two paths calling
folio_inc_lru_refs() or folio_inc_lru_refs_walk().

1. folio_mark_accessed() from syscall access
2. walk_update_folio() from page table walker or look_around

For the latter, we can't actually cross multiple
lruvecs/memcgs, because of get_pfn_folio() as shown
below:

static struct folio *get_pfn_folio(unsigned long pfn, struct mem_cgroup *memcg,
                                   struct pglist_data *pgdat)
{
        ...
        rcu_read_lock();
        if (folio_memcg(folio) != memcg)
                folio = NULL;
        rcu_read_unlock();
        ...
}

Consider a common library file used by 100 memcgs, with
its folio residing in the root memcg.

When walking the 99 non-root memcgs, even if the folio
has been accessed, we won't get a chance to call
folio_inc_lru_refs_walk() for it. As a result, this folio
may eventually reach the eviction rmap path. With a high
mapcount, folio_referenced() could become quite
expensive.

That's what I was trying to point out here as well:

https://lore.kernel.org/linux-mm/20261102213915.57805-1-baohua@xxxxxxxxxx/#t

For the former, you can get the lruvec with:

lruvec = lruvec_live_lock_irq(folio_lruvec(folio));

I'm wondering whether we could eventually unify them
so that both support cross-memcg inc_refs.

This isn't a blocker for the patch, just something I've
been thinking about.

> Without the lock, aging can advance max_seq while a promotion is in
> flight.  Generations are indexed by seq % MAX_NR_GENS, so the folio
> flags can return to the value read earlier and the cmpxchg succeeds
> with a target gen computed from a stale max_seq.  Recheck max_seq after
> a gen-changing cmpxchg: if it moved, account the committed update and
> retry with LRU_REF_FORCE, which promotes the folio to the newest gen.
> The retry may count the access twice, which is acceptable and avoids
> folio_activate(), whose LRU lock causes latency jitter.  smp_rmb()
> orders the flags read, including the one returned by a failed
> cmpxchg(), before the max_seq read.
>
> Also simplify the post-CAS accounting guards: gen is only assigned
> non-negative values after the old_gen < 0 early exit, so gen != old_gen
> implies gen >= 0, and lru_gen_update_size() needs no explicit guard.
> The active/inactive ABI update keeps its gen >= 0 check because an
> off-LRU folio has no lruvec to update.
>
> Signed-off-by: Kairui Song <kasong@xxxxxxxxxxx>
> ---

Thanks,
Barry