Re: [PATCH v4 15/17] mm/mglru: make folio_inc_lru_refs lruvec lockless
From: Kairui Song
Date: Mon Nov 09 2026 - 03:09:27 EST
On Mon, Nov 9, 2026 at 5:48 AM Barry Song <baohua@xxxxxxxxxx> wrote:
>
> On Fri, Oct 16, 2026 at 2:16 AM Kairui Song via B4 Relay
> <devnull+kasong.tencent.com@xxxxxxxxxx> wrote:
> >
> > From: Kairui Song <kasong@xxxxxxxxxxx>
> >
> > The lruvec spinlock in folio_inc_lru_refs was only taken to keep
> > concurrent aging from corrupting the size counters. This is no
> > longer needed: the per-generation counters are atomic, and the
> > active/inactive ABI counters are updated per folio and linearized by
> > the CAS on the folio flags, so concurrent aging can no longer corrupt
> > them. The same argument already allowed folio_reset_lru_refs() to
> > drop the lock.
> >
> > Use folio_lruvec_live_get()/folio_lruvec_live_put() to hold the RCU
> > read lock around the lruvec lookup, and drop the spinlock entirely.
> >
>
> Yes, this is cool. We have two paths calling
> folio_inc_lru_refs() or folio_inc_lru_refs_walk().
>
> 1. folio_mark_accessed() from syscall access
> 2. walk_update_folio() from page table walker or look_around
>
> For the latter, we can't actually cross multiple
> lruvecs/memcgs, because of get_pfn_folio() as shown
> below:
>
> static struct folio *get_pfn_folio(unsigned long pfn, struct mem_cgroup *memcg,
> struct pglist_data *pgdat)
> {
> ...
> rcu_read_lock();
> if (folio_memcg(folio) != memcg)
> folio = NULL;
> rcu_read_unlock();
> ...
> }
>
> Consider a common library file used by 100 memcgs, with
> its folio residing in the root memcg.
>
> When walking the 99 non-root memcgs, even if the folio
> has been accessed, we won't get a chance to call
> folio_inc_lru_refs_walk() for it. As a result, this folio
> may eventually reach the eviction rmap path. With a high
> mapcount, folio_referenced() could become quite
> expensive.
>
> That's what I was trying to point out here as well:
>
> https://lore.kernel.org/linux-mm/20261102213915.57805-1-baohua@xxxxxxxxxx/#t
Hi Barry,
Thanks for the review, I saw that too, very interesting approach! And
I believe it will helpful.
> For the former, you can get the lruvec with:
>
> lruvec = lruvec_live_lock_irq(folio_lruvec(folio));
>
> I'm wondering whether we could eventually unify them
> so that both support cross-memcg inc_refs.
Definately, actually for that link you posted, combined with the code
snip above, after FG, I think we can further simplify that, no need to
check mem_cgroup_is_descendant, we just do:
- folio_inc_lru_refs_walk: if the folio belongs to the lruvec being
walked, the batch stays in walk->lruvec
- folio_inc_lru_refs: if the folio is a remote one, per-folio, no batch
non-descendant folio sharing across memcg are actually quite common
e.g., libc and shared mapped memory, this could be more helpful? And
it seems fast for the walker.
folio_inc_lru_refs_walk follow the same rule as folio_inc_lru_refs
(with MAPPED flag), just faster, skipping RCU and the speculative
check afterwards as the walker owns the seq it is advancing, and
caller handles the batch update of counter. While folio_inc_lru_refs
only requires the caller to hold an reference to the folio it heavily
relies on RCU / CAS conventions, not easy to batch things there, but
it's still accetablily fast already (e.g. the fio read test showed
almost no performance penalty).
And I think the main reason MGLRU can't operate cross memcg was, lack
of a protocol to stablilze the memcg's gen and holding a lock is too
costly, now folio_inc_lru_refs handles that, and FG
lives on top of that. So we can try that out right after the FG series.