[PATCH v4 01/17] mm/memcontrol: allow update of LRU statistic without holding LRU lock

[Barry Song (Xiaomi)]

On Fri, Oct 16, 2026 at 2:15 AM Kairui Song via B4 Relay <devnull+kasong.tencent.com@xxxxxxxxxx> wrote:
>
> From: Kairui Song <kasong@xxxxxxxxxxx>
>
> To enable moving file pages in folio_mark_accessed directly and lazily
> for MGLRU, allow updating the LRU statistic atomically without holding a
> lock. It may cause temporary counter underflow, which should be fine as
> we still follow final consistency of the counter, and it only serves as
> a estimated metrics, a little inaccuracy has no visible effect.

This might be a good idea. Updating the LRU size without
holding lru_lock could open up possibilities for further
optimizations.

Recently, I've been thinking about something in the aging
path. For example, suppose a folio belongs to the root
memcg and is heavily shared by some of its descendant
memcgs, which also access it.

Currently, aging in those descendant memcgs will simply
skip this folio. Eventually, we'll have to rely on the
rmap of the root memcg to determine whether the folio is
hot or cold within the root memcg.

Since the shared folio may have a very large mapcount,
walking its rmap could be quite expensive.

So I've been thinking about this and put together some
very PoC code. It doesn't really work; it's just meant to
illustrate what I have in mind:


diff --git a/mm/vmscan.c b/mm/vmscan.c
index 7ddef3b488b3..31b02478269b 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -3618,7 +3618,7 @@ static struct folio *get_pfn_folio(unsigned long pfn, struct mem_cgroup *memcg,
 		return NULL;
 
 	rcu_read_lock();
-	if (folio_memcg(folio) != memcg)
+	if (!mem_cgroup_is_descendant(memcg, folio_memcg(folio)))
 		folio = NULL;
 	rcu_read_unlock();
 
@@ -3723,7 +3723,7 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end,
 			continue;
 
 		if (last != folio) {
-			walk_update_folio(walk, args->vma, walk->lruvec, last, dirty);
+			walk_update_folio(walk, args->vma, folio_lruvec(last), last, dirty);
 
 			last = folio;
 			dirty = false;
@@ -3736,7 +3736,7 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end,
 		walk->mm_stats[MM_LEAF_YOUNG] += nr;
 	}
 
-	walk_update_folio(walk, args->vma, walk->lruvec, last, dirty);
+	walk_update_folio(walk, args->vma, folio_lruvec(last), last, dirty);
 	last = NULL;
 
 	if (i < PTRS_PER_PTE && get_next_vma(PMD_MASK, PAGE_SIZE, args, &start, &end))
@@ -3812,7 +3812,7 @@ static void walk_pmd_range_locked(pud_t *pud, unsigned long addr, struct vm_area
 			goto next;
 
 		if (last != folio) {
-			walk_update_folio(walk, vma, walk->lruvec, last, dirty);
+			walk_update_folio(walk, vma, folio_lruvec(last), last, dirty);
 
 			last = folio;
 			dirty = false;
@@ -3826,7 +3826,7 @@ static void walk_pmd_range_locked(pud_t *pud, unsigned long addr, struct vm_area
 		i = i > MIN_LRU_BATCH ? 0 : find_next_bit(bitmap, MIN_LRU_BATCH, i) + 1;
 	} while (i <= MIN_LRU_BATCH);
 
-	walk_update_folio(walk, vma, walk->lruvec, last, dirty);
+	walk_update_folio(walk, vma, folio_lruvec(last), last, dirty);
 
 	lazy_mmu_mode_disable();
 	spin_unlock(ptl);
@@ -4499,7 +4499,7 @@ bool lru_gen_look_around(struct page_vma_mapped_walk *pvmw, unsigned int nr)
 			continue;
 
 		if (last != folio) {
-			walk_update_folio(walk, vma, lruvec, last, dirty);
+			walk_update_folio(walk, vma, folio_lruvec(last), last, dirty);
 
 			last = folio;
 			dirty = false;
@@ -4511,7 +4511,7 @@ bool lru_gen_look_around(struct page_vma_mapped_walk *pvmw, unsigned int nr)
 		young += nr;
 	}
 
-	walk_update_folio(walk, vma, lruvec, last, dirty);
+	walk_update_folio(walk, vma, folio_lruvec(last), last, dirty);
 
 	lazy_mmu_mode_disable();
 

We may want to explore this after mglru-fg. What are your
thoughts on this?

>
> Signed-off-by: Kairui Song <kasong@xxxxxxxxxxx>
> ---
>  include/linux/mm_inline.h | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h
> index ab69b9930893..597f013c8e04 100644
> --- a/include/linux/mm_inline.h
> +++ b/include/linux/mm_inline.h
> @@ -47,11 +47,10 @@ static __always_inline void __update_lru_size(struct lruvec *lruvec,
>  {
>         struct pglist_data *pgdat = lruvec_pgdat(lruvec);
>
> -       lockdep_assert_held(&lruvec->lru_lock);
>         WARN_ON_ONCE(nr_pages != (int)nr_pages);
>
>         mod_lruvec_state(lruvec, NR_LRU_BASE + lru, nr_pages);
> -       __mod_zone_page_state(&pgdat->node_zones[zid],
> +       mod_zone_page_state(&pgdat->node_zones[zid],
>                                 NR_ZONE_LRU_BASE + lru, nr_pages);
>  }
>
>
> --
> 2.56.0
>
>

Best,
Barry