Re: [PATCH v2 7/7] mm/mglru: batch move folios to the second-oldest gen's LRU

[Lian Wang]

On Fri, 28 Aug 2026 07:47:04 +0800 "Barry Song (Xiaomi)" <baohua@xxxxxxxxxx> wrote:

> Detect folios that need to move from the oldest generation to
> the second-oldest generation, and batch-move them together.
> This can significantly reduce the sys time of inc_min_seq(),
> especially when the other type is significantly behind the
> preferred type.
>
> Assisted-by: gemini:gemini-3.6-flash
> Signed-off-by: Barry Song (Xiaomi) <baohua@xxxxxxxxxx>
> Reviewed-by: Baoquan He <baoquan.he@xxxxxxxxx>
> Tested-by: Xueyuan Chen <xueyuan.chen21@xxxxxxxxx>
> ---
>  mm/vmscan.c | 20 +++++++++++++++++++-
>  1 file changed, 19 insertions(+), 1 deletion(-)
>
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index 17524e96fe64..2b6f3f05ce60 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -3931,6 +3931,19 @@ static void clear_mm_walk(void)
>  		kfree(walk);
>  }
>
> +static inline void flush_lru_batch(struct list_head *head, struct list_head **batch_end,
> +				   struct list_head *dst)
> +{
> +	LIST_HEAD(movable);
> +
> +	if (!*batch_end)
> +		return;
> +
> +	list_cut_position(&movable, head, *batch_end);
> +	list_splice_tail_init(&movable, dst);
> +	*batch_end = NULL;
> +}

This is safe because `batch_end` always marks a contiguous prefix of `head`:
the loop walks from head to tail and flushes the prefix before moving any folio
that was already promoted by aging.

> +
>  static bool inc_min_seq(struct lruvec *lruvec, int type, int swappiness)
>  {
>  	int zone;
> @@ -3953,8 +3966,10 @@ static bool inc_min_seq(struct lruvec *lruvec, int type, int swappiness)
>  			lru_gen_is_active(lruvec, target_gen));
>  	/* prevent cold/hot inversion if the type is evictable */
>  	for (zone = 0; zone < MAX_NR_ZONES; zone++) {
> +		struct list_head *target_list = &lrugen->folios[target_gen][type][zone];
>  		struct list_head *head = &lrugen->folios[old_gen][type][zone];
>  		struct list_head *pos = head->next;
> +		struct list_head *batch_end = NULL;
>  		long delta = 0;
>
>  		while (pos != head) {
> @@ -3974,7 +3989,7 @@ static bool inc_min_seq(struct lruvec *lruvec, int type, int swappiness)
>  			new_gen = __folio_inc_gen(folio, old_gen, &gen_increased);
>  			if (gen_increased) {
>  				delta += nr_pages;
> -				list_move_tail(&folio->lru, &lrugen->folios[new_gen][type][zone]);
> +				batch_end = &folio->lru;
>
>  				/* don't count the workingset being lazily promoted */
>  				if (refs + workingset != BIT(LRU_REFS_WIDTH) + 1) {
> @@ -3984,11 +3999,14 @@ static bool inc_min_seq(struct lruvec *lruvec, int type, int swappiness)
>  						   lrugen->protected[hist][type][tier] + nr_pages);
>  				}
>  			} else {
> +				flush_lru_batch(head, &batch_end, target_list);
>  				list_move(&folio->lru, &lrugen->folios[new_gen][type][zone]);
>  			}
>  			if (!--remaining)
>  				break;
>  		}
> +		flush_lru_batch(head, &batch_end, target_list);

The flush in the `else` keeps every already-promoted folio ahead of the
normal old-to-target batch. This final flush also covers the batch-limit exit
before the counters are updated and the lock can be dropped.

> +
>  		WRITE_ONCE(lrugen->nr_pages[old_gen][type][zone],
>  			   lrugen->nr_pages[old_gen][type][zone] - delta);
>  		WRITE_ONCE(lrugen->nr_pages[target_gen][type][zone],
> --
> 2.34.1

The moved list entries and `delta` therefore cover the same folios, while the
relative order of each normal-folio run is preserved. Looks good to me.

Reviewed-by: Lian Wang <lianux.mm@xxxxxxxxx>