Re: [PATCH v4 1/5] mm: rmap: support batched checks of the references for large folios

[Baolin Wang]

Hi Ryan,

On 2025/12/24 21:24, Ryan Roberts wrote:
> On 23/12/2025 05:48, Baolin Wang wrote:
> > Currently, folio_referenced_one() always checks the young flag for each PTE
> > sequentially, which is inefficient for large folios. This inefficiency is
> > especially noticeable when reclaiming clean file-backed large folios, where
> > folio_referenced() is observed as a significant performance hotspot.
> >
> > Moreover, on Arm64 architecture, which supports contiguous PTEs, there is already
> > an optimization to clear the young flags for PTEs within a contiguous range.
> > However, this is not sufficient. We can extend this to perform batched operations
> > for the entire large folio (which might exceed the contiguous range: CONT_PTE_SIZE).
> >
> > Introduce a new API: clear_flush_young_ptes() to facilitate batched checking
> > of the young flags and flushing TLB entries, thereby improving performance
> > during large folio reclamation. And it will be overridden by the architecture
> > that implements a more efficient batch operation in the following patches.
> >
> > Signed-off-by: Baolin Wang <baolin.wang@xxxxxxxxxxxxxxxxx>
>
> With the 2 niggles below addressed:
>
> Reviewed-by: Ryan Roberts <ryan.roberts@xxxxxxx>

Thanks.

> > ---
> >   include/linux/mmu_notifier.h |  9 +++++----
> >   include/linux/pgtable.h      | 35 +++++++++++++++++++++++++++++++++++
> >   mm/rmap.c                    | 29 +++++++++++++++++++++++++++--
> >   3 files changed, 67 insertions(+), 6 deletions(-)
> >
> > diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h
> > index d1094c2d5fb6..dbbdcef4abf1 100644
> > --- a/include/linux/mmu_notifier.h
> > +++ b/include/linux/mmu_notifier.h
> > @@ -515,16 +515,17 @@ static inline void mmu_notifier_range_init_owner(
> >   	range->owner = owner;
> >   }
> >   
> > -#define ptep_clear_flush_young_notify(__vma, __address, __ptep)		\
> > +#define ptep_clear_flush_young_notify(__vma, __address, __ptep, __nr)	\
>
> I think I previously suggested that this should be renamed to
> clear_flush_young_ptes_notify() given that it is now a batch operation. Were
> others against that or did you forget?

Sorry, I missed this comment. Yes, sounds reasonable to me. Will do in 
the next version.

> >   ({									\
> >   	int __young;							\
> >   	struct vm_area_struct *___vma = __vma;				\
> >   	unsigned long ___address = __address;				\
> > -	__young = ptep_clear_flush_young(___vma, ___address, __ptep);	\
> > +	unsigned int ___nr = __nr;					\
> > +	__young = clear_flush_young_ptes(___vma, ___address, __ptep, ___nr);	\
> >   	__young |= mmu_notifier_clear_flush_young(___vma->vm_mm,	\
> >   						  ___address,		\
> >   						  ___address +		\
> > -							PAGE_SIZE);	\
> > +						  ___nr * PAGE_SIZE);	\
> >   	__young;							\
> >   })
> >   
> > @@ -650,7 +651,7 @@ static inline void mmu_notifier_subscriptions_destroy(struct mm_struct *mm)
> >   
> >   #define mmu_notifier_range_update_to_read_only(r) false
> >   
> > -#define ptep_clear_flush_young_notify ptep_clear_flush_young
> > +#define ptep_clear_flush_young_notify clear_flush_young_ptes
> >   #define pmdp_clear_flush_young_notify pmdp_clear_flush_young
> >   #define ptep_clear_young_notify ptep_test_and_clear_young
> >   #define pmdp_clear_young_notify pmdp_test_and_clear_young
> > diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
> > index 2f0dd3a4ace1..fcf7a7820061 100644
> > --- a/include/linux/pgtable.h
> > +++ b/include/linux/pgtable.h
> > @@ -1087,6 +1087,41 @@ static inline void wrprotect_ptes(struct mm_struct *mm, unsigned long addr,
> >   }
> >   #endif
> >   
> > +#ifndef clear_flush_young_ptes
> > +/**
> > + * clear_flush_young_ptes - Clear the access bit and perform a TLB flush for PTEs
> > + *			    that map consecutive pages of the same folio.
> > + * @vma: The virtual memory area the pages are mapped into.
> > + * @addr: Address the first page is mapped at.
> > + * @ptep: Page table pointer for the first entry.
> > + * @nr: Number of entries to clear access bit.
> > + *
> > + * May be overridden by the architecture; otherwise, implemented as a simple
> > + * loop over ptep_clear_flush_young().
> > + *
> > + * Note that PTE bits in the PTE range besides the PFN can differ. For example,
> > + * some PTEs might be write-protected.
> > + *
> > + * Context: The caller holds the page table lock.  The PTEs map consecutive
> > + * pages that belong to the same folio.  The PTEs are all in the same PMD.
> > + */
> > +static inline int clear_flush_young_ptes(struct vm_area_struct *vma,
> > +					 unsigned long addr, pte_t *ptep,
> > +					 unsigned int nr)
> > +{
> > +	int young;
> > +
> > +	young = ptep_clear_flush_young(vma, addr, ptep);
> > +	while (--nr) {
> > +		ptep++;
> > +		addr += PAGE_SIZE;
> > +		young |= ptep_clear_flush_young(vma, addr, ptep);
> > +	}
>
> I think it's better to avoid the two ptep_clear_flush_young() calls if we can.
> Personally I think we should just go for the simple:
>
> 	for (i = 0; i < nr; ++i, ++ptep, addr += PAGE_SIZE)
> 		young |= ptep_clear_flush_young(vma, addr, ptep);

ACK. Thanks for reviewing.