Re: [RESEND] [PATCH v2] arm64/mm: Elide TLB flush in certain pte protection transitions

[Dev Jain]

On 04/11/25 5:45 am, Anshuman Khandual wrote:
> On 17/10/25 9:32 PM, Dev Jain wrote:
> > Currently arm64 does an unconditional TLB flush in mprotect(). This is not
> > required for some cases, for example, when changing from PROT_NONE to
> > PROT_READ | PROT_WRITE (a real usecase - glibc malloc does this to emulate
> > growing into the non-main heaps), and unsetting uffd-wp in a range.
> >
> > Therefore, implement pte_needs_flush() for arm64, which is already
> > implemented by some other arches as well.
> >
> > Running a userspace program changing permissions back and forth between
> > PROT_NONE and PROT_READ | PROT_WRITE, and measuring the average time taken
> > for the none->rw transition, I get a reduction from 3.2 microseconds to
> > 2.85 microseconds, giving a 12.3% improvement.
> >
> > Reviewed-by: Kefeng Wang <wangkefeng.wang@xxxxxxxxxx>
> > Signed-off-by: Dev Jain <dev.jain@xxxxxxx>
> > ---
> > mm-selftests pass. Based on 6.18-rc1.
> >
> > v1->v2:
> >   - Drop PTE_PRESENT_INVALID and PTE_AF checks, use ptdesc_t instead of
> >     pteval_t, return !!diff (Ryan)
> >
> >   arch/arm64/include/asm/tlbflush.h | 27 +++++++++++++++++++++++++++
> >   1 file changed, 27 insertions(+)
> >
> > diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h
> > index 18a5dc0c9a54..40df783ba09a 100644
> > --- a/arch/arm64/include/asm/tlbflush.h
> > +++ b/arch/arm64/include/asm/tlbflush.h
> > @@ -524,6 +524,33 @@ static inline void arch_tlbbatch_add_pending(struct arch_tlbflush_unmap_batch *b
> >   {
> >   	__flush_tlb_range_nosync(mm, start, end, PAGE_SIZE, true, 3);
> >   }
> > +
> > +static inline bool __pte_flags_need_flush(ptdesc_t oldval, ptdesc_t newval)
> > +{
> > +	ptdesc_t diff = oldval ^ newval;
> > +
> > +	/* invalid to valid transition requires no flush */
> > +	if (!(oldval & PTE_VALID))
> Using pte_valid() helper would be better.
>
> 	if (!pte_valid(oldval))
> 		return false;

Unfortunately, that would need asm/pgtable.h, which already includes asm/tlbflush.h,
leading to circular include.

If I put all this code into asm/pgtable.h, then pte_valid() needs a pte_t or a pmd_t
to work on, which forces us to break away from the nice abstraction of directing
both pte_needs_flush and huge_pmd_needs_flush to __pte_flags_need_flush.


> > +		return false;
> > +
> > +	/* Transition in the SW bits requires no flush */
> > +	diff &= ~PTE_SWBITS_MASK;
> > +
> > +	return !!diff;
> > +}
> > +
> > +static inline bool pte_needs_flush(pte_t oldpte, pte_t newpte)
> > +{
> > +	return __pte_flags_need_flush(pte_val(oldpte), pte_val(newpte));
> > +}
> > +#define pte_needs_flush pte_needs_flush
> > +
> > +static inline bool huge_pmd_needs_flush(pmd_t oldpmd, pmd_t newpmd)
> > +{
> > +	return __pte_flags_need_flush(pmd_val(oldpmd), pmd_val(newpmd));
> > +}
> > +#define huge_pmd_needs_flush huge_pmd_needs_flush
> > +
> >   #endif
> >   
> >   #endif