Re: [PATCH 8/8] mm/gup: batch contiguous same-folio PTEs into one refcount grab

[David Hildenbrand (Arm)]

On 11/2/26 20:30, Rik van Riel wrote:
> follow_page_pte() now walks a whole page table in one call, but still
> resolves and commits each PTE on its own, so a PTE-mapped large folio
> (mTHP) pays one try_grab_folio() per subpage.
> 
> Add gup_pte_batch_len() and give gup_grab_folio_pages() a run
> length. Each run starts with a full per-PTE resolve, then
> gup_pte_batch_len() finds how many more PTEs extend it with a pte_same()
> scan rather than a re-derivation of each page, and the run is committed
> with one refcount grab. Runs stop at folio boundaries, so plain base
> pages are unaffected.
> 
> Gather the dirty bits from all the PTEs in a batch, in order to mark
> the folio dirty for FOLL_WRITE.
> 
> Same benchmark as the previous change, with before again taken at the
> base of the series:
> 
>   gup_test -L -m 256 -n 65536 -r 16 -t
>                         before      after
>   64 kB mTHP           2946 us      231 us  (12.8x)
>   4 kB base (control)  2753 us     1198 us   (2.3x)
>   2 MB THP (control)     72 us       71 us
> 
> The 4 kB column is the previous change's 2.3x, unchanged: base pages
> share no folio, so there is nothing to batch. 64 kB mTHP goes from that
> change's 2.2x to 12.8x, a further 5.9x from committing a run with one
> refcount grab instead of one per subpage.
> 
> With FOLL_TOUCH, folio_mark_accessed() runs once per run instead of
> once per page: 512 calls before and 32 after over 2 MB of 64 kB
> folios, leaving them referenced but inactive after the first call.
> That matches the huge-page and fast paths, which mark once
> per folio. (Measured by Yuan-Hao Hsu.)
> 
> Suggested-by: David Hildenbrand <david@xxxxxxxxxx>
> Signed-off-by: Rik van Riel <riel@xxxxxxxxxxx>
> ---
>  mm/gup.c | 49 ++++++++++++++++++++++++++++++++++++++++++++++---
>  1 file changed, 46 insertions(+), 3 deletions(-)
> 
> diff --git a/mm/gup.c b/mm/gup.c
> index 67e20b326be6e..47f204e21a3a7 100644
> --- a/mm/gup.c
> +++ b/mm/gup.c
> @@ -937,6 +937,35 @@ static long gup_lookup_pte(struct vm_area_struct *vma, unsigned long address,
>  	return 0;
>  }
>  
> +/*
> + * Return how many PTEs map consecutive pages of the same folio and can be
> + * committed as one run. Always at least 1.
> + *
> + * The write-fault and unshare checks in gup_lookup_pte() are per PTE, but a
> + * writable run needs no repeat: a writable anon page is exclusive. A read-only
> + * run under FOLL_WRITE or FOLL_PIN does need the per-page check, so it stays
> + * one page at a time.

That's a clear indication that this needs more work.

I'll try to get to it soon and see how it could be done in a cleaner way, likely
it should be based on top of the gup-fast series where we will have helpers to
check for required unsharing of multiple PTEs.

-- 
Cheers,

David