[PATCH 0/2] mm/gup: batch PTE-mapped large folios in follow_page_mask()

[Rik van Riel]

follow_page_mask() reports the size of the page it found with a page_mask:
a bitmask of the enclosing naturally-aligned huge page. That form can only
describe a power-of-two block aligned to its own size, so a PTE-mapped
large folio (mTHP) is walked one page at a time even though it maps a
contiguous run, while the PMD/PUD-mapped cases already return their whole
mapping in one step.

Patch 1 replaces the page_mask output with nr_pages (a plain count) and
adds an @end argument bounding the walk, with no functional change: the
huge PMD/PUD paths report the same stride as before, just as a count.

Patch 2 uses the new interface to batch the PTE case: follow_pte_batch()
counts consecutive present PTEs that map consecutive pages of the same
folio with a uniform write bit, bounded by the page table, @end, the VMA,
and the folio itself, then follow_page_pte() hands the whole run back in
one call.

Measured with mm/gup_test.c (PIN_LONGTERM_BENCHMARK, the slow
pin_user_pages() path) on a 256 MB MADV_HUGEPAGE anonymous region in a
4 CPU VM, median get time over 16 iterations. Each folio size was confirmed
through the per-size anon_fault_alloc counters (4096 folios for 64 kB, 128
for 2 MB):

  gup_test -L -m 256 -n 65536 -r 16 -t
                        before      after
  64 kB mTHP            3140 us      412 us   (7.6x)
  2 MB THP (control)      78 us       76 us
  4 kB base (control)   3010 us     3042 us

The PMD-mapped 2 MB THP already returns the whole mapping in one step, so
it stays fast and unchanged. The 4 kB baseline shows the per-page walk cost
that the 64 kB case paid before this change; only the PTE-mapped large
folio case improves.

Both patches apply standalone against mm-unstable; no other series is
required.

Rik van Riel (2):
  mm/gup: pass an end address to follow_page_mask() and return a page
    count
  mm/gup: batch contiguous PTE-mapped large folios in follow_page_mask()

 mm/gup.c | 139 ++++++++++++++++++++++++++++++++++++++-----------------
 1 file changed, 97 insertions(+), 42 deletions(-)

base-commit: 7368ccdeff50c27809d3a8276c85bae7e402c96c
---
Cc: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
Cc: linux-mm@xxxxxxxxx
To: linux-kernel@xxxxxxxxxxxxxxx
Cc: Dave Hansen <dave.hansen@xxxxxxxxxxxxxxx>
Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
Cc: Suren Baghdasaryan <surenb@xxxxxxxxxx>
Cc: Lorenzo Stoakes <ljs@xxxxxxxxxx>
Cc: Vlastimil Babka <vbabka@xxxxxxxxxx>
Cc: David Hildenbrand <david@xxxxxxxxxx>
Cc: "Liam R. Howlett" <liam@xxxxxxxxxxxxx>
Cc: Mike Rapoport <rppt@xxxxxxxxxx>
Cc: Michal Hocko <mhocko@xxxxxxxx>
Cc: Jason Gunthorpe <jgg@xxxxxxxx>
Cc: John Hubbard <jhubbard@xxxxxxxxxx>
Cc: Peter Xu <peterx@xxxxxxxxxx>
Cc: Matthew Wilcox <willy@xxxxxxxxxxxxx>
Cc: Usama Arif <usamaarif642@xxxxxxxxx>
Cc: Rik van Riel <riel@xxxxxxxxxxx>
Cc: kernel-team@xxxxxxxx
-- 
2.53.0-Meta