Re: [PATCH] mm/gup: batch PTE-mapped large folios in gup_fast_pte_range()

[Yuan-Hao Hsu]

On Wed, 07 Oct 2026 06:47:06 -0400, Rik van Riel wrote:
> Since this is lockless, the PTEs could change under
> us between when we walk them in folio_pte_batch_flags()
> and the batched try_grab_folio().
>
> Is that race condition safe, or do we need to re-check
> afterward that:
> 1) the ptes still point at the same folio, and/or
> 2) the folio still spans the entire range, and was
>    not split before we grabbed the refcount?

It is safe, because the batch only runs after the first page of the
run has gone through the unchanged per-page path: the folio reference
is already held (and for FOLL_PIN the _pincount already raised, with
the smp_mb__after_atomic() in try_grab_folio_fast()) before any of
the other ptes is read.

2) With that reference held the folio cannot be split: both
__folio_split() and folio_split_unmapped() have to folio_ref_freeze()
the folio to the expected count and back off with -EAGAIN otherwise.
So folio_nr_pages() and page + i are stable for the scan.

1) The per-page path re-checks the pte because it has to pin first
and look at the pte second ("(1) pin, then (2) check pte" in the
comment above gup_fast_pte_range()).  For the rest of the run that is
already the order: each pte is read while the folio is held, so
seeing it map page + i is the same state the per-page path is in
right after its re-check.  A pte changing after that is the race
GUP-fast accepts anyway: the other side clears the pte and then
checks for a pin or an extra reference, both of which are per folio,
so it sees ours.  try_grab_folio() only adds references to a folio we
already hold and does not look at the ptes, so re-reading them after
it would not tell us anything new.

The pmd re-check after the grab is the batch's counterpart of the
per-page pmd check for THP collapse (pmdp_collapse_flush() detaching
the page table while it stays readable with interrupts off), same as
in gup_fast_pmd_leaf(): if the pmd changed, the extra references are
dropped and the next iteration bails out to the slow path.  The other
early returns in gup_fast_pte_batch() happen before any extra
reference is taken, so every bail-out leaves only the head page's
reference.

fork (write_protect_seq around the whole walk) and the read-only
FOLL_PIN check (gup_must_unshare() per page, after the pte read) are
unchanged from the per-page path.

Yuan-Hao Hsu