Re: [PATCH 1/4] mm: flush stale TLB entries before installing a CoW'ed folio

From: Zi Yan

Date: Thu Nov 19 2026 - 12:04:25 EST


On 19 Nov 2026, at 11:49, David Hildenbrand (Arm) wrote:

> On 11/19/26 17:47, Zi Yan wrote:
>> On 18 Nov 2026, at 4:20, David Hildenbrand (Arm) wrote:
>>
>>> Zapping PTEs in zap_pte_range() defers the TLB flush until after dropping
>>> the PTL for clean page-cache folios.
>>>
>>> A concurrent write fault in a MAP_PRIVATE mapping can then CoW the
>>> page-cache folio and install a new anon folio without a prior TLB
>>> flush:
>>>
>>> CPU0 CPU1
>>> LOAD(ADDR)
>>> -> #PF
>>> installs R/O PTE -> A
>>> fuse_open()
>>> -> invalidate_inode_pages2()
>>> -> zap_pte_range()
>>> clears R/O PTE
>>> // delayed flush
>>> // Other CPUs (incl CPU0) still cache ADDR -> A
>>> STORE(ADDR)
>>> -> #PF
>>> do_cow_fault()
>>> copies A to B, installs R/W PTE -> B
>>> // no TLB flush: PTE was none
>>>
>>> CPUs with stale TLB entries will still read from the old folio while writes
>>> go to the new folio, until the TLB is flushed.
>>>
>>> We really must process pending TLB flushes if we are not faulting the
>>> same folio back in.
>>>
>>> Fix it by flushing the TLB before installing an anon (CoW'ed) folio in
>>> set_pte_range() if a TLB flush is pending. Zapping itself stays as fast
>>> as before.
>>>
>>> Other faults are not affected: unmap_mapping_*() users only remove
>>> folios from the page-cache after their own TLB flush, so until then, a
>>> refault maps the same folio again. The exception is a concurrent
>>> invalidation that removes the already-unmapped folio from the
>>> page-cache before the zapper flushed the TLB; that is an unlikely corner
>>> case that we'll not handle for now.
>>>
>>> Note that we don't care about MADV_DONTNEED followed by a new read
>>> fault, as MADV_DONTNEED only guarantees its semantics once the operation
>>> completed. CoW faults will flush the TLB either way.
>>>
>>> For example, imagine that MADV_DONTNEED zaps a page-cache folio and
>>> doesn't flush the TLB (but still holds a reference). If the page-cache
>>> replaces the folio, a new read fault might map the new folio without
>>> flushing the TLB. That's okay, as MADV_DONTNEED did not complete yet.
>>> Same applies to MADV_DONTNEED of CoW'ed folios when re-faulting the
>>> page-cache folio.
>>>
>>> Memory reclaim, which removes folios from the page-cache before performing
>>> its own batched TLB flush, will be handled separately.
>>>
>>> Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
>>
>> This is really old.
>>
>>> Reported-by: Cenk Yılmaz <cenk@xxxxxxxxxxxxxxxxxxx>
>>> Debugged-by: Cenk Yılmaz <cenk@xxxxxxxxxxxxxxxxxxx>
>>> Cc: stable@xxxxxxxxxxxxxxx
>>> Signed-off-by: David Hildenbrand (Arm) <david@xxxxxxxxxx>
>>> ---
>>> mm/memory.c | 6 ++++++
>>> 1 file changed, 6 insertions(+)
>>>
>>> diff --git a/mm/memory.c b/mm/memory.c
>>> index d2f48d78b0d7b..e1d9759e6fde1 100644
>>> --- a/mm/memory.c
>>> +++ b/mm/memory.c
>>> @@ -5888,6 +5888,12 @@ void set_pte_range(struct vm_fault *vmf, struct folio *folio,
>>> /* copy-on-write page */
>>> if (write && !(vma->vm_flags & VM_SHARED)) {
>>> VM_BUG_ON_FOLIO(nr != 1, folio);
>>> + /*
>>> + * Concurrent zapping might have dropped the PTL but not flushed
>>> + * the TLB yet. Flush before installing a new folio.
>>> + */
>>> + if (mm_tlb_flush_pending(vma->vm_mm))
>>> + flush_tlb_page(vma, addr);
>>
>> This looks like an adhoc fix, but it is minimal for backport.
>> Hopefully we can find a cleaner refactoring later.
>
> I'm afraid if we don't want to throw away performance there is no way around it.
>
> We have similar handling on the uffd-wp path ...

Performance or our sanity. :)

Anyway, this is for a later discussion.

Best Regards,
Yan, Zi