Re: [PATCH 1/4] mm: flush stale TLB entries before installing a CoW'ed folio

From: David Hildenbrand (Arm)

Date: Thu Nov 19 2026 - 11:49:38 EST


On 11/19/26 17:47, Zi Yan wrote:
> On 18 Nov 2026, at 4:20, David Hildenbrand (Arm) wrote:
>
>> Zapping PTEs in zap_pte_range() defers the TLB flush until after dropping
>> the PTL for clean page-cache folios.
>>
>> A concurrent write fault in a MAP_PRIVATE mapping can then CoW the
>> page-cache folio and install a new anon folio without a prior TLB
>> flush:
>>
>> CPU0 CPU1
>> LOAD(ADDR)
>> -> #PF
>> installs R/O PTE -> A
>> fuse_open()
>> -> invalidate_inode_pages2()
>> -> zap_pte_range()
>> clears R/O PTE
>> // delayed flush
>> // Other CPUs (incl CPU0) still cache ADDR -> A
>> STORE(ADDR)
>> -> #PF
>> do_cow_fault()
>> copies A to B, installs R/W PTE -> B
>> // no TLB flush: PTE was none
>>
>> CPUs with stale TLB entries will still read from the old folio while writes
>> go to the new folio, until the TLB is flushed.
>>
>> We really must process pending TLB flushes if we are not faulting the
>> same folio back in.
>>
>> Fix it by flushing the TLB before installing an anon (CoW'ed) folio in
>> set_pte_range() if a TLB flush is pending. Zapping itself stays as fast
>> as before.
>>
>> Other faults are not affected: unmap_mapping_*() users only remove
>> folios from the page-cache after their own TLB flush, so until then, a
>> refault maps the same folio again. The exception is a concurrent
>> invalidation that removes the already-unmapped folio from the
>> page-cache before the zapper flushed the TLB; that is an unlikely corner
>> case that we'll not handle for now.
>>
>> Note that we don't care about MADV_DONTNEED followed by a new read
>> fault, as MADV_DONTNEED only guarantees its semantics once the operation
>> completed. CoW faults will flush the TLB either way.
>>
>> For example, imagine that MADV_DONTNEED zaps a page-cache folio and
>> doesn't flush the TLB (but still holds a reference). If the page-cache
>> replaces the folio, a new read fault might map the new folio without
>> flushing the TLB. That's okay, as MADV_DONTNEED did not complete yet.
>> Same applies to MADV_DONTNEED of CoW'ed folios when re-faulting the
>> page-cache folio.
>>
>> Memory reclaim, which removes folios from the page-cache before performing
>> its own batched TLB flush, will be handled separately.
>>
>> Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
>
> This is really old.
>
>> Reported-by: Cenk Yılmaz <cenk@xxxxxxxxxxxxxxxxxxx>
>> Debugged-by: Cenk Yılmaz <cenk@xxxxxxxxxxxxxxxxxxx>
>> Cc: stable@xxxxxxxxxxxxxxx
>> Signed-off-by: David Hildenbrand (Arm) <david@xxxxxxxxxx>
>> ---
>> mm/memory.c | 6 ++++++
>> 1 file changed, 6 insertions(+)
>>
>> diff --git a/mm/memory.c b/mm/memory.c
>> index d2f48d78b0d7b..e1d9759e6fde1 100644
>> --- a/mm/memory.c
>> +++ b/mm/memory.c
>> @@ -5888,6 +5888,12 @@ void set_pte_range(struct vm_fault *vmf, struct folio *folio,
>> /* copy-on-write page */
>> if (write && !(vma->vm_flags & VM_SHARED)) {
>> VM_BUG_ON_FOLIO(nr != 1, folio);
>> + /*
>> + * Concurrent zapping might have dropped the PTL but not flushed
>> + * the TLB yet. Flush before installing a new folio.
>> + */
>> + if (mm_tlb_flush_pending(vma->vm_mm))
>> + flush_tlb_page(vma, addr);
>
> This looks like an adhoc fix, but it is minimal for backport.
> Hopefully we can find a cleaner refactoring later.

I'm afraid if we don't want to throw away performance there is no way around it.

We have similar handling on the uffd-wp path ...

--
Cheers,

David