[PATCH 3/4] RFC: mm/rmap: don't defer TLB flushes when unmapping lazyfree folios

From: David Hildenbrand (Arm)

Date: Wed Nov 18 2026 - 04:21:48 EST


When reclaim discards a lazyfree folio, it clears the PTE but delays the
TLB flush. Until flushed, mapping a new folio into the now-none PTE
can leave some CPUs still reading from the old folio while others
already write to the new folio.

This is hard to trigger in practice and would mostly look like a weird
use-after-free bug in an allocator that uses MADV_FREE, but it can be
reproduced reliably with three threads pinned to different CPUs:

(1) Thread A populates the pages, MADV_FREE's them and then
MADV_PAGEOUT's them.
(2) Thread B reads all pages to keep TLB entries cached and verifies the
content of pages published by thread C.
(3) Thread C waits until mincore() indicates that a page is no longer
present, writes a new value to it, and publishes it to thread B.

Thread B can then read the old value after the new folio was mapped and
the content updated.

One way to handle it would be adding conditional flushes to
do_anonymous_page(), migrate_vma_insert_page(), UFFDIO_COPY and
UFFDIO_MOVE ... essentially anywhere we install new anonymous folios,
which would mean that these places flush quite frequently on concurrent
reclaim.

To perform targeted flushes, we'd have to keep some PTE markers behind,
which is also rather nasty and not an easy fix.

So let's not defer the TLB flush for lazyfree folios, which includes
folios in VM_DROPPABLE mappings.

To avoid flushing if the folio was redirtied or has additional
references, perform all non-deferred TLB flushes only once we know that
we won't restore the PTEs, right before removing the rmap.

The downside is that we no longer batch TLB flushes when discarding
lazyfree folios, which harms performance, but is the right thing to do
for now.

Fixes: 854e9ed09ded ("mm: support madvise(MADV_FREE)")
Cc: stable@xxxxxxxxxxxxxxx
Signed-off-by: David Hildenbrand (Arm) <david@xxxxxxxxxx>
---
mm/rmap.c | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/mm/rmap.c b/mm/rmap.c
index 09c33b9ec5850..3935eed23a361 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -2228,6 +2228,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,
struct mmu_notifier_range range;
enum ttu_flags flags = (enum ttu_flags)(long)arg;
unsigned long nr_pages = 1, end_addr;
+ bool flush;
unsigned long pfn;
int ptes = 0;

@@ -2253,6 +2254,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,

while (page_vma_mapped_walk(&pvmw)) {
nr_pages = 1;
+ flush = false;

/*
* If the folio is in an mlock()d vma, we must not swap it out.
@@ -2339,11 +2341,18 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,
* architecture must guarantee that a clear->dirty
* transition on a cached TLB entry is written through
* and traps if the PTE is unmapped.
+ *
+ * If we don't defer the flush, we flush only once we
+ * know that we won't restore the PTEs below. We never
+ * defer the flush for lazyfree folios: if we discard
+ * them, the PTE will be none, and a page fault could
+ * map a new folio while stale TLB entries still exist.
*/
- if (should_defer_flush(mm, flags))
+ if (!folio_test_lazyfree(folio) &&
+ should_defer_flush(mm, flags))
set_tlb_ubc_flush_pending(mm, pteval, address, end_addr);
else
- flush_tlb_range(vma, address, end_addr);
+ flush = true;
if (pte_dirty(pteval))
folio_mark_dirty(folio);
} else {
@@ -2402,6 +2411,8 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma,
add_mm_counter(mm, mm_counter_file(folio), -nr_pages);
}
finish_unmap:
+ if (flush)
+ flush_tlb_range(vma, address, end_addr);
folio_remove_rmap_ptes(folio, page, nr_pages, vma);
if (vma_test_any_mask(vma, VMA_LOCKED_MASK))
mlock_drain_local();

--
2.56.1