[PATCH RFC 1/5] mm/khugepaged: fix filemap_release_folio() interaction with GUP dirty
From: Pedro Falcato
Date: Thu Nov 19 2026 - 15:11:37 EST
Whether a folio can be "released" (drop attached buffer_heads, iomap
state, etc) is a complicated matter.
Namely, the act of calling pin_user_pages()/get_user_pages(FOLL_WRITE)
and later calling folio_mark_dirty() on these same folios relies on
on these structures to have been set up by ->page_mkwrite() or
->write_begin() elsewhere. ->dirty_folio() implementations usually
cannot set these up themselves, as ->dirty_folio() is not allowed to
block.
[get_user_pages(), etc]
->page_mkwrite() called here, or before
->private correctly populated with whatever the filesystem
expects.
folio is marked as dirty, initially.
*write finishes*
folio_lock();
folio_mark_dirty();
->dirty_folio()
dirty_folio expects ->private to be populated by
->page_mkwrite(), ->write_begin().
folio_unlock();
So, what happens if filemap_release_folio() is randomly called on
folios?
T0 | T1
[get_user_pages(), etc]
->page_mkwrite() called here
->private is ok
folio is marked as dirty, initially.
*write finishes*
| *some op*
| filemap_release_folio()
folio_lock();
folio_mark_dirty();
->dirty_folio()
dirty_folio expects ->private
folio_unlock();
In, e.g. ext4's case, this results in a WARN_ON(!folio_buffers(folio))
and very likely data loss, as block_dirty_folio() will not find any
buffers.
Thus, introduce a new helper to help figure out if
filemap_release_folio() can be called. filemap_may_release_folio()
mainly takes into account the refcount to help figure out if it's safe
to release. The NFS ->private refcount funkyness will be handled in a
separate patch.
As a part of this change, move try_to_unmap() back to before calling
filemap_may_release_folio(). This is required for mappings which don't
require write notification, and as such the dirty=1 -> folio dirty=1
transition is actually meaningful.
Reported-by: David Hildenbrand <david@xxxxxxxxxx>
Cc: stable@xxxxxxxxxxxxxxx
Fixes: d824ec2a1546 ("mm: do not reclaim private data from pinned page")
Signed-off-by: Pedro Falcato <pfalcato@xxxxxxx>
---
mm/internal.h | 48 ++++++++++++++++++++++++++++++++++++++++++++++++
mm/khugepaged.c | 19 +++++++++++++------
2 files changed, 61 insertions(+), 6 deletions(-)
diff --git a/mm/internal.h b/mm/internal.h
index 85cf7becf858..4c22d27ace96 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -718,6 +718,54 @@ static inline bool folio_needs_release(struct folio *folio)
(mapping && mapping_release_always(mapping));
}
+static inline bool filemap_may_release_folio(struct folio *folio, int extra_refs)
+{
+ /*
+ * Whether we may "release a folio" (drop attached buffers,
+ * etc) is a bit more complex than what you could be hoping for.
+ * There are two types of dirtying users in the kernel:
+ * 1) ones which take the regular page fault/write(2) paths.
+ * These regular paths (which fall into ->page_mkwrite or ->write_begin)
+ * take enough care to e.g establish buffers and attach them to the folio.
+ *
+ * 2) ones that don't necessarily do that.
+ * Essentially, the act of writing to a pinned/GUP'd page relies on the
+ * above (for GUP-like operations, ->page_mkwrite). However, the folio
+ * does not stay locked all the time. It is released somewhere in between
+ * get_user_pages() and whatever it is you're doing with the page (e.g IO
+ * submission). These users rely on the following logic:
+ *
+ * [get_user_pages(), etc]
+ * ->page_mkwrite() called here, or before
+ * ->private correctly populated with whatever the filesystem
+ * expects.
+ * folio is marked as dirty, initially.
+ * *write finishes*
+ * folio_lock();
+ * folio_mark_dirty();
+ * ->dirty_folio()
+ * dirty_folio expects ->private to be populated by
+ * ->page_mkwrite(), ->write_begin().
+ * folio_unlock();
+ *
+ * Thus, it's not possible to call filemap_release_folio() if _anyone_
+ * has a reference we don't expect, that can plausibly dirty the folio.
+ * Additionally, we can't have mappings for these. These checks are
+ * racy, but the folio lock guarantees that whomever is e.g starting IO
+ * on these pages will inevitably wait for us to finish.
+ */
+ VM_WARN_ON_FOLIO(!folio_test_locked(folio), folio);
+ if (!folio_needs_release(folio))
+ return true;
+
+ /* There's no way to know whether these are write-mapped, so give up. */
+ if (folio_mapped(folio))
+ return false;
+ if (folio_ref_count(folio) != extra_refs + folio_expected_ref_count(folio))
+ return false;
+ return true;
+}
+
extern unsigned long highest_memmap_pfn;
/*
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 964f78c3856b..1f1b93371a7e 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -2396,6 +2396,15 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
goto out_unlock;
}
+ if (!folio_isolate_lru(folio)) {
+ result = SCAN_DEL_PAGE_LRU;
+ goto out_unlock;
+ }
+
+ if (folio_mapped(folio))
+ try_to_unmap(folio,
+ TTU_IGNORE_MLOCK | TTU_BATCH_FLUSH);
+
if (!is_shmem && (folio_test_dirty(folio) ||
folio_test_writeback(folio))) {
/*
@@ -2404,11 +2413,13 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
* since first write.
*/
result = SCAN_PAGE_DIRTY_OR_WRITEBACK;
+ folio_putback_lru(folio);
goto out_unlock;
}
- if (!folio_isolate_lru(folio)) {
- result = SCAN_DEL_PAGE_LRU;
+ if (!filemap_may_release_folio(folio, 2)) {
+ result = SCAN_PAGE_COUNT;
+ folio_putback_lru(folio);
goto out_unlock;
}
@@ -2418,10 +2429,6 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,
goto out_unlock;
}
- if (folio_mapped(folio))
- try_to_unmap(folio,
- TTU_IGNORE_MLOCK | TTU_BATCH_FLUSH);
-
xas_lock_irq(&xas);
VM_BUG_ON_FOLIO(folio != xa_load(xas.xa, index), folio);
--
2.56.1