[RFC PATCH 20/57] mm/collapse: put the sources back

[Kiryl Shutsemau]

From: "Kiryl Shutsemau (Meta)" <kas@xxxxxxxxxx>

Fill in the putback: for every installed candidate, lower the barriers
the freeze raised, span by span.

This is also what wakes the faulters the collapse held up.  They sleep on
a source folio's lock; once it is dropped they refault and find present
PTEs pointing at the new folio.

The order within a span is important:

 - Unfreeze first.  Rmap removal munlocks under VM_LOCKED, and
   munlock_folio() takes a reference a frozen folio forbids.

 - Then drop the rmap.  Until it is gone the expected count still holds
   the span's mapping references; afterwards they belong to the round, so
   every folio_remove_rmap_ptes() is paired with a folio_put_refs() for
   the same slots.

 - Then unlock, which is the wake.  Holding the lock until here keeps
   lock-taking rmap walkers out, and the window it leaves -- a live folio
   with no PTEs -- is one any teardown of a mapped folio passes through.

 - Drop the references strictly last, the round's included.  Waiters wait
   without a reference of their own, so the round's has to outlive the
   unlock.

The stale swapcache entry goes too: the copy has replaced what it
described.

Assisted-by: Claude-Code:claude-opus-5
Signed-off-by: Kiryl Shutsemau (Meta) <kas@xxxxxxxxxx>
---
 mm/collapse.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 50 insertions(+)

diff --git a/mm/collapse.c b/mm/collapse.c
index ab7476471b8d..f65f413339bf 100644
--- a/mm/collapse.c
+++ b/mm/collapse.c
@@ -1439,6 +1439,56 @@ static void collapse_install(struct vm_area_struct *vma,
 static void collapse_putback(struct vm_area_struct *vma,
 			     struct collapse_control *cc)
 {
+	unsigned int i;
+
+	for (i = 0; i < cc->nr_candidates; i++) {
+		struct collapse_candidate *cand = &cc->candidates[i];
+		const unsigned int nr_pages = candidate_nr_pages(cand);
+		unsigned int k = 0;
+
+		if (cand->state != CAND_INSTALLED)
+			continue;
+
+		while (k < nr_pages) {
+			struct folio *folio;
+			unsigned int nr;
+
+			/* A slot with no source has nothing to put back */
+			if (pte_none_or_zero(cand->saved_ptes[k])) {
+				k++;
+				continue;
+			}
+
+			folio = pte_folio(cand->saved_ptes[k]);
+			nr = collapse_saved_span_len(cand, k, nr_pages);
+
+			/*
+			 * Unfreeze before the rmap drop: rmap removal munlocks
+			 * under VM_LOCKED, and munlock_folio() takes a reference
+			 * a frozen folio forbids.  The expected count still
+			 * holds the span's mapping references; once the rmap is
+			 * gone they are ours to drop, so every
+			 * folio_remove_rmap_ptes() is paired with a
+			 * folio_put_refs() for the same slots.  The folio lock
+			 * is held until the wake below, so lock-taking rmap
+			 * walkers stay excluded, and the stale-rmap window this
+			 * leaves -- live folio, no PTEs -- is one any teardown of
+			 * a mapped folio passes through.
+			 */
+			folio_ref_unfreeze(folio,
+					   folio_expected_ref_count(folio) + 1);
+			folio_remove_rmap_ptes(folio,
+					       pte_page(cand->saved_ptes[k]),
+					       nr, vma);
+			folio_unlock(folio);
+
+			/* The copy replaced it; drop the stale swap entry */
+			free_swap_cache(folio);
+			folio_put_refs(folio, nr + 1);
+
+			k += nr;
+		}
+	}
 }
 
 /*
-- 
2.54.0