[PATCH 1/3] mm/huge_memory: add folio_split_driver_managed()

[Matthew Brost]

Add a lightweight structural split primitive for large (compound) folios
that a driver allocated with __GFP_COMP and manages entirely by itself,
outside of the core mm's view.

The existing split paths - split_folio() and folio_split_unmapped() -
are built for folios that the mm owns: they perform a refcount freeze,
walk and remap the rmap, and take the anon_vma / i_mmap locks, and
folio_split_unmapped() further assumes an anon, pagecache-style refcount
model (nr_pages + 1). None of that applies to a folio that is:

  - singly referenced (the caller holds the only reference),
  - not mapped through the rmap (folio_mapped() == 0),
  - not in the page cache or swap cache (folio->mapping == NULL),
  - not on any LRU or the deferred-split list.

For such a folio the split is purely structural: because nothing else in
the kernel can reach it, there is no need to freeze the refcount or touch
any mapping. folio_split_driver_managed() therefore performs only the
compound and split-accounting teardown via __split_unmapped_folio() and
then hands each resulting order-@new_order folio its own reference,
mirroring split_page() for compound folios. The caller keeps the original
reference on the first resulting folio and is responsible for freeing all
of them individually.

The immediate user is TTM's GPU page pool, which allocates higher-order
compound pages, maps them into userspace via VM_PFNMAP (never through the
rmap), and needs to split them into order-0 folios under memory pressure
so pages can be backed up to shmem and freed one at a time.

A CONFIG_TRANSPARENT_HUGEPAGE=n stub is provided so callers can build
without the split machinery; it warns and returns -EINVAL.

Cc: Maarten Lankhorst <maarten.lankhorst@xxxxxxxxxxxxxxx>
Cc: Maxime Ripard <mripard@xxxxxxxxxx>
Cc: Thomas Zimmermann <tzimmermann@xxxxxxx>
Cc: David Airlie <airlied@xxxxxxxxx>
Cc: Simona Vetter <simona@xxxxxxxx>
Cc: Christian Koenig <christian.koenig@xxxxxxx>
Cc: Huang Rui <ray.huang@xxxxxxx>
Cc: Matthew Auld <matthew.auld@xxxxxxxxx>
Cc: Matthew Brost <matthew.brost@xxxxxxxxx>
Cc: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
Cc: David Hildenbrand <david@xxxxxxxxxx>
Cc: Lorenzo Stoakes <ljs@xxxxxxxxxx>
Cc: Zi Yan <ziy@xxxxxxxxxx>
Cc: Baolin Wang <baolin.wang@xxxxxxxxxxxxxxxxx>
Cc: "Liam R. Howlett" <liam@xxxxxxxxxxxxx>
Cc: Nico Pache <npache@xxxxxxxxxx>
Cc: Ryan Roberts <ryan.roberts@xxxxxxx>
Cc: Dev Jain <dev.jain@xxxxxxx>
Cc: Barry Song <baohua@xxxxxxxxxx>
Cc: Lance Yang <lance.yang@xxxxxxxxx>
Cc: Tvrtko Ursulin <tvrtko.ursulin@xxxxxxxxxx>
Cc: Dave Airlie <airlied@xxxxxxxxxx>
Cc: dri-devel@xxxxxxxxxxxxxxxxxxxxx
Cc: linux-kernel@xxxxxxxxxxxxxxx
Cc: linux-mm@xxxxxxxxx
Suggested-by: Matthew Wilcox <willy@xxxxxxxxxxxxx>
Signed-off-by: Matthew Brost <matthew.brost@xxxxxxxxx>
Assisted-by: GitHub-Copilot:claude-opus-4.8

---

The patch is based on drm-tip rather than the core MM branches to
facilitate Intel CI testing and initial review. It can be rebased onto
the core MM branches in a subsequent revision.
---
 include/linux/huge_mm.h |  8 ++++++
 mm/huge_memory.c        | 63 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 71 insertions(+)

diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
index ad20f7f8c179..35661d82d54a 100644
--- a/include/linux/huge_mm.h
+++ b/include/linux/huge_mm.h
@@ -402,6 +402,7 @@ enum split_type {
 int __split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
 		unsigned int new_order);
 int folio_split_unmapped(struct folio *folio, unsigned int new_order);
+int folio_split_driver_managed(struct folio *folio, unsigned int new_order);
 unsigned int min_order_for_split(struct folio *folio);
 int split_folio_to_list(struct folio *folio, struct list_head *list);
 int folio_check_splittable(struct folio *folio, unsigned int new_order,
@@ -656,6 +657,13 @@ static inline int split_folio_to_list(struct folio *folio, struct list_head *lis
 	return -EINVAL;
 }
 
+static inline int folio_split_driver_managed(struct folio *folio,
+					     unsigned int new_order)
+{
+	VM_WARN_ON_ONCE_FOLIO(1, folio);
+	return -EINVAL;
+}
+
 static inline int folio_split(struct folio *folio, unsigned int new_order,
 		struct page *page, struct list_head *list)
 {
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 2bccb0a53a0a..06f9a5f35df8 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -4185,6 +4185,69 @@ int folio_split_unmapped(struct folio *folio, unsigned int new_order)
 	return ret;
 }
 
+/**
+ * folio_split_driver_managed() - split an exclusively-owned, off-LRU folio
+ * @folio: folio to split. Must be a large (compound) folio that is owned
+ *         exclusively by the caller and is invisible to the core mm.
+ * @new_order: the order of the folios after the split.
+ *
+ * This is a lightweight structural split for folios that a driver allocated
+ * and manages itself (for example TTM's GPU page pool, which allocates
+ * higher-order compound pages with __GFP_COMP and maps them into userspace
+ * via VM_PFNMAP rather than through the rmap). Such folios are:
+ *
+ *   - singly referenced (the caller holds the only reference),
+ *   - not mapped through the rmap (folio_mapcount() == 0),
+ *   - not in the page cache or swap cache (folio->mapping == NULL),
+ *   - not on any LRU or the deferred-split list.
+ *
+ * Because nothing else in the kernel can reach the folio, this helper does
+ * not perform the refcount freeze / remap / anon_vma & i_mmap locking dance
+ * that split_folio() and folio_split_unmapped() require. It performs only
+ * the compound and split-accounting teardown and then hands each resulting
+ * folio its own reference, mirroring split_page() for compound folios.
+ *
+ * The caller is responsible for freeing the resulting folios individually.
+ *
+ * Context: caller holds the only reference and excludes concurrent access.
+ * Does not sleep.
+ *
+ * Return: 0 on success, negative errno on failure.
+ */
+int folio_split_driver_managed(struct folio *folio, unsigned int new_order)
+{
+	unsigned int old_order = folio_order(folio);
+	unsigned int split_nr = 1U << new_order;
+	unsigned int nr = 1U << old_order;
+	unsigned int i;
+
+	if (new_order >= old_order)
+		return -EINVAL;
+
+	VM_WARN_ON_ONCE_FOLIO(folio_ref_count(folio) != 1, folio);
+	VM_WARN_ON_ONCE_FOLIO(folio_mapped(folio), folio);
+	VM_WARN_ON_ONCE_FOLIO(folio->mapping, folio);
+	VM_WARN_ON_ONCE_FOLIO(folio_test_swapcache(folio), folio);
+	VM_WARN_ON_ONCE_FOLIO(folio_test_lru(folio), folio);
+
+	/*
+	 * Structural + split-accounting teardown only. No mapping/xarray, no
+	 * refcount freeze: the folio is frozen-by-ownership already.
+	 */
+	__split_unmapped_folio(folio, new_order, &folio->page, NULL, NULL,
+			       SPLIT_TYPE_UNIFORM);
+
+	/*
+	 * Give every resulting head folio its own reference. The original
+	 * reference stays on the first one, exactly like split_page().
+	 */
+	for (i = split_nr; i < nr; i += split_nr)
+		set_page_refcounted(folio_page(folio, i));
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(folio_split_driver_managed);
+
 /*
  * This function splits a large folio into smaller folios of order @new_order.
  * @page can point to any page of the large folio to split. The split operation
-- 
2.34.1