Re: [PATCH v2 02/33] drm/ttm/pool: Add ttm_pool_page_order_nodma() helper

[Matthew Brost]

On Sat, Jul 11, 2026 at 12:39:38PM +0200, Christian König wrote:
> On 7/10/26 23:54, Matthew Brost wrote:
> > TTM records the allocation order of a multi-order page in page->private at
> > allocation time. For pools that do not use dma_alloc the order is stored
> > directly, while dma_alloc pools store a struct ttm_pool_dma pointer there
> > instead. Drivers that do their own DMA mapping (e.g. Xe) need the per-page
> > order to walk a populated ttm_tt one chunk at a time, but cannot rely on
> > folio_order(): TTM allocates high-order pages with alloc_pages_node()
> > without __GFP_COMP, so they are not compound and folio_order() always
> > returns 0.
> 
> Well what is the justification of XE doing the DMA mapping themselves?
> 

This maps to `TTM_ALLOCATION_POOL_USE_DMA_ALLOC`, which is clear in Xe, so
we use this helper function to determine the page order in various
places. I exported this because I'd want to have fish out the pool and
use ttm_pool_page_order.

That said, Xe also owns the DMA mappings. To be honest, I didn't realize
`ttm_sg_tt_init()` was commonly used, but having Xe manage DMA mappings
has several advantages (I believe we inherited this part from Faith /
Dave PoC work on Xe):

- Using an SG list or IOVA allocation gives smaller pages a contiguous
  address space, which can then be mapped using larger GPU pages.
- SG list or IOVA mapping requires only a single IOMMU sync for a
  collection of pages, whereas each `dma_map_page()` or `dma_unmap_page`
  call requires its own IOMMU synchronization.
- Using IOVA mapping (later in the series) allows individual page
  linkages to be replaced without modifying the IOVA programmed into the
  GPU page tables.

The second point is a fairly significant performance issue. I recently
switched GPUSVM over to `dma_iova_alloc()`, and if I recall correctly, a
chain of mappings with a single synchronization was roughly 100x faster
than individual `dma_map_page()` calls for a 2 MiB region backed by 4 KiB
pages.

It should not be quite as severe in TTM, since it generally does not use
lower-order pages, but mapping something like a 32 MiB BO will still be
significantly faster when using an SG list or IOVA allocation.

The TTM core should really be fixed in this area, or drivers should at
least be encouraged not to use this path, since implementing the mapping
on the driver side is fairly trivial.

Matt

> Regards,
> Christian.
> 
> > 
> > Expose ttm_pool_page_order_nodma(), a thin wrapper that returns the order
> > recorded in page->private. It is only valid for pages from a pool that does
> > not use dma_alloc; whether a TTM device uses dma_alloc is fixed at ttm
> > device init time, so callers know from their device configuration that this
> > helper applies. Use it from the existing internal ttm_pool_page_order() to
> > keep a single source of truth.
> > 
> > Cc: Carlos Santa <carlos.santa@xxxxxxxxx>
> > Cc: Ryan Neph <ryanneph@xxxxxxxxxx>
> > Cc: Christian Koenig <christian.koenig@xxxxxxx>
> > Cc: Huang Rui <ray.huang@xxxxxxx>
> > Cc: Matthew Auld <matthew.auld@xxxxxxxxx>
> > Cc: Maarten Lankhorst <maarten.lankhorst@xxxxxxxxxxxxxxx>
> > Cc: Maxime Ripard <mripard@xxxxxxxxxx>
> > Cc: Thomas Zimmermann <tzimmermann@xxxxxxx>
> > Cc: David Airlie <airlied@xxxxxxxxx>
> > Cc: Simona Vetter <simona@xxxxxxxx>
> > Cc: dri-devel@xxxxxxxxxxxxxxxxxxxxx
> > Cc: linux-kernel@xxxxxxxxxxxxxxx
> > Cc: Thomas Hellström <thomas.hellstrom@xxxxxxxxxxxxxxx>
> > Assisted-by: GitHub_Copilot:claude-opus-4.8
> > Signed-off-by: Matthew Brost <matthew.brost@xxxxxxxxx>
> > ---
> >  drivers/gpu/drm/ttm/ttm_pool.c | 25 ++++++++++++++++++++++++-
> >  include/drm/ttm/ttm_pool.h     |  2 ++
> >  2 files changed, 26 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/gpu/drm/ttm/ttm_pool.c b/drivers/gpu/drm/ttm/ttm_pool.c
> > index dbe977412a81..d34592d4dbc7 100644
> > --- a/drivers/gpu/drm/ttm/ttm_pool.c
> > +++ b/drivers/gpu/drm/ttm/ttm_pool.c
> > @@ -461,6 +461,29 @@ static unsigned int ttm_pool_shrink(int nid, unsigned long num_to_free)
> >  	return num_pages;
> >  }
> >  
> > +/**
> > + * ttm_pool_page_order_nodma() - Allocation order of a non-dma_alloc TTM page
> > + * @p: The page to query.
> > + *
> > + * Return the allocation order that the TTM pool allocator recorded in
> > + * @p->private at allocation time. This only works for pages that were
> > + * allocated from a pool which does *not* use dma_alloc (i.e. pages backed by
> > + * alloc_pages_node()), where TTM stores the order directly in page->private.
> > + *
> > + * For dma_alloc pools, page->private instead holds a struct ttm_pool_dma
> > + * pointer and this helper must not be used. Whether a TTM device uses
> > + * dma_alloc is fixed at ttm device init time, so callers are expected to know
> > + * from their TTM device configuration that their pages are not dma_alloc
> > + * backed before using this helper.
> > + *
> > + * Return: The allocation order of the page.
> > + */
> > +unsigned int ttm_pool_page_order_nodma(struct page *p)
> > +{
> > +	return p->private;
> > +}
> > +EXPORT_SYMBOL(ttm_pool_page_order_nodma);
> > +
> >  /* Return the allocation order based for a page */
> >  static unsigned int ttm_pool_page_order(struct ttm_pool *pool, struct page *p)
> >  {
> > @@ -470,7 +493,7 @@ static unsigned int ttm_pool_page_order(struct ttm_pool *pool, struct page *p)
> >  		return dma->vaddr & ~PAGE_MASK;
> >  	}
> >  
> > -	return p->private;
> > +	return ttm_pool_page_order_nodma(p);
> >  }
> >  
> >  /*
> > diff --git a/include/drm/ttm/ttm_pool.h b/include/drm/ttm/ttm_pool.h
> > index 26ee592e1994..753203980e2c 100644
> > --- a/include/drm/ttm/ttm_pool.h
> > +++ b/include/drm/ttm/ttm_pool.h
> > @@ -94,6 +94,8 @@ long ttm_pool_backup(struct ttm_pool *pool, struct ttm_tt *ttm,
> >  int ttm_pool_restore_and_alloc(struct ttm_pool *pool, struct ttm_tt *tt,
> >  			       const struct ttm_operation_ctx *ctx);
> >  
> > +unsigned int ttm_pool_page_order_nodma(struct page *p);
> > +
> >  int ttm_pool_mgr_init(unsigned long num_pages);
> >  void ttm_pool_mgr_fini(void);
> >  
>