Re: [PATCH v3 28/33] drm/xe: Use IOVA-based DMA mapping for eligible tt BOs

[Matthew Brost]

On Wed, Nov 04, 2026 at 11:39:30AM +0000, Matthew Auld wrote:
> On 03/11/2026 20:21, Matthew Brost wrote:
> > On Tue, Nov 03, 2026 at 07:00:24PM +0000, Matthew Auld wrote:
> > > On 11/07/2026 03:56, Matthew Brost wrote:
> > > > Add support for mapping a ttm_tt's pages using the IOVA-based DMA API
> > > > (dma_iova_try_alloc()/link()/sync()/unlink()/free()) as an alternative
> > > > to the scatter-gather list. An IOVA mapping is a single contiguous DMA
> > > > range, which is cheaper to walk and program than an sg list.
> > > > 
> > > > Whether the IOVA API can be used depends on the IOMMU mode, so the
> > > > decision is made once per tt at xe_ttm_tt_create() time and recorded in
> > > > struct xe_ttm_tt::use_iova. To keep the scope limited, only device BOs
> > > > that are not GGTT-mapped are eligible: dma-buf imports manage their own
> > > > mapping and GGTT bindings are programmed directly from the sg list.
> > > > dma_iova_try_alloc() may still fail, in which case the tt transparently
> > > > falls back to the sg list, which remains the default.
> > > > 
> > > > The IOVA reservation is allocated at tt creation and freed at tt
> > > > destroy. xe_tt_map_iova() links the tt pages (coalescing physically
> > > > contiguous runs) and syncs the IOTLB; xe_tt_unmap_iova() unlinks them
> > > > but keeps the reservation so the tt can be remapped without
> > > > re-allocating IOVA space. xe_tt_map()/xe_tt_unmap() dispatch between the
> > > > IOVA and sg backends, and xe_bo_dma_unmap_pinned() learns about the IOVA
> > > > case.
> > > > 
> > > > Consumers that walked the sg list via xe_res_first_sg(xe_bo_sg()) now go
> > > > through the new xe_res_first_tt() helper, which initializes either an
> > > > IOVA cursor (xe_res_first_iova()) or an sg cursor depending on the tt's
> > > > mapping mode.
> > > > 
> > > > Cc: Carlos Santa <carlos.santa@xxxxxxxxx>
> > > > Cc: Ryan Neph <ryanneph@xxxxxxxxxx>
> > > > Cc: Christian Koenig <christian.koenig@xxxxxxx>
> > > > Cc: Huang Rui <ray.huang@xxxxxxx>
> > > > Cc: Matthew Auld <matthew.auld@xxxxxxxxx>
> > > > Cc: Maarten Lankhorst <maarten.lankhorst@xxxxxxxxxxxxxxx>
> > > > Cc: Maxime Ripard <mripard@xxxxxxxxxx>
> > > > Cc: Thomas Zimmermann <tzimmermann@xxxxxxx>
> > > > Cc: David Airlie <airlied@xxxxxxxxx>
> > > > Cc: Simona Vetter <simona@xxxxxxxx>
> > > > Cc: dri-devel@xxxxxxxxxxxxxxxxxxxxx
> > > > Cc: linux-kernel@xxxxxxxxxxxxxxx
> > > > Cc: Thomas Hellström <thomas.hellstrom@xxxxxxxxxxxxxxx>
> > > > Assisted-by: GitHub_Copilot:claude-opus-4.8
> > > > Signed-off-by: Matthew Brost <matthew.brost@xxxxxxxxx>
> > > 
> > > 
> > > <snip>
> > > 
> > > >    static void xe_ttm_tt_destroy(struct ttm_device *ttm_dev, struct ttm_tt *tt)
> > > >    {
> > > > +	struct xe_ttm_tt *xe_tt = container_of(tt, struct xe_ttm_tt, ttm);
> > > > +
> > > > +	/*
> > > > +	 * Release the IOVA reservation made in xe_ttm_tt_create(). The pages
> > > > +	 * must already have been unlinked by xe_tt_unmap().
> > > > +	 */
> > > > +	if (xe_tt->use_iova) {
> > > > +		struct xe_device *xe = ttm_to_xe_device(ttm_dev);
> > > > +
> > > > +		xe_assert(xe, !xe_tt->iova_linked);
> > > > +		dma_iova_free(xe->drm.dev, &xe_tt->iova);
> > > 
> > > Do we know if this is allowed? I think there is xe_bo_pci_dev_remove_all()
> > > which requires that evict_all will correctly tear down iommu mappings on
> > > stuff like pci unplug and I don't think tt_destroy is called in that flow?
> > 
> > But unpopulate is called which unlinks / syncs (TLB flush for the IOVA).
> > 
> > > Does having the iova here cause issues? Also, for VRAM flow I think there is
> > > a tt_create() via pipeline gutting which will now create a new iova?
> > > 
> > 
> > Maybe we shuffle this around a bit so the iova alloc is tied to
> > populate/unpopulate. It was been a while I forget exactly why I did this
> > way.
> > 
> > > Aside from that, maybe we can land 27/28 as standalone?
> > 
> > Yes, let me extract these two into standalone patches today and
> > reworking if needed.
> 
> Quickly double checked this, indeed looks to be needed:
> 

Thanks was able recreate this + verify a local fix.

27/28 has one TTM dependency [1] and also is staged behind dma mapping
accounting [2]. So maybe once [2] lands an updated 27/28 can get posted
as standalone.

Matt

[1] https://patchwork.freedesktop.org/patch/739339/?series=170216&rev=2
[2] https://patchwork.freedesktop.org/series/177790/

> [  808.058892] [IGT] core_hotunplug: finished subtest
> hotreplug-lateclose-with-load, SUCCESS
> [  808.058928] [IGT] core_hotunplug: exiting, ret=0
> [  808.059673] BUG: kernel NULL pointer dereference, address:
> [  808.061068] RIP: 0010:iommu_get_dma_domain+0x10/0x20
> [  808.061141] PKRU: 55555554
> [  808.061144] Call Trace:
> [  808.061146]  <TASK>
> [  808.061149]  dma_iova_free+0x18/0xf0
> [  808.061153]  xe_ttm_tt_destroy+0x23/0x40 [xe]
> [  808.061231]  ttm_bo_put+0x242/0x3d0 [ttm]
> [  808.061412]  ? drm_gpuvm_bo_put+0x199/0x200 [drm_gpuvm]
> [  808.061418]  xe_vma_destroy_late+0x89/0x90 [xe]
> [  808.061476]  xe_vm_close_and_put+0x5de/0x920 [xe]
> [  808.061534]  xe_file_close+0x105/0x190 [xe]
> [  808.061712]  drm_file_free+0x218/0x260
> [  808.061717]  drm_release_noglobal+0x6d/0xd0
> [  808.061721]  __fput+0xf9/0x270
> [  808.061725]  task_work_run+0x8b/0xb0
> [  808.061730]  do_exit+0x229/0x8b0
> [  808.061735]  do_group_exit+0x8b/0x90
> [  808.061739]  __x64_sys_exit_group+0x17/0x20
> [  808.061744]  x64_sys_call+0x18c6/0x18d0
> [  808.061749]  do_syscall_64+0xee/0x350
> [  808.061753]  ? __pte_offset_map+0x1d/0xd0
> [  808.061757]  ? next_uptodate_folio+0xbb/0x3a0
> [  808.061762]  ? set_pte_range+0x20d/0x290
> [  808.061766]  ? filemap_map_pages+0x8e1/0x9c0
> [  808.061772]  ? count_memcg_events+0x7c/0x1b0
> [  808.061776]  ? handle_mm_fault+0xbde/0x14f0
> [  808.061781]  ? __se_sys_close+0x7a/0xe0
> [  808.061785]  ? __se_sys_close+0x7a/0xe0
> [  808.061790]  ? do_user_addr_fault+0x272/0x6c0
> [  808.061794]  ? irqentry_exit+0x3f/0x640
> [  808.061799]  ? do_syscall_64+0x42/0x350
> [  808.061803]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
> [  808.061808] RIP: 0033:0x7f95428c71b8
> 
> I assume this is dev->iommu_group = NULL post unplug.
> 
> > 
> > Matt
>