Re: [PATCH 2/8] iommu/vt-d: Add entry_sync support for PASID entry updates

[Baolu Lu]

On 3/23/26 21:07, Jason Gunthorpe wrote:
> On Sun, Mar 15, 2026 at 04:11:36PM +0800, Baolu Lu wrote:
> > > > > ATC invalidations should always be done after the PASID entry is
> > > > > written. During a hitless update both translations are unpredictably
> > > > > combined, this is unavoidable and OK.
> > > >
> > > > The VT-d spec (Sections 6.5.2.5 and 6.5.2.6) explicitly mandates that an
> > > > IOTLB invalidation must precede the Device-TLB invalidation. If we only
> > > > do the device-TLB invalidation in the sync callback, we risk the device
> > > > re-fetching a stale translation from the IOMMU's internal IOTLB.
> > >
> > > It is a little weird that is says that, that is worth checking into.
> > >
> > > The other text is clear that the IOTLB is cached by DID,PASID only, so
> > > if the new PASID entry has a DID,PASID which is already coherent in
> > > the IOTLB it should not need any IOTLB flushing.
> > >
> > > ie flushing the PASID table should immediately change any ATC fetches
> > > from using DID,old_PASID to DID,new_PASID.
> > >
> > > If there is some issue where the PASID flush doesn't fence everything
> > > (ie an ATC fetch of DID,old_PASID can be passed by an ATC invalidation)
> > > then you may need IOTLB invalidations not to manage coherence but to
> > > manage ordering. That is an important detail if true.
> >
> > On Intel hardware, the PASID-cache and IOTLB are not inclusive. A PASID-
> > cache invalidation forces a re-fetch of the pasid entry, but it does not
> > automatically purge downstream IOTLB entries.
>
> It doesn't matter, the updated PASID entry will point to a new DID and
> the IOTLB (new DID,PASID) entry will be valid in the IOTLB.
>
> We don't need to flush the IOTLB, we just need to ensure that all
> lookups done with (old DID,PASID) are completed before sending any
> invalidation.

Yes, you are right.

> > The spec-mandated IOTLB flush serves as a synchronization barrier to
> > ensure that in-flight translation requests are drained and the
> > internal IOMMU state is consistent before the invalidation request
> > is sent over PCIe to the device's ATC.
>
> A fencing requirement does make sense, but does it have to be done by
> flushing the entire DID,PASID? It is ugly to have to drop the IOTLB
> just because a context entry changed.

I believe the full [old_DID, PASID] invalidation is a functional
necessity rather than just a fencing requirement. Even though the new
PASID entry points to a new_DID, leaving stale translations tagged with
[old_DID, PASID] in the IOTLB is problematic.

However, I agree that IOTLB and Device-TLB invalidation should not be
part of the entry_sync for a PASID entry; instead, it belongs in the
domain replacement logic.

> Can you do a 4k IOVA 0 invalidation and get the same fence?
>
> Jason

Thanks,
baolu