Re: [PATCH 2/8] iommu/vt-d: Add entry_sync support for PASID entry updates

[Baolu Lu]

On 3/12/26 19:44, Jason Gunthorpe wrote:
> On Thu, Mar 12, 2026 at 03:50:03PM +0800, Baolu Lu wrote:
> > If I understand your remark correctly, the driver should only need the
> > following in the sync callback:
> >
> > - clflush (if non-coherent) to ensure the entry is in physical memory.
> > - PASID cache invalidation to force the hardware to re-read the entry.
>
> Yes
>
> > - Device-TLB invalidation to drop local device caches.
>
> I have prefered to keep this outside the entry_set system since it has
> nothing to do with updating the context entry.
>
> There should be only one ATS flush after the new entry is installed.

Okay, I will move the devtlb_invalidation_with_pasid() calls outside of
the entry_sync system, right after the call to the writer returns.

> > > ATC invalidations should always be done after the PASID entry is
> > > written. During a hitless update both translations are unpredictably
> > > combined, this is unavoidable and OK.
> >
> > The VT-d spec (Sections 6.5.2.5 and 6.5.2.6) explicitly mandates that an
> > IOTLB invalidation must precede the Device-TLB invalidation. If we only
> > do the device-TLB invalidation in the sync callback, we risk the device
> > re-fetching a stale translation from the IOMMU's internal IOTLB.
>
> It is a little weird that is says that, that is worth checking into.
>
> The other text is clear that the IOTLB is cached by DID,PASID only, so
> if the new PASID entry has a DID,PASID which is already coherent in
> the IOTLB it should not need any IOTLB flushing.
>
> ie flushing the PASID table should immediately change any ATC fetches
> from using DID,old_PASID to DID,new_PASID.
>
> If there is some issue where the PASID flush doesn't fence everything
> (ie an ATC fetch of DID,old_PASID can be passed by an ATC invalidation)
> then you may need IOTLB invalidations not to manage coherence but to
> manage ordering. That is an important detail if true.

On Intel hardware, the PASID-cache and IOTLB are not inclusive. A PASID-
cache invalidation forces a re-fetch of the pasid entry, but it does not
automatically purge downstream IOTLB entries. The spec-mandated IOTLB
flush serves as a synchronization barrier to ensure that in-flight
translation requests are drained and the internal IOMMU state is
consistent before the invalidation request is sent over PCIe to the
device's ATC.

Without this "IOTLB -> Wait Descriptor -> ATC" sequence, there is a risk
that the device re-populates its ATC from a stale entry still residing
in the IOMMU's internal IOTLB, even after the PASID entry itself has
been updated.

Thanks,
baolu