Re: [PATCH 1/1] iommu/vt-d: Fix missed device TLB cache tag

From: Baolu Lu
Date: Wed Jun 19 2024 - 20:52:51 EST


On 6/20/24 12:46 AM, Jason Gunthorpe wrote:
On Wed, Jun 19, 2024 at 09:53:45AM +0800, Lu Baolu wrote:
When a domain is attached to a device, the required cache tags are
assigned to the domain so that the related caches could be flushed
whenever it is needed. The device TLB cache tag is created selectively
by checking the ats_enabled field of the device's iommu data. This
creates an ordered dependency between attach and ATS enabling paths.

The device TLB cache tag will not be created if device's ATS is enabled
after the domain attachment. This causes some devices, for example
intel_vpu, to malfunction.
What? How is this even possible?

ATS is controlled exclusively by the iommu driver, how can it be
changed without the driver knowing??

Yes. ATS is currently controlled exclusively by the iommu driver. The
intel iommu driver enables PCI/ATS on the probe path after the default
domain is attached. That means when the default domain is attached to
the device, the ats_supported is set, but ats_enabled is cleared. So the
cache tag for the device TLB won't be created.

A possible solution is to move ATS enabling to a place before the
default domain attachment. However, this is not future-proof,
considering that we will eventually hand over the ATS control to the
device drivers. Therefore, this fix creates the device TLB cache tags as
long as ats_supported is true and relies on ats_enabled to decide
whether device TLB needs to be invalidated.

Best regards,
baolu