Re: [PATCH v9 04/11] iommu: Add sva iommu_domain support

[Baolu Lu]

Hi Kevin,

On 2022/6/27 16:29, Tian, Kevin wrote:
> > From: Lu Baolu <baolu.lu@xxxxxxxxxxxxxxx>
> > Sent: Tuesday, June 21, 2022 10:44 PM
> >
> > The sva iommu_domain represents a hardware pagetable that the IOMMU
> > hardware could use for SVA translation. This adds some infrastructure
> > to support SVA domain in the iommu common layer. It includes:
> >
> > - Extend the iommu_domain to support a new IOMMU_DOMAIN_SVA
> > domain
> >    type. The IOMMU drivers that support SVA should provide the sva
> >    domain specific iommu_domain_ops.
> > - Add a helper to allocate an SVA domain. The iommu_domain_free()
> >    is still used to free an SVA domain.
> > - Add helpers to attach an SVA domain to a device and the reverse
> >    operation.
> >
> > Some buses, like PCI, route packets without considering the PASID value.
> > Thus a DMA target address with PASID might be treated as P2P if the
> > address falls into the MMIO BAR of other devices in the group. To make
> > things simple, the attach/detach interfaces only apply to devices
> > belonging to the singleton groups, and the singleton is immutable in
> > fabric i.e. not affected by hotplug.
> >
> > The iommu_attach/detach_device_pasid() can be used for other purposes,
> > such as kernel DMA with pasid, mediation device, etc.
>
> I'd split this into two patches. One for adding iommu_attach/
> detach_device_pasid() and set/block_dev_pasid ops, and the
> other for adding SVA.

Yes. Make sense.

> >   struct iommu_domain {
> >   	unsigned type;
> >   	const struct iommu_domain_ops *ops;
> >   	unsigned long pgsize_bitmap;	/* Bitmap of page sizes in use */
> > -	iommu_fault_handler_t handler;
> > -	void *handler_token;
> >   	struct iommu_domain_geometry geometry;
> >   	struct iommu_dma_cookie *iova_cookie;
> > +	union {
> > +		struct {	/* IOMMU_DOMAIN_DMA */
> > +			iommu_fault_handler_t handler;
> > +			void *handler_token;
> > +		};
>
> why is it DMA domain specific? What about unmanaged
> domain? Unrecoverable fault can happen on any type
> including SVA. Hence I think above should be domain type
> agnostic.

The report_iommu_fault() should be replaced by the new
iommu_report_device_fault(). Jean has already started this work.

https://lore.kernel.org/linux-iommu/Yo4Nw9QyllT1RZbd@myrica/

Currently this is only for DMA domains, hence Robin suggested to make it
exclude with the SVA domain things.

https://lore.kernel.org/linux-iommu/f3170016-4d7f-e78e-db48-68305f683349@xxxxxxx/

> > +		struct {	/* IOMMU_DOMAIN_SVA */
> > +			struct mm_struct *mm;
> > +		};
> > +	};
> >   };
>
>
>
> > +
> > +struct iommu_domain *iommu_sva_domain_alloc(struct device *dev,
> > +					    struct mm_struct *mm)
> > +{
> > +	const struct iommu_ops *ops = dev_iommu_ops(dev);
> > +	struct iommu_domain *domain;
> > +
> > +	domain = ops->domain_alloc(IOMMU_DOMAIN_SVA);
> > +	if (!domain)
> > +		return NULL;
> > +
> > +	domain->type = IOMMU_DOMAIN_SVA;
>
> It's a bit weird that the type has been specified when calling
> ops->domain_alloc while it still leaves to the caller to set the
> type. But this is not caused by this series. could be cleaned
> up separately.

Yes. Robin has patches to refactor the domain allocation interface,
let's wait and see what it looks like finally.

> > +
> > +	mutex_lock(&group->mutex);
> > +	curr = xa_cmpxchg(&group->pasid_array, pasid, NULL, domain,
> > GFP_KERNEL);
> > +	if (curr)
> > +		goto out_unlock;
>
> Need check xa_is_err(old).

Either

(1) old entry is a valid pointer, or
(2) xa_is_err(curr)

are failure cases. Hence, "curr == NULL" is the only check we need. Did
I miss anything?

Best regards,
baolu