Re: [PATCH v6 08/12] iommu/sva: Use attach/detach_pasid_dev in SVA interfaces

[Baolu Lu]

On 2022/5/10 23:23, Jason Gunthorpe wrote:
> On Tue, May 10, 2022 at 02:17:34PM +0800, Lu Baolu wrote:
>
> > +/**
> > + * iommu_sva_bind_device() - Bind a process address space to a device
> > + * @dev: the device
> > + * @mm: the mm to bind, caller must hold a reference to mm_users
> > + * @drvdata: opaque data pointer to pass to bind callback
> > + *
> > + * Create a bond between device and address space, allowing the device to access
> > + * the mm using the returned PASID. If a bond already exists between @device and
> > + * @mm, it is returned and an additional reference is taken. Caller must call
> > + * iommu_sva_unbind_device() to release each reference.
> > + *
> > + * iommu_dev_enable_feature(dev, IOMMU_DEV_FEAT_SVA) must be called first, to
> > + * initialize the required SVA features.
> > + *
> > + * On error, returns an ERR_PTR value.
> > + */
> > +struct iommu_sva *
> > +iommu_sva_bind_device(struct device *dev, struct mm_struct *mm, void *drvdata)
> > +{
> > +	int ret = -EINVAL;
> > +	struct iommu_sva *handle;
> > +	struct iommu_domain *domain;
> > +
> > +	/*
> > +	 * TODO: Remove the drvdata parameter after kernel PASID support is
> > +	 * enabled for the idxd driver.
> > +	 */
> > +	if (drvdata)
> > +		return ERR_PTR(-EOPNOTSUPP);
>
> Why is this being left behind? Clean up the callers too please.

Okay, let me try to.

> > +	/* Allocate mm->pasid if necessary. */
> > +	ret = iommu_sva_alloc_pasid(mm, 1, (1U << dev->iommu->pasid_bits) - 1);
> > +	if (ret)
> > +		return ERR_PTR(ret);
> > +
> > +	mutex_lock(&iommu_sva_lock);
> > +	/* Search for an existing bond. */
> > +	handle = xa_load(&dev->iommu->sva_bonds, mm->pasid);
> > +	if (handle) {
> > +		refcount_inc(&handle->users);
> > +		goto out_success;
> > +	}
>
> How can there be an existing bond?
>
> dev->iommu is per-device
>
> The device_group_immutable_singleton() insists on a single device
> group
>
> Basically 'sva_bonds' is the same thing as the group->pasid_array.

Yes, really.

> Assuming we leave room for multi-device groups this logic should just
> be
>
> 	group = iommu_group_get(dev);
> 	if (!group)
> 		return -ENODEV;
>
> 	mutex_lock(&group->mutex);
> 	domain = xa_load(&group->pasid_array, mm->pasid);
> 	if (!domain || domain->type != IOMMU_DOMAIN_SVA || domain->mm != mm)
> 		domain = iommu_sva_alloc_domain(dev, mm);
>
> ?

Agreed. As a helper in iommu core, how about making it more generic like
below?

+struct iommu_domain *iommu_get_domain_for_dev_pasid(struct device *dev,
+                                                   iosid_t pasid,
+                                                   unsigned int type)
+{
+       struct iommu_domain *domain;
+       struct iommu_group *group;
+
+       if (!pasid_valid(pasid))
+               return NULL;
+
+       group = iommu_group_get(dev);
+       if (!group)
+               return NULL;
+
+       mutex_lock(&group->mutex);
+       domain = xa_load(&group->pasid_array, pasid);
+       if (domain && domain->type != type)
+               domain = NULL;
+       mutex_unlock(&group->mutex);
+       iommu_group_put(group);
+
+       return domain;
+}

> And stick the refcount in the sva_domain
>
> Also, given the current arrangement it might make sense to have a
> struct iommu_domain_sva given that no driver is wrappering this in
> something else.

Fair enough. How about below wrapper?

+struct iommu_sva_domain {
+       /*
+        * Common iommu domain header, *must* be put at the top
+        * of the structure.
+        */
+       struct iommu_domain domain;
+       struct mm_struct *mm;
+       struct iommu_sva bond;
+}

The refcount is wrapped in bond.

Best regards,
baolu