Re: [PATCH v2 14/18] iommu/arm-smmu-v3: Support domains with shared CDs

[Baolu Lu]

On 6/7/23 7:59 PM, Jason Gunthorpe wrote:
> On Wed, Jun 07, 2023 at 12:06:07AM +0530, Michael Shavit wrote:
> > > What we definately shouldn't do is try to have different SVA
> > > iommu_domain's pointing at the same ASID. That is again making SVA
> > > special, which we are trying to get away from 😄
> > Fwiw, this change is preserving the status-quo in that regard;
> > arm-smmu-v3-sva.c is already doing this. But yes, I agree that
> > resolving the limitation is a better long term solution... and
> > something I can try to look at further.
> I suppose we also don't really have a entirely clear picture what
> allocating multiple SVA domains should even do in the iommu driver.
>
> The driver would like to share the ASID, but things are much cleaner
> for everything if the driver model has ASID 1:1 with the iommu_domain.

This means that each ASID should be mapped to a single IOMMU domain.
This is conceptually right as iommu_domain represents a hardware page
table. For SVA, it's an mm_struct.

So in my mind, each sva_domain should have a 1:1 relationship with an
mm_struct. Each sva_domain could have a 1:N relationship with ASID (or
PCI PASID), but in current implementation, it's a 1:1 relationship due
to the current global pasid policy for SVA.

> It suggests we are missing some core code in iommu_sva_bind_device()
> to try to re-use existing SVA iommu_domains. This would certainly be
> better than trying to teach every driver how to share and refcount
> its ASID concept...
>
> Today we have this super hacky iommu_get_domain_for_dev_pasid()
> thing that allows SVA domain reuse for a single device.
>
> Possibly what we should do is conver the u32 pasid in the mm_struct to
> a struct iommu_mm_data * and put alot more stuff in there. eg a linked
> list of all SVA domains.

I don't quite follow "a linked list of all SVA domains". If my above
understanding is correct, then there should be a single sva_domain for
each mm_struct. The iommu_sva_domain_alloc() takes the responsibility to
allocate/free and refcount the sva domain. The sva bind code could
simply:

	domain = iommu_get_sva_domain(dev, mm);
	iommu_attach_device_pasid(domain, dev, pasid);

and the sva unbind code:

	iommu_detach_device_pasid(domain, dev, pasid);
	iommu_put_sva_domain(domain);

Perhaps, we can further drop struct iommu_sva and make the sva
bind/unbind interfaces to return the sva domain instead?

Best regards,
baolu