RE: [RFC PATCH 5/6] iommu: Support mm PASID 1:1 with sva domain

[Zhang, Tina]

> -----Original Message-----
> From: Jason Gunthorpe <jgg@xxxxxxxx>
> Sent: Tuesday, July 11, 2023 1:29 AM
> To: Zhang, Tina <tina.zhang@xxxxxxxxx>
> Cc: Tian, Kevin <kevin.tian@xxxxxxxxx>; Joerg Roedel <joro@xxxxxxxxxx>; Will
> Deacon <will@xxxxxxxxxx>; Lu Baolu <baolu.lu@xxxxxxxxxxxxxxx>; Michael
> Shavit <mshavit@xxxxxxxxxx>; iommu@xxxxxxxxxxxxxxx; linux-
> kernel@xxxxxxxxxxxxxxx
> Subject: Re: [RFC PATCH 5/6] iommu: Support mm PASID 1:1 with sva domain
> 
> On Fri, Jul 07, 2023 at 09:34:40AM +0800, Tina Zhang wrote:
> > Each mm bound to devices gets a PASID and a corresponding sva domain
> > allocated in iommu_sva_bind_device(), which are referenced by iommu_mm
> > field of the mm. And that PASID and sva domain get released in
> > iommu_sva_
> > unbind_device() when no devices are binding to that mm. As a result,
> > during the life cycle, sva domain has 1:1 with mm PASID.
> >
> > Since the required info of PASID and sva domain are kept in struct
> > iommu_mm_data of a mm, use mm->iommu_mm field instead of the old
> pasid
> > field in mm struct.
> 
> This is not technically right, the domains need to be a list, and we need to
> search the list. Almost always the list will have 0 or 1 entries in it.
OK. Then, the mapping between sva_domain and pasid is n:1. IIUC, the reason why we want this n:1 is because sva_domain is a type of iommu_domain and therefore may have some hardware IOMMU specific configurations that cannot be shared between different IOMMUs.

> 
> > @@ -88,31 +98,41 @@ struct iommu_sva *iommu_sva_bind_device(struct
> device *dev, struct mm_struct *mm
> >  		goto out_unlock;
> >  	}
> >
> > -	if (domain) {
> > -		domain->users++;
> > -		goto out;
> > +	if (unlikely(domain)) {
> > +		/* Re-attach the device to the same domain? */
> > +		if (domain == sva_domain) {
> > +			goto out;
> > +		} else {
> > +			/* Didn't get detached from the previous domain? */
> > +			ret = -EBUSY;
> > +			goto out_unlock;
> > +		}
> >  	}
> 
> And if we do all of this we should just get rid of the horrible
> iommu_get_domain_for_dev_pasid() entirely.
I think it depends on whether we could get rid of iommu_group->pasid_array, right? In the current implementation, it seems iommu_group->pasid_array is expected to identify the usages that need PASID-granular DMA address translation (i.e., not only for sva usage).

> 
> > diff --git a/include/linux/iommu.h b/include/linux/iommu.h index
> > 20135912584ba..1511ded7bc910 100644
> > --- a/include/linux/iommu.h
> > +++ b/include/linux/iommu.h
> > @@ -1175,20 +1175,20 @@ static inline bool
> > tegra_dev_iommu_get_stream_id(struct device *dev, u32 *stream  #ifdef
> > CONFIG_IOMMU_SVA  static inline void mm_pasid_init(struct mm_struct
> > *mm)  {
> > -	mm->pasid = IOMMU_PASID_INVALID;
> > +	mm->iommu_mm = &default_iommu_mm;
> >  }
> >  static inline bool mm_valid_pasid(struct mm_struct *mm)  {
> > -	return mm->pasid != IOMMU_PASID_INVALID;
> > +	return mm->iommu_mm->pasid != IOMMU_PASID_INVALID;
> >  }
> 
> And these can now just test if the iommu_mmu->sva_domain is NULL
Right. Baolu also shared a similar suggestion. Thanks.

Regards,
-Tina

> 
> Jaon