Re: [RFC PATCH v4 03/16] iommu/arm-smmu-v3: Add initial pSMMU realm viommu plumbing

[Jason Gunthorpe]

On Wed, Sep 09, 2026 at 03:39:31PM +0530, Aneesh Kumar K.V wrote:
> Jason Gunthorpe <jgg@xxxxxxxx> writes:
> 
> > On Mon, Sep 07, 2026 at 03:15:25PM +0530, Aneesh Kumar K.V wrote:
> >
> >> I looked into this, and it becomes fairly complicated. We can move all
> >> vdev/TDI-related code to arm-smmu-realm-v3.c, but that would result in:
> >
> > I was going for the opposite, you'd move everything out of arm-smmu-v3
> > and into the arm-cca-host and obtain the viommu through tsm_ops not
> > through iommu_ops.
> >
> > I guess I pointed to that in another email.
> >
> > The only thing arm-smmu-v3 should provide is a simple function to give
> > the pdev phys and irq parameters. arm-cca-host calls that when it
> > creates an viommu object.
> >
> 
> 
> So ended up with
> 
> static const struct tsm_viommu_ops cca_tsm_viommu_ops = {
> 	.owner = THIS_MODULE,
> 	.type = IOMMU_VIOMMU_TYPE_ARM_REALM_SMMUV3,
> 	.get_size = cca_viommu_get_size,
> 	.init = cca_viommu_init,
> };

OK, may also be fine in the main tsm ops? 

> struct cca_viommu {
> 	struct cca_psmmu *psmmu;
> 	struct iommu_viommu_provider *iommu_provider;
> 	const struct iommufd_viommu_ops *iommu_ops; // backing SMMU ops

??

The viommu created for a RMM owned vSMMU should have no connection to
the normal SMMU driver? It just needs the psmmu information.

We need to adjust iommufd side to know that this object cannot accept
an iommu_domain, only a kvm fd. I don't want to pass in a fake
iommu_domain that has nothing to do with how RMM will operate things.

> static struct iommu_domain *
> cca_viommu_alloc_domain_nested(struct iommufd_viommu *viommu, u32 flags,
> 			       const struct iommu_user_data *user_data)
> {
> 	struct cca_viommu *cca = viommu->provider_data;
> 
> 	return cca->iommu_ops->alloc_domain_nested(viommu, flags, user_data);
> }
> 
> static int cca_viommu_cache_invalidate(struct iommufd_viommu *viommu,
> 				       struct iommu_user_data_array *array)
> {
> 	struct cca_viommu *cca = viommu->provider_data;
> 
> 	return cca->iommu_ops->cache_invalidate(viommu, array);
> }

These ops should fail (just be NULL) not reflect to a psmmu.
 
> struct cca_vdevice {
> 	struct iommufd_vdevice core;
> 	struct pci_tsm_context *tsm_context;
> 	struct cca_host_tdi host_tdi;

Is the extra tdi struct still needed? Isn't cca_vdevice the tdi struct?

> 	u32 l2_sid;
> };
> 
> static const struct iommufd_viommu_ops cca_viommu_ops = {
> 	.destroy = cca_viommu_destroy,
> 	.alloc_domain_nested = cca_viommu_alloc_domain_nested,
> 	.cache_invalidate = cca_viommu_cache_invalidate,
> 	.vdevice_size = VDEVICE_STRUCT_SIZE(struct cca_vdevice, core),
> 	.vdevice_init = cca_vdevice_init,
> 	.vdevice_tsm_req = cca_vdevice_tsm_req,
> };

> 
> and on iommu side
> 
> struct iommu_viommu_provider {
> 	size_t size;
> 	int (*init)(struct iommufd_viommu *viommu, struct device *dev,
> 		    enum iommu_viommu_type type,
> 		    struct iommu_domain *parent_domain,
> 		    const struct iommu_user_data *user_data);
> 	int (*get_params)(struct iommu_viommu_provider *provider,
> 			  struct device *dev, enum iommu_viommu_type type,
> 			  void *params, size_t params_size);
> 	void (*release)(struct iommu_viommu_provider *provider);
> 	void *data;
> };

Not sure what this is?

> The TSM disconnect path will now fail while any vdevice is alive or
> active. 

Yes, that's makes sense.

> Destroying a vdevice will unlock and destroy the VDEV. 

Yes

> I think we can also unmap its MMIO mappings at that point, provided
> we track the mapping requests in a list alongside the vdevice
> details.

The mmio is owned by vfio, there should be a handshake in VFIO to
remove mmio when it becomes private, otherwise I don't think we need
to do anything more?

> All CCA operations will use pci_tsm_pf0::lock, though I think the
> locking can be made more fine-grained.

Sure
 
> I will send a cleaned-up series so that we can review the code changes.

Does it seems reasonable to you? Was there any oddness with modeling
the vdev/bind through the viommu?

Anyhow I'll look more closely when you are ready. Thanks

Jason