Re: [RFC PATCH v4 03/16] iommu/arm-smmu-v3: Add initial pSMMU realm viommu plumbing

[Jason Gunthorpe]

On Thu, Sep 10, 2026 at 09:52:12AM +0000, Tian, Kevin wrote:
> > The mmio is owned by vfio, there should be a handshake in VFIO to
> > remove mmio when it becomes private, otherwise I don't think we need
> > to do anything more?
> 
> for TDX a callback in VFIO is invoked to revoke the dmabuf.
> 
> btw I didn't see any dmabuf related in this series (even in a hackish
> way as Yilun sent out last year). Is it not required by CCA or just
> skipped now waiting for the framework to be settled?

I think just one step at a time :\ I think CCA needs it as well, you
also cannot safely leave private MMIO floating around the host
userspace on CCA.

> So viommu will become the abstraction point for tsm operations and
> directly talks to the tsm driver instead of going through the merged 
> tsm_ops (bind/unbind/guest_req).

More precisely for what was being called the TDI - ie it is the
abstraction point for the host side of a guest virtual PCI device.

iommufd vdevice already represents this, so we use it again.
 
> currently what tsm_ops provides additionally is more about synchronization
> with connect/disconnect, otherwise just calls into underlying tsm driver. As
> long as viommu creation holds a reference to tsm then disconnection is
> blocked then it's safe.

Yes, this already has to be true or our lifecylce model is
nonsense. Once the iommufd creates the vdevice the guest is running
and we cannot disconnect it without unplugging it from the guest. Thus
the locking can rely on that. Within an iommufd ioctl context the vdev
and underlying connection must be stable.

> and IOMMU_VDEVICE_TSM_BIND will be removed. 

Replaced, the same logical operations should flow through the viommu
command ioctl

> Seems originally this series
> does SMC_RMI_VDEV_CREATE and SMC_RMI_VDEV_LOCK both in tsm
> bind op. Then the proposal is moving them all to vdevice creation time so
> no separate bind step is required.

Hmm, lock still should be guest visible on CCA, and the device has to
start up in the guest as TDISP unlocked. I'm not 100% on the RMI side
of the flow, but this does not look right.

VDEV create should be done before the realm is started. We need this
setup right because Linux VM is going to validate the vdev during boot
if it is affiliated to a vSMMU. lock should be done only when
requested by the guest. The guest must start with a normal unlocked
T=0 PCI device.

> Does it mean that ARM CCA is essentially an early-bind model i.e. the TDI
> starts in locked state from guest p.o.v.?

No

> but https://lore.kernel.org/all/yq5aecfbzjyk.fsf@xxxxxxxxxx/ seems to indicate
> that guest_req() still handles the bind request from the guest:
> 
> "
> - cca_tsm_guest_req() now handles TSM_REQ_SET_TDI_STATE requests for
>    the unlocked, locked, and running states.
> "

Right the guest side request to lock should come that way.

> 
> a bit confusing, but maybe due to stale info cited from different timing...
> 
> for TDX we're currently pursuing a late-bind model (i.e. TDI starts in shared
> state in guest), so a separate bind interface is still necessary. We could look
> at whether it should be an explicit @bind viommu op or carried by 
> @guest_req (say, if most preparation works can be moved to vdevice creation).

It should go over the viommu command ioctl. It looked like there was
alway some iommu specific format to the command in the tsm version of
this, so that is the appropriate way.

> btw per past discussions VFIO should block some operations (reset, etc.)
> while a TDI is locked. Do we expect a callback into VFIO to notify the locked
> state, or require VFIO's own interface to place a cdev into a special state
> which prevents sensitive operations even before binding it to iommufd
> (and allow creating TDI vdevice only on such idevice)?

At a minimum I think we have to synchronize with vfio, somehow, that
when the device enforces its private MMIO so it can unmap it. Thay may
be an argument we need a few standard viommu ops so the core code can
capture lock/unlocks and do this prep work. Or maybe a special cdev
state is simpler.

This should be figured out before any uapi is settled.

I don't know about reset, that sounds like something the pci core
should deal with?

Jason