Re: [RFC PATCH v4 03/16] iommu/arm-smmu-v3: Add initial pSMMU realm viommu plumbing

[Jason Gunthorpe]

On Wed, Sep 16, 2026 at 05:54:57AM +0000, Tian, Kevin wrote:
> > At least for ARM there is effectively no entanglement with the actual
> > host iommu driver. The viommu is entirely provided by software in the
> > RMM world, so it can have its own dedicated driver. In ARM T=1
> > transactions are alwayus routed to the RMM's iommu and there is no
> > relation to the host.
> > 
> > I am interested how Intel works here, but I thought it was similar.
> 
> Largely yes. Main difference at Intel side is that TDX still relies on the
> host to initiate iotlb invalidation (upon notification from KVM on S-EPT
> change). Currently we put this logic in intel-iommu driver but it's more
> about wrapping invalidation info and passing it to the firmware. Moving
> it into the tsm driver should be straightforward.
> 
> Maybe there'll be other subtle connections to host iommu driver but
> it doesn't sound a hard problem to solve.

Okay, so I saw the driver posting for basic iommu support, can we try
to rework that to be split out like Aneesh is doing so everything
about TDX calls lives in tsm and intel iommu only provides a small API
surface to exchange whatever details are needed to bootstrap TDX
module?

> > AMD is different and I suspect AMD will have to continue to use the
> > viommu from the AMD iommu driver, but I am not sure.
> 
> ARM/Intel may support guest viommu in the future. 

ARM supports guest viommu today, it is in the public spec. Secure
guest vSMMU is entirely handled inside the RMM and has no connection
to the host iommu driver. It is a <100 line ++ on top of Aneesh's
work, Nicolin posted a draft at one point in those threads.

I anticipate a future intel guest T=1 viommu should be the same.

Thus I expect Intel/ARM to have two viommus, one that handles the T=1
stream owned by the TSM driver and implemented entirely by calling
TDX/RMM.

One that handles the T=0 stream owned by the iommu driver - and it
already exists.

> So AMD's case is a good reference. 

I think, AMD is completely different. I keep forgetting thier thing,
but IIRC they have a secure DTE but instead of having the secure word
control the translation it controls the RMP and you end up using the
host's translation for T=1 traffic. This is fundamentally different
from how Intel and ARM are doing it where the actually IOVA translate
is under the control of the secure world.

Both Intel and ARM put the S-EPT into the iommu HW directly.

So, I expect Intel to have an API similar to ARM. When you create the
TSM viommu you tell it if the TDX module should create a secure guest
visible VT-d emulation. TDX module has to perform the entire emulation
because it must be trusted. Existing viommu ops should cover the
remaining to register pdevices as vdevices, provide the vBDF and so
on.

> > How/when the tsm driver links this to a arch specific "bind/unbind"
> > operation is more up to that driver, but I would expect what is
> > thought of as "bind" should be the affiliation of the device's T=1
> > stream with the viommu and the target VM. It should not be sensitive
> > to the TDISP state.
> 
> Not sure about this part.
> 
> Each arch has its own definition about the binding flow (about 'how'),
> but sharing a common step by sending TDISP message to transit the
> TDI into the CONFIG_LOCKED state upon guest request (i.e. 'when').

Sure, the LOCKED command can be relayed from the guest, but that
shouldn't be called BIND. locked/unlock/run/err is taking a iommufd
vdev that is already affiliated with the VM to a specific TDISP state

> According to the TDISP spec, memory reads/writes with T bit set is
> accepted only when the TDI is in RUN state (except MSI/MSI-X writes
> are allowed with T bit set in LOCKED but I don't think any arch supports
> it yet).

Sure

> So your definition of 'bind' essentially affiliate it to the RUN state?

No, it is informing the secure world that a physical PCI function is
now a virtual PCI function, is a TDI, and is in a certain VM.

Outside virtual hotplug this is a permanent action when the VM is
created.

> > That is not prohibited, the TSM driver could do some auto
> > "bind/unbind" whatever that means triggered by ops or tdisp state
> > changing under the covers. But this cannot leak out as some kind of
> > asynchronous vdev destruction.
> 
> Maybe it'd be clearer using an example e.g. ARM to clarify the
> suggested split. Or wait for Aneesh's next version...

In ARM:

BIND is RMI_VDEV_CREATE it links a physical device to a virtual
device in a realm.

RMI_VSMMU_CREATE can attach a vSMMU to the realm and there is some way
to link the VDEV And the VSMMU together

Some sequence of RMI_VDEV_COMMUNICATE, RMI_VDEV_LOCK, RMI_VDEV_UNLOCK
and a few others manipulate the UNLOCKED/LOCKED/RUN/ERR TDISP state of
the VDEV.

I assume TDX has the same general shape, I don't know how you could
implement this in a radically different way?

So iommufd viommu create calls RMI_VSMMU_CREATE
   iommufd vdev create calls RMI_VDEV_CREATE
   iommufd viommu op ioctl calls the COMMUNICATE/LOCK/UNLOCK

Jason