RE: [PATCH v2 3/4] iommu/hyperv: Add para-virtualized IOMMU support for Hyper-V guest

[Michael Kelley]

From: Yu Zhang <zhangyu1@xxxxxxxxxxxxxxxxxxx> Sent: Friday, July 10, 2026 12:34 AM
> 
> On Thu, Jul 09, 2026 at 07:08:26PM +0000, Michael Kelley wrote:
> > From: Yu Zhang <zhangyu1@xxxxxxxxxxxxxxxxxxx> Sent: Thursday, July 2, 2026 9:05 AM
> > >
> > > Add a para-virtualized IOMMU driver for Linux guests running on Hyper-V.
> > > This driver implements stage-1 IO translation within the guest OS.
> > > It integrates with the Linux IOMMU core, utilizing Hyper-V hypercalls
> > > for:
> > >  - Capability discovery
> > >  - Domain allocation, configuration, and deallocation
> > >  - Device attachment and detachment
> > >  - IOTLB invalidation
> > >
> > > The driver constructs x86-compatible stage-1 IO page tables in the
> > > guest memory using consolidated IO page table helpers. This allows
> > > the guest to manage stage-1 translations independently of vendor-
> > > specific drivers (like Intel VT-d or AMD IOMMU).
> > >
> > > Hyper-V consumes this stage-1 IO page table when a device domain is
> > > created and configured, and nests it with the host's stage-2 IO page
> > > tables, therefore eliminating the VM exits for guest IOMMU mapping
> > > operations. For unmapping operations, VM exits to perform the IOTLB
> > > flush are still unavoidable.
> > >
> > > To identify a device in its hypercall interface, the driver looks up the
> > > logical device ID prefix registered for the device's PCI domain (see the
> > > logical device ID registry in hv_common.c) and combines it with the PCI
> > > function number of the endpoint device.
> > >
> > > Co-developed-by: Wei Liu <wei.liu@xxxxxxxxxx>
> > > Signed-off-by: Wei Liu <wei.liu@xxxxxxxxxx>
> > > Co-developed-by: Easwar Hariharan <easwar.hariharan@xxxxxxxxxxxxxxxxxxx>
> > > Signed-off-by: Easwar Hariharan <easwar.hariharan@xxxxxxxxxxxxxxxxxxx>
> > > Signed-off-by: Yu Zhang <zhangyu1@xxxxxxxxxxxxxxxxxxx>
> > > ---
> > >  arch/x86/hyperv/hv_init.c       |   4 +
> > >  arch/x86/include/asm/mshyperv.h |   4 +
> > >  drivers/iommu/Kconfig           |   1 +
> > >  drivers/iommu/hyperv/Kconfig    |  16 +
> > >  drivers/iommu/hyperv/Makefile   |   1 +
> > >  drivers/iommu/hyperv/iommu.c    | 620 ++++++++++++++++++++++++++++++++
> > >  drivers/iommu/hyperv/iommu.h    |  51 +++
> > >  7 files changed, 697 insertions(+)
> > >  create mode 100644 drivers/iommu/hyperv/Kconfig
> > >  create mode 100644 drivers/iommu/hyperv/iommu.c
> > >  create mode 100644 drivers/iommu/hyperv/iommu.h
> > >
> > > diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
> > > index 55a8b6de2865..094f9f7ddb72 100644
> > > --- a/arch/x86/hyperv/hv_init.c
> > > +++ b/arch/x86/hyperv/hv_init.c
> > > @@ -578,6 +578,10 @@ void __init hyperv_init(void)
> > >  	old_setup_percpu_clockev = x86_init.timers.setup_percpu_clockev;
> > >  	x86_init.timers.setup_percpu_clockev = hv_stimer_setup_percpu_clockev;
> > >
> > > +#ifdef CONFIG_HYPERV_PVIOMMU
> > > +	x86_init.iommu.iommu_init = hv_iommu_init;
> > > +#endif
> > > +
> >
> > This approach to .iommu_init is a bit different from the Intel VT-d and
> > AMD IOMMU initialization. Those cases detect the existence of the
> > IOMMU first via a "detect" function that is called in pci_iommu_alloc().
> > If the detect function finds an IOMMU, it sets .iommu_init. Any
> > reason not to use the same approach for the Hyper-V pvIOMMU?
> > One problem with exactly the same approach is that Hyper-V
> > hypercalls aren't set up at the time pci_iommu_alloc() runs.
> 
> Yes. That's why I did not follow Intel VT-d and AMD IOMMU's approach -
> the hv_hypercall_pg is not ready yet.
> 
> > So you'd have to call the "detect" function here in hyperv_init(),
> > and have the detect function set .iommu_init if pvIOMMU
> > support is present.
> >
> 
> The detecion of the presense and capabilities of the pvIOMMU are done
> in one hypercall. But I guess we can:
> - do the HVCALL_GET_IOMMU_CAPABILITIES in hyperv_init();
> - check the presense and only set .iommu_init to hyperv_iommu_init()
>   if pvIOMMU is present;
> - and then do other capalibities check in hv_iommu_init();
> - only give the error log if an pvIOMMU is present yet its capabilities
>   are not legal.
> So below errors will not be printed for guest kernels built with
> CONFIG_HYPERV_PVIOMMU and running on a host w/o one.

I see your point about detection and capabilities coming from
single hypercall, and that separating those two functions
would duplicate code. My biggest concern is about errors in the
dmesg log for a valid configuration where the host doesn't
supply a pvIOMMU. Fixing that problem in the context of the
current code structure would be acceptable.

A minor concern is arguably misusing the .iommu_init function
to do detection. But that function is only called once at boot
time, so leaving it set to hv_iommu_init() even if there isn't
a Hyper-V pvIOMMU is probably more a conceptual issue
than a real issue. I wouldn't object if you prefer to leave that
"as is" to avoid duplicating the hypercall.

One new thought:  Have you considered the hibernate/resume
cycle? Does anything need to be done with the pvIOMMU to
make it functional again after resume? I see that the Intel and
AMD IOMMU drivers have suspend and resume functions. I
don't know enough about the Hyper-V pvIOMMU to know if it
might also need suspend and resume functions.

Michael

> 
> > While the code currently in this patch works, it generates boot
> > time errors if the kernel is built with CONFIG_HYPERV_PVIOMMU
> > but run in a guest on a host without pvIOMMU support:
> >
> > [    0.101673] Hyper-V pvIOMMU: HVCALL_GET_IOMMU_CAPABILITIES failed, status 2
> > [    0.101675] Hyper-V pvIOMMU: HVCALL_GET_IOMMU_CAPABILITIES failed: -22
> >
> > We really don't want errors if it's just the case that there's no
> > pvIOMMU support. A less alarming message (at INFO level instead
> > of ERROR level) about running without an IOMMU might be OK, but
> > perhaps is unnecessary since you have an INFO message if the
> > pvIOMMU is found and successfully initialized.
> >