[PATCH v4 00/24] iommu/amd: Introduce AMD Hardware-accelerated Virtualized IOMMU (vIOMMU) Support

[Suravee Suthikulpanit]

AMD IOMMU introduces the vIOMMU feature, which provides partial hardware
acceleration when implementing Guest IOMMUs. This feature provides
acceleration for guest Command Buffer, Event Log, and PPR Log. This
eliminates the CPU overhead needed for the supporting HV intercepts and
reduces the latency of these operations.

When a guest attempts to access guest IOMMU MMIO registers with offsets
between 8KB and 12KB (i.e. the 3rd 4K region) such as the Command Buffer,
Event Log and PPR Log head and tail pointer registers, this is serviced
directly by the IOMMU. When the IOMMU accesses a Command Buffer, PPR Log
or a COMPLETION_WAIT store location in memory, it directly accesses guest
physical memory. The HV/VMM continues to trap and emulate the IOMMU
configuration MMIO registers between 0KB and 4KB (i.e. the 1st 4K
region), which are primarily used during initialization.

Additionally, the HV must initialize the vIOMMU feature, map MMIO resources
between the VMs and the IOMMU, manage additional supporting data structures
in memory (e.g. GPA->SPA translation DTE, Device ID and Domain ID mapping
tables), and allocate/map vIOMMU Private Address region used as backing
storage memory for the IOMMU. Support for new IOMMU command and events
specifically for vIOMMU are also added.

Guest IOMMUs are IOMMUs exposed to VMs with additional support from VMM
(QEMU) to generate guest ACPI IVRS table and define guest PCI topology for
IOMMU and pass-through VFIO devices, which are not covered by this series.

For more detail, please see the vIOMMU section of the AMD IOMMU
Specification[1].

This is version 4 of the AMD HW-vIOMMU series. It is implemented on top of
the IOMMUFD vIOMMU, vDevice, and nested-domain framework in Linux
v7.1.0-rc4.

The series is organized into the following subsets:

  Patch 1-3   : Preparatory patches
  Patch 4-8   : Introduce IOMMUFD vIOMMU support and VF MMIO setup
  Patch 9-14  : Introduce and map vIOMMU Private Address (IPA) region
  Patch 15-18 : Introduce IOMMUFD vDevice support for AMD
  Patch 19-24 : Translate-device-ID pool, per-vIOMMU translation DTE, and
                PCI-reserve relocation

Changes since v3:
(https://lore.kernel.org/linux-iommu/20260629153535.15775-1-suravee.suthikulpanit@xxxxxxx/)

Series scope:
  * Grows from 22 to 24 patches with split of the per-vIOMMU
    translate-device-id work into three patches plus PCI-reserve
    relocation.

IPA / DTE infrastructure (patches 9-14):
  * Patch 9: Register the owning IOMMU in viommu_pdom->iommu_array via
    amd_iommu_pdom_bind_iommu() so map/unmap use domain_flush_pages_v1().
    Remove amd_iommu_flush_private_vm_region().

IOMMUFD vDevice support (patches 16-18):
  * Drop gid/gDevId from struct iommu_dev_data; obtain guest device ID
    via iommufd_viommu_get_vdev_id() when programming nested DTEs.
  * Use ndom->viommu->gid for domain and device ID mapping table
    updates.
  * Validate vDevice virt_id against VIOMMU_MAX_GDEVID.

Translate device ID (TransDevID) - pool and relocation (patches 19-24):
  * Track allocated pool slots as pointers to the owning struct
    amd_iommu_viommu instead of a separate ALLOCATED state enum.
  * Move translate DTE / VFctrl helpers to patch 21;
    amd_iommu_set_translate_dte() takes struct iommufd_viommu * and reads
    trans_devid from struct amd_iommu_viommu.
  * Patch 22: add amd_iommu_trans_devid_alloc/free() with vIOMMU owner
    tracking in the per-segment xarray.
  * Patch 23: assign one translate-device-id per vIOMMU on init; program
    translation DTE and VFctrl TransDevID; add trans_devid_lock.
  * Patch 24: when PCI attach reserves a BDF already allocated to a
    vIOMMU, relocate that vIOMMU to a new translate-device-id before
    marking the BDF reserved.

Testing done:
  * Single/Multiple vIOMMU instances
  * Single/Multiple VFIO devices per vIOMMU instance.

[1] IOMMU Specification: https://docs.amd.com/v/u/en-US/48882_3.11_IOMMU_PUB
[2] Linux git tree: https://github.com/AMDESE/linux-iommu/tree/linux-7.1.0-rc4-amd-viommu_upstream_v4

Thank you,
Suravee

Suravee Suthikulpanit (24):
  iommu/amd: Make amd_iommu_completion_wait() non-static
  iommu/amd: Introduce vIOMMU-specific events and event
  iommu/amd: Detect and initialize AMD vIOMMU feature
  iommu/amd: Introduce IOMMUFD vIOMMU support for AMD
  iommu/amd: Allocate Guest IDs for IOMMUFD vIOMMU instances
  iommu/amd: Map vIOMMU VF and VF Control MMIO BARs
  iommu/amd: Add support for AMD vIOMMU VF MMIO region
  iommu/amd: Introduce Reset vMMIO Command
  iommu/amd: Introduce and map vIOMMU private IPA region
  iommu/amd: Pass iommu to device_flush_dte()
  iommu/amd: Export amd_iommu_alloc_dev_data() helper
  iommu/amd: Pass iommu and devid to amd_iommu_make_clear_dte()
  iommu/amd: Assign IOMMU Private Address domain to IOMMU
  iommu/amd: Add per-VM private IPA alloc/map helpers
  iommu/amd: Add helper functions to manage DevID / DomID mapping tables
  iommu/amd: Introduce IOMMUFD vDevice support for AMD
  iommu/amd: Introduce helper function for updating domain ID mapping
    table
  iommu/amd: Introduce helper function for updating device ID mapping
    table
  iommu/amd: Add per-segment translate device ID pool
  iommu/amd: Reserve translate-device-id for PCI requestor aliases
  iommu/amd: Add translation DTE and VFctrl TransDevID helpers
  iommu/amd: Add translate-device-id alloc/free with vIOMMU owner
  iommu/amd: Assign per-vIOMMU translate device ID
  iommu/amd: Relocate vIOMMU translate-device-id on PCI reserve

 drivers/iommu/amd/Makefile          |   2 +-
 drivers/iommu/amd/amd_iommu.h       |  51 ++-
 drivers/iommu/amd/amd_iommu_types.h | 105 ++++++
 drivers/iommu/amd/amd_viommu.h      |  60 +++
 drivers/iommu/amd/init.c            |  23 +-
 drivers/iommu/amd/iommu.c           | 290 ++++++++++++---
 drivers/iommu/amd/iommufd.c         | 134 +++++++
 drivers/iommu/amd/nested.c          |  38 +-
 drivers/iommu/amd/trans_devid.c     | 329 ++++++++++++++++
 drivers/iommu/amd/viommu.c          | 558 ++++++++++++++++++++++++++++
 include/uapi/linux/iommufd.h        |  10 +
 11 files changed, 1543 insertions(+), 57 deletions(-)
 create mode 100644 drivers/iommu/amd/amd_viommu.h
 create mode 100644 drivers/iommu/amd/trans_devid.c
 create mode 100644 drivers/iommu/amd/viommu.c

-- 
2.34.1