[PATCH v2 00/26] iommu/amd: Introduce AMD Hardware-accelerated Virtualized IOMMU (vIOMMU) Support

[Suravee Suthikulpanit]

AMD IOMMU introduces the vIOMMU feature, which provides partial hardware
acceleration when implementing Guest IOMMUs. This feature provides
acceleration for guest Command Buffer, Event Log, and PPR Log. This
eliminates the CPU overhead needed for the supporting HV intercepts and
reduces the latency of these operations.

When a guest attempts to access guest IOMMU MMIO registers with offsets
between 8KB and 12KB (i.e. the 3rd 4K region) such as the Command Buffer,
Event Log and PPR Log head and tail pointer registers, this is serviced
directly by the IOMMU. When the IOMMU accesses a Command Buffer, PPR Log
or a COMPLETION_WAIT store location in memory, it directly accesses guest
physical memory. The HV/VMM continues to trap and emulate the IOMMU
configuration MMIO registers between 0KB and 4KB (i.e. the 1st 4K
region), which are primarily used during initialization.

Additionally, the HV must initialize the vIOMMU feature, map MMIO resources
between the VMs and the IOMMU, manage additional supporting data structures
in memory (e.g. GPA->SPA translation DTE, Device ID and Domain ID mapping
tables), and allocate/map vIOMMU Private Address region used as backing
storage memory for the IOMMU. Support for new IOMMU command and events
specifically for vIOMMU are also added.

Guest IOMMUs are IOMMUs exposed to VMs with additional support from VMM
(QEMU) to generate guest ACPI IVRS table and define guest PCI topology for
IOMMU and pass-through VFIO devices, which are not covered by this series.

For more detail, please see the vIOMMU section of the AMD IOMMU
Specification[1].

This is version 2 of the AMD HW-vIOMMU series previously posted as v1[2].
It is implemented on top of the IOMMUFD vIOMMU, vDevice, and hw_queue
framework in Linux v7.1.0-rc4.

The series is organized into the following subsets:

  Patch 1-3   : Preparatory patches
  Patch 4-5   : Introduce IOMMUFD vIOMMU support for AMD
  Patch 6-8   : Introduce AMD vIOMMU VF MMIO and VFCtrl MMIO
  Patch 9-12  : Introduce AMD vIOMMU Private Address support
  Patch 13-16 : Introduce IOMMUFD vDevice support for AMD
  Patch 17-21 : Translation DTE, KVM FD, and translate-device-ID pool
  Patch 22-26 : IOMMUFD hw_queue, VIOMMU_COMMAND ioctl, enable vIOMMU

Changes since v1:
(https://lore.kernel.org/linux-iommu/20260330084206.9251-1-suravee.suthikulpanit@xxxxxxx/)

Rebase and scope:
  * Rebased on Linux v7.1.0-rc4.

Guest ID (GID) - design change (patch 5-6):
  * v1: A single global IDA; each GID is unique across all AMD IOMMUs in
    the system.
  * v2: A per-amd_iommu IDA (gid_ida), initialized when vIOMMU MMIO is
    set up on that IOMMU. GIDs are allocated in amd_iommufd_viommu_init()
    and freed on destroy - one GID per IOMMUFD vIOMMU object, unique within
    that IOMMU only. A VM with guest IOMMUs behind multiple IOMMUs may
    therefore hold multiple GIDs. This is separate from TransDevID, which
    remains one per VM (kvmfd) and is shared across vIOMMUs for that VM.

Translate device ID (TransDevID) - design change (patches 17-21):
  * v1: VMM supplies trans_devid via struct iommu_viommu_amd; driver
    programs translation DTE and VFctrl guest-misc register.
  * v2: Driver allocates trans_devid per VM from a per-PCI-segment pool
    (trans_devid.c), reserves IDs used by real PCI functions on attach,
    and keys allocation to kvmfd so multiple vIOMMU instances for one VM
    share a single GPA->SPA translation DTE.
  * struct iommu_viommu_amd gains kvmfd; trans_devid removed from UAPI.

Userspace vIOMMU control path - design change (patches 24-25):
  * v1: Extended IOMMU_OPTION with IOMMU_OPTION_VIOMMU and
    set_option/get_option ops.
  * v2: New IOMMU_VIOMMU_COMMAND ioctl with set_command/get_command ops
    for indexed read/write of vIOMMU register arrays (e.g. guest MMIO
    shadow). AMD backend moves guest MMIO accessors to vfctrl_mmio.c.

Single-patch / implementation notes:
  * Patch 4: Gate amd_iommufd_get_viommu_size() on HW vIOMMU capability.
  * Patch 8: Reset-vMMIO helper declaration in amd_iommu.h (rebase).
  * Patch 11-12: Per-VM IPA map/unmap via iommu_map/iommu_unmap (was
    pt_iommu_amdv1_map_pages in v1 per-VM helpers).
  * Patch 26: iommu_feature_enable_and_check(); improved error unwind in
    amd_viommu_init(); enable vIOMMU by default when supported.

Upcoming series (in subsequent submission):
  * Extended Interrupt Remapping (guest Event / PPR log interrupts)
  * KVM/AVIC integration and guest event injection support

Testing done:
  * Single/Multiple vIOMMU instances
  * Single/Multiple VFIO devices per vIOMMU instance.

[1] IOMMU Specification: https://docs.amd.com/v/u/en-US/48882_3.10_PUB
[2] Linux git tree: https://github.com/AMDESE/linux-iommu/tree/linux-7.1.0-rc4-amd-viommu_upstream_v2

Thank you,
Suravee

Suravee Suthikulpanit (26):
  iommu/amd: Make amd_iommu_completion_wait() non-static
  iommu/amd: Introduce vIOMMU-specific events and event
  iommu/amd: Detect and initialize AMD vIOMMU feature
  iommu/amd: Introduce IOMMUFD vIOMMU support for AMD
  iommu/amd: Allocate Guest IDs for IOMMUFD vIOMMU instances
  iommu/amd: Map vIOMMU VF and VF Control MMIO BARs
  iommu/amd: Add support for AMD vIOMMU VF MMIO region
  iommu/amd: Introduce Reset vMMIO Command
  iommu/amd: Introduce domain for IOMMU Private Address (IPA) region
  iommu/amd: Assign IOMMU Private Address domain to IOMMU
  iommu/amd: Allocate and map vIOMMU private regions
  iommu/amd: Add per-VM private IPA alloc/map helpers
  iommu/amd: Add helper functions to manage DevID / DomID mapping tables
  iommu/amd: Introduce IOMMUFD vDevice support for AMD
  iommu/amd: Introduce helper function for updating domain ID mapping
    table
  iommu/amd: Introduce helper function for updating device ID mapping
    table
  iommu/amd: Pass KVM FD from userspace when initializing vIOMMU
  iommu/amd: Add translation DTE and VFctrl TransDevID helpers
  iommu/amd: Add per-segment translate device ID pool
  iommu/amd: Reserve translate-device-id for PCI requestor aliases
  iommu/amd: Map kvmfd to shared translate device ID for vIOMMU
  iommufd: Add hw_queue_init and split queue alloc paths
  iommu/amd: Add support for vIOMMU HW queues initialization
  iommufd: Introduce vIOMMU command via VIOMMU_COMMAND ioctl
  iommu/amd: Handle set/get command for AMD vIOMMU
  iommu/amd: Introduce logic to check and enable vIOMMU feature

 drivers/iommu/amd/Makefile              |   2 +-
 drivers/iommu/amd/amd_iommu.h           |  46 +++
 drivers/iommu/amd/amd_iommu_types.h     | 129 ++++++
 drivers/iommu/amd/amd_viommu.h          |  73 ++++
 drivers/iommu/amd/init.c                |  33 +-
 drivers/iommu/amd/iommu.c               | 225 ++++++++--
 drivers/iommu/amd/iommufd.c             | 245 +++++++++++
 drivers/iommu/amd/nested.c              |  18 +
 drivers/iommu/amd/trans_devid.c         | 317 ++++++++++++++
 drivers/iommu/amd/vfctrl_mmio.c         | 146 +++++++
 drivers/iommu/amd/viommu.c              | 529 ++++++++++++++++++++++++
 drivers/iommu/iommufd/iommufd_private.h |   1 +
 drivers/iommu/iommufd/main.c            |   3 +
 drivers/iommu/iommufd/viommu.c          | 150 +++++--
 include/linux/iommufd.h                 |  10 +-
 include/uapi/linux/iommufd.h            |  51 +++
 16 files changed, 1909 insertions(+), 69 deletions(-)
 create mode 100644 drivers/iommu/amd/amd_viommu.h
 create mode 100644 drivers/iommu/amd/trans_devid.c
 create mode 100644 drivers/iommu/amd/vfctrl_mmio.c
 create mode 100644 drivers/iommu/amd/viommu.c

-- 
2.34.1