[PATCH v5 0/2] iommu: Avoid setting C-bit for MMIO addresses

[Wei Wang]

AMD APM specifies that any pages corresponding to MMIO addresses must be
configured with the C-bit clear. The current iommu implementation sets
the C-bit on all PTEs in the IOMMU page tables. This is incorrect for PTEs
backed by MMIO, and can break PCIe peer-to-peer communication when IOVA is
used. Fix this by avoiding the C-bit for MMIO-backed mappings.

v4->v5 change:
- Handle the rare case where adjacent RAM and MMIO ranges are mapped in a
  single vfio_pin_map_dma() call. vfio_pin_map_dma() issues separate
  vfio_iommu_map() calls for RAM and MMIO, and vfio_pin_pages_remote()
  sets dma->prot to indicate MMIO mappings.
  Clear IOMMU_MMIO in vfio_pin_pages_remote() when !rsvd, since valid
  MMIO mappings always have rsvd == true.
  v4 link: https://lore.kernel.org/all/SI2PR01MB439311868FBFB7DDAD19692CDCB4A@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/

v3->v4 change:
- In the 2nd patch, moved the VM_IO check into the
  if (is_invalid_reserved_pfn(*pfn)) {} code block to avoid checking it on
  error paths.
  v3 link: https://lore.kernel.org/lkml/SI2PR01MB439337D5513729BAC122F526DCCDA@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/

v2->v3 changes:
- re-implement the iommu part based on the iommu tree which has the
  iommupt patches merged.
  v2 link: https://lore.kernel.org/lkml/SI2PR01MB439373CA7A023D8EC4C42040DCC7A@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/

v1->v2 changes:
- 1 used page_is_ram() in the AMD IOMMU driver to detect non-RAM
  addresses, avoiding changes to upper-layer callers (vfio and iommufd).
  v2 instead lets upper layers explicitly indicate MMIO mappings via the
  IOMMU_MMIO prot flag. This avoids the potential overhead of
  page_is_ram(). (suggested by Jason Gunthorpe) 
  v1 link: https://lore.kernel.org/lkml/SI2PR01MB439358422CCAABADBEB21D7CDCF0A@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/

Wei Wang (2):
  iommupt: Do not set C-bit on MMIO backed PTEs
  vfio/type1: Set IOMMU_MMIO in dma->prot for MMIO-backed addresses

 drivers/iommu/generic_pt/fmt/amdv1.h  |  3 ++-
 drivers/iommu/generic_pt/fmt/x86_64.h |  3 ++-
 drivers/vfio/vfio_iommu_type1.c       | 15 ++++++++++-----
 3 files changed, 14 insertions(+), 7 deletions(-)


base-commit: 46fe65a2c28ecf5df1a7475aba1f08ccf4c0ac1b
-- 
2.51.0