[PATCH v11 00/16] iommu: Prepare to deliver page faults to user space

[Lu Baolu]

When a user-managed page table is attached to an IOMMU, it is necessary
to deliver IO page faults to user space so that they can be handled
appropriately. One use case for this is nested translation, which is
currently being discussed in the mailing list.

I have posted a RFC series [1] that describes the implementation of
delivering page faults to user space through IOMMUFD. This series has
received several comments on the IOMMU refactoring, which I am trying to
address in this series.

The major refactoring includes:

- [PATCH 01 ~ 04] Move include/uapi/linux/iommu.h to
  include/linux/iommu.h. Remove the unrecoverable fault data definition.
- [PATCH 05 ~ 06] Remove iommu_[un]register_device_fault_handler().
- [PATCH 07 ~ 10] Separate SVA and IOPF. Make IOPF a generic page fault
  handling framework.
- [PATCH 11 ~ 16] Improve iopf framework.

This is also available at github [2].

[1] https://lore.kernel.org/linux-iommu/20230530053724.232765-1-baolu.lu@xxxxxxxxxxxxxxx/
[2] https://github.com/LuBaolu/intel-iommu/commits/preparatory-io-pgfault-delivery-v11

Change log:
v11:
 - Cleanup IOMMU_PAGE_RESP_PASID_VALID flag bit.
 - Cleanup code comments.

v10: https://lore.kernel.org/linux-iommu/20240122054308.23901-1-baolu.lu@xxxxxxxxxxxxxxx/
  - Make iopf_group_response() return void, as nobody can do anything
    with the failure.
  - Make iommu_report_device_fault() automatically respond to
    unhandleable faults and change its return type to void.
  - PATCH 01 ~ 14 are in good shapes now.

v9: https://lore.kernel.org/linux-iommu/20231220012332.168188-1-baolu.lu@xxxxxxxxxxxxxxx/
  - Protecting the assignment of dev->iommu->fault_param with RCU.
  - Extending the fault parameter's lifetime to the entire path of iopf
    handling.
  - Since iopf_queue_flush_dev() can only be called before
    iopf_queue_remove_device(), there's no need to hold a reference
    count.
  - Improve iopf_queue_remove_device() as per Jason's comments on the
    device removal sequence from the iopf queue. This will likely
    require changes to the iommu drivers, which are supposed to be
    addressed in separate series.
  - Track the iopf_group as a whole instead of the last fault within the
    group to simplify the fault report and response paths.
  - PATCH 01 ~ 11 are in good shapes now.

v8: https://lore.kernel.org/linux-iommu/20231207064308.313316-1-baolu.lu@xxxxxxxxxxxxxxx/
 - Drop PATCH 12/12 as it is no longer necessary to drain page requests
   page requests during PASID translation changes.
 - Separate PATCH 11/12 into two distinct patches. The first patch
   refines locking scheme for protecting per-device fault data, while
   the second patch replaces mutex with RCU to enhance locking
   efficiency.
 - PATCH 01 ~ 10 are in good shapes now.

v7: https://lore.kernel.org/linux-iommu/20231115030226.16700-1-baolu.lu@xxxxxxxxxxxxxxx/
 - Rebase to v6.7-rc1.
 - Export iopf_group_response() for global use.
 - Release lock when calling iopf handler.
 - The whole series has been verified to work for SVA case on Intel
   platforms by Zhao Yan. Add her Tested-by to affected patches.

v6: https://lore.kernel.org/linux-iommu/20230928042734.16134-1-baolu.lu@xxxxxxxxxxxxxxx/
 - [PATCH 09/12] Check IS_ERR() against the iommu domain. [Jingqi/Jason]
 - [PATCH 12/12] Rename the comments and name of iopf_queue_flush_dev(),
   no functionality changes. [Kevin]
 - All patches rebased on the latest iommu/core branch.

v5: https://lore.kernel.org/linux-iommu/20230914085638.17307-1-baolu.lu@xxxxxxxxxxxxxxx/
 - Consolidate per-device fault data management. (New patch 11)
 - Improve iopf_queue_flush_dev(). (New patch 12)

v4: https://lore.kernel.org/linux-iommu/20230825023026.132919-1-baolu.lu@xxxxxxxxxxxxxxx/
 - Merge iommu_fault_event and iopf_fault. They are duplicate.
 - Move iommu_report_device_fault() and iommu_page_response() to
   io-pgfault.c.
 - Move iommu_sva_domain_alloc() to iommu-sva.c.
 - Add group->domain and use it directly in sva fault handler.
 - Misc code refactoring and refining.

v3: https://lore.kernel.org/linux-iommu/20230817234047.195194-1-baolu.lu@xxxxxxxxxxxxxxx/
 - Convert the fault data structures from uAPI to kAPI.
 - Merge iopf_device_param into iommu_fault_param.
 - Add debugging on domain lifetime for iopf.
 - Remove patch "iommu: Change the return value of dev_iommu_get()".
 - Remove patch "iommu: Add helper to set iopf handler for domain".
 - Misc code refactoring and refining.

v2: https://lore.kernel.org/linux-iommu/20230727054837.147050-1-baolu.lu@xxxxxxxxxxxxxxx/
 - Remove unrecoverable fault data definition as suggested by Kevin.
 - Drop the per-device fault cookie code considering that doesn't make
   much sense for SVA.
 - Make the IOMMU page fault handling framework generic. So that it can
   available for use cases other than SVA.

v1: https://lore.kernel.org/linux-iommu/20230711010642.19707-1-baolu.lu@xxxxxxxxxxxxxxx/

Lu Baolu (16):
  iommu: Move iommu fault data to linux/iommu.h
  iommu/arm-smmu-v3: Remove unrecoverable faults reporting
  iommu: Remove unrecoverable fault data
  iommu: Cleanup iopf data structure definitions
  iommu: Merge iopf_device_param into iommu_fault_param
  iommu: Remove iommu_[un]register_device_fault_handler()
  iommu: Merge iommu_fault_event and iopf_fault
  iommu: Prepare for separating SVA and IOPF
  iommu: Make iommu_queue_iopf() more generic
  iommu: Separate SVA and IOPF
  iommu: Refine locking for per-device fault data management
  iommu: Use refcount for fault data access
  iommu: Improve iopf_queue_remove_device()
  iommu: Track iopf group instead of last fault
  iommu: Make iopf_group_response() return void
  iommu: Make iommu_report_device_fault() return void

 include/linux/iommu.h                         | 262 +++++++---
 drivers/iommu/intel/iommu.h                   |   4 +-
 drivers/iommu/iommu-sva.h                     |  71 ---
 include/uapi/linux/iommu.h                    | 161 ------
 .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c   |  14 +-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c   | 103 ++--
 drivers/iommu/intel/iommu.c                   |  28 +-
 drivers/iommu/intel/svm.c                     |  41 +-
 drivers/iommu/io-pgfault.c                    | 473 ++++++++++--------
 drivers/iommu/iommu-sva.c                     |  71 ++-
 drivers/iommu/iommu.c                         | 233 ---------
 MAINTAINERS                                   |   1 -
 drivers/iommu/Kconfig                         |   4 +
 drivers/iommu/Makefile                        |   3 +-
 drivers/iommu/intel/Kconfig                   |   1 +
 15 files changed, 583 insertions(+), 887 deletions(-)
 delete mode 100644 drivers/iommu/iommu-sva.h
 delete mode 100644 include/uapi/linux/iommu.h

-- 
2.34.1