Re: [PATCH v6 09/10] iommu/riscv: Add dirty tracking support for second-stage domains

[Gong Shuai]

Hi Fangyu,

>From: Fangyu Yu <fangyu.yu@xxxxxxxxxxxxxxxxx>
>
>Add hardware dirty tracking support for second-stage (iohgatp) domains
>used in KVM VFIO device pass-through.
>
>The RISC-V IOMMU can automatically set the dirty bit in PTEs on write
>access when DC.tc.GADE is set and the hardware has AMO_HWAD capability.
>Wire this up to the iommufd dirty tracking interface:
>
>  - riscv_iommu_set_dirty_tracking(): Always enabled dirty tracking for
>    second-stage domain.
>
>  - riscv_iommu_dirty_ops: Exposes set_dirty_tracking and the generic
>    page-table read_and_clear_dirty via IOMMU_PT_DIRTY_OPS(riscv_64).
>
>  - domain_alloc_paging_flags: Assigns dirty_ops to second-stage domains
>    when AMO_HWAD is advertised in hardware capabilities.
>
>  - riscv_iommu_capable: Reports IOMMU_CAP_DIRTY_TRACKING when
>    AMO_HWAD is present.
>
>Signed-off-by: Fangyu Yu <fangyu.yu@xxxxxxxxxxxxxxxxx>
>Reviewed-by: Nutty Liu <nutty.liu@xxxxxxxxxxx>
>---
> drivers/iommu/riscv/iommu.c | 22 ++++++++++++++++++++++
> 1 file changed, 22 insertions(+)
>
>diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c
>index fc7a80393833..9f572baeca74 100644
>--- a/drivers/iommu/riscv/iommu.c
>+++ b/drivers/iommu/riscv/iommu.c
>@@ -1377,6 +1377,21 @@ static int riscv_iommu_attach_paging_domain(struct iommu_domain *iommu_domain,
> 	return 0;
> }
> 
>+static int riscv_iommu_set_dirty_tracking(struct iommu_domain *iommu_domain,
>+					  bool enable)
>+{
>+	/*
>+	 * Always enabled and the dirty bitmap is cleared prior to
>+	 * set_dirty_tracking().
>+	 */
>+	return 0;
>+}
>+
>+static const struct iommu_dirty_ops riscv_iommu_dirty_ops = {
>+	IOMMU_PT_DIRTY_OPS(riscv_64),
>+	.set_dirty_tracking = riscv_iommu_set_dirty_tracking,
>+};
>+
> static const struct iommu_domain_ops riscv_iommu_paging_domain_ops = {
> 	IOMMU_PT_DOMAIN_OPS(riscv_64),
> 	.attach_dev = riscv_iommu_attach_paging_domain,
>@@ -1439,6 +1454,9 @@ static int riscv_iommu_init_domain_s2(struct device *dev,
> 	if (domain->gscid < 0)
> 		return -ENOMEM;
> 
>+	if (iommu->caps & RISCV_IOMMU_CAPABILITIES_AMO_HWAD)
>+		domain->domain.dirty_ops = &riscv_iommu_dirty_ops;

This sets dirty_ops based on whether the allocating device's IOMMU
supports AMO_HWAD. As I understand it, the domain/HWPT is not tied
to that IOMMU instance. A later attach may target a device behind
another instance without AMO_HWAD, and the attach can still succeed
with GADE disabled, potentially causing guest page faults.

Shouldn't we therefore revalidate AMO_HWAD cap for domains with
dirty_ops and reject this combination?

FYI, both AMD and Intel have attach-time dirty-tracking checks. See
drivers/iommu/intel/iommu.c:3195 and drivers/iommu/amd/iommu.c:3221.

Thanks,
Shuai

>+
> 	return 0;
> }
> 
>@@ -1559,9 +1577,13 @@ static struct iommu_group *riscv_iommu_device_group(struct device *dev)
> 
> static bool riscv_iommu_capable(struct device *dev, enum iommu_cap cap)
> {
>+	struct riscv_iommu_device *iommu = dev_to_iommu(dev);
>+
> 	switch (cap) {
> 	case IOMMU_CAP_CACHE_COHERENCY:
> 		return dev_is_dma_coherent(dev);
>+	case IOMMU_CAP_DIRTY_TRACKING:
>+		return !!(iommu->caps & RISCV_IOMMU_CAPABILITIES_AMO_HWAD);
> 	default:
> 		return false;
> 	}
>-- 
>2.50.1