Re: [PATCH v8 7/7] iommu/riscv: Paging domain support

[Jason Gunthorpe]

On Fri, Jul 19, 2024 at 02:33:13PM +0800, Zong Li wrote:

> > +static void riscv_iommu_free_paging_domain(struct iommu_domain *iommu_domain)
> > +{
> > +       struct riscv_iommu_domain *domain = iommu_domain_to_riscv(iommu_domain);
> > +       const unsigned long pfn = virt_to_pfn(domain->pgd_root);
> > +
> > +       WARN_ON(!list_empty(&domain->bonds));
> 
> Hi Tomasz,
> I recently hit the issue here when I removed a device. I think we need
> to unlink the device's bond in domain before releasing domain.
> Do you mind pick the following modification in this patch? Thanks.
> 
> diff --git a/drivers/iommu/riscv/iommu.c b/drivers/iommu/riscv/iommu.c
> index 859cdd455576..f0707147e706 100644
> --- a/drivers/iommu/riscv/iommu.c
> +++ b/drivers/iommu/riscv/iommu.c
> @@ -1541,6 +1541,8 @@ static void riscv_iommu_release_device(struct device *dev)
>  {
>         struct riscv_iommu_info *info = dev_iommu_priv_get(dev);
> 
> +       riscv_iommu_bond_unlink(info->domain, dev);
> +
>         synchronize_rcu();
>         kfree(info);
>  }

That does not seem the right place to put that, or something else has
gone wrong becuase:

> >  static const struct iommu_ops riscv_iommu_ops = {
> > +       .pgsize_bitmap = SZ_4K,
> >         .of_xlate = riscv_iommu_of_xlate,
> >         .identity_domain = &riscv_iommu_identity_domain,
> >         .blocked_domain = &riscv_iommu_blocking_domain,
> >         .release_domain = &riscv_iommu_blocking_domain,

The above use of release_domain will attach to the blocking domain
prior to calling release_device and when doing so everything about any
previous domain should have been fully cleaned up.

So info->domain should be the blocking domain during release_device
(if not that is another bug)

And attaching the blocking domain should have cleared the
domain->bonds of any prior paging domain.

Jason