Re: [PATCH 6/7] iommu: Use right way to retrieve iommu_ops

[Lu Baolu]

On 1/25/22 8:20 AM, Robin Murphy wrote:
> On 2022-01-24 07:11, Lu Baolu wrote:
> > The common iommu_ops is hooked to both device and domain. When a helper
> > has both device and domain pointer, the way to get the iommu_ops looks
> > messy in iommu core. This sorts out the way to get iommu_ops. The device
> > related helpers go through device pointer, while the domain related ones
> > go through domain pointer.
> >
> > Signed-off-by: Lu Baolu <baolu.lu@xxxxxxxxxxxxxxx>
> > ---
> >   include/linux/iommu.h |  8 ++++++++
> >   drivers/iommu/iommu.c | 25 ++++++++++++++-----------
> >   2 files changed, 22 insertions(+), 11 deletions(-)
> >
> > diff --git a/include/linux/iommu.h b/include/linux/iommu.h
> > index aa5486243892..111b3e9c79bb 100644
> > --- a/include/linux/iommu.h
> > +++ b/include/linux/iommu.h
> > @@ -385,6 +385,14 @@ static inline void iommu_iotlb_gather_init(struct 
> > iommu_iotlb_gather *gather)
> >       };
> >   }
> > +static inline const struct iommu_ops *dev_iommu_ops_get(struct device 
> > *dev)
> > +{
> > +    if (dev && dev->iommu && dev->iommu->iommu_dev)
> > +        return dev->iommu->iommu_dev->ops;
> > +
> > +    return NULL;
>
> This probably warrants at least a WARN, but it's arguable to just assume 
> that valid ops must be installed if iommu_probe_device() has succeeded. 
> The device ops are essentially for internal use within the IOMMU 
> subsystem itself, so we should be able to trust ourselves not to misuse 
> the helper.

I agree that we could add a WARN() here. The expectation is that every
device going through the IOMMU interfaces or helpers should have been
probed by iommu_probe_device().

> > +}
> > +
> >   #define IOMMU_BUS_NOTIFY_PRIORITY        0
> >   #define IOMMU_GROUP_NOTIFY_ADD_DEVICE        1 /* Device added */
> >   #define IOMMU_GROUP_NOTIFY_DEL_DEVICE        2 /* Pre Device removed */
> > diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
> > index 5230c6d90ece..6631e2ea44df 100644
> > --- a/drivers/iommu/iommu.c
> > +++ b/drivers/iommu/iommu.c
> > @@ -764,6 +764,7 @@ EXPORT_SYMBOL_GPL(iommu_group_set_name);
> >   static int iommu_create_device_direct_mappings(struct iommu_group 
> > *group,
> >                              struct device *dev)
> >   {
> > +    const struct iommu_ops *ops = dev_iommu_ops_get(dev);
> >       struct iommu_domain *domain = group->default_domain;
> >       struct iommu_resv_region *entry;
> >       struct list_head mappings;
> > @@ -785,8 +786,8 @@ static int 
> > iommu_create_device_direct_mappings(struct iommu_group *group,
> >           dma_addr_t start, end, addr;
> >           size_t map_size = 0;
> > -        if (domain->ops->apply_resv_region)
> > -            domain->ops->apply_resv_region(dev, domain, entry);
> > +        if (ops->apply_resv_region)
> > +            ops->apply_resv_region(dev, domain, entry);
>
> Strictly I think this was a domain op, as it was about reserving the 
> IOVA range in the given DMA domain. Also taking the domain as an 
> argument is a bit of a giveaway. However it's now just dead code either 
> way since there are no remaining implementations, and no reason for any 
> new ones.

This callback is a dead code. I will cleanup it.

$ git grep apply_resv_region
drivers/iommu/iommu.c:          if (ops->apply_resv_region)
drivers/iommu/iommu.c:                  ops->apply_resv_region(dev, 
domain, entry);
include/linux/iommu.h: * @apply_resv_region: Temporary helper call-back 
for iova reserved ranges
include/linux/iommu.h:  void (*apply_resv_region)(struct device *dev,

> >           start = ALIGN(entry->start, pg_size);
> >           end   = ALIGN(entry->start + entry->length, pg_size);
> > @@ -831,8 +832,10 @@ static int 
> > iommu_create_device_direct_mappings(struct iommu_group *group,
> >   static bool iommu_is_attach_deferred(struct iommu_domain *domain,
> >                        struct device *dev)
> >   {
> > -    if (domain->ops->is_attach_deferred)
> > -        return domain->ops->is_attach_deferred(domain, dev);
> > +    const struct iommu_ops *ops = dev_iommu_ops_get(dev);
> > +
> > +    if (ops->is_attach_deferred)
> > +        return ops->is_attach_deferred(domain, dev);
>
> Similarly if this takes a domain as its first argument then it's de 
> facto a domain method. However, I'd concur that logically it *is* a 
> device op, so let's drop that (unused) domain argument if we're cleaning 
> up.
>
> Maybe there's even an argument for factoring this out to a standard flag 
> in dev_iommu rather than an op at all?

Make it part of dev_iommu looks more attractive. Let me check how many
efforts will it take. If a lot of changes required, maybe we can remove
@domain in this series and then switch it to a dev_iommu flag in a
separated series.

> The others covered here look OK - we can blame PCI for page response 
> being weirdly device-centric - however could we also convert all the 
> feasible instances of dev->bus->iommu_ops to dev_iommu_ops() as well?

Sure.

> (Subtly implying that I'm also not a fan of having "_get" in the name 
> for a non-refcounted lookup...) Obviously iommu_probe_device() and 
> iommmu_domain_alloc() still need bus ops at this point, but I'm working 
> on that... :)

Thanks and glad to know that.

> Thanks,
> Robin.

Best regards,
baolu