Re: [PATCH V2 2/3] iommu: of: Ignore all errors except EPROBE_DEFER

[Laurent Pinchart]

Hi Sricharan,

On Thursday 18 May 2017 17:26:14 Sricharan R wrote:
> On 5/18/2017 4:09 PM, Laurent Pinchart wrote:
> > On Thursday 18 May 2017 15:37:09 Sricharan R wrote:
> >> While deferring the probe of IOMMU masters,
> >> xlate and add_device callback can pass back error values
> >> like -ENODEV, which means IOMMU cannot be connected
> >> with that master for real reasons. So rather than
> >> killing the master's probe for such errors, just
> >> ignore the errors and let the master work without
> >> an IOMMU.
> > 
> > I don't think this is a good idea. Why should we allow IOMMU drivers to
> > return an error if we're always going to ignore the error value ? That
> > will lead to drivers implementing slightly different behaviours, which
> > will be painful the day we'll need to start acting based on the error
> > value.
> 
> The of_iommu_configure interface, before this series, was returning either
> correct 'iommu_ops' or NULL. Also there was no return value from
> of_dma_configure which calls of_iommu_configure. This means that if we block
> only -ENODEV now and let the other errors, the probe of the master devices
> can be killed for reasons apart from deferring. This would be a change in
> behavior introduced. All of xlate, add_device, of_pci_map_rid and others
> can return values apart from -ENODEV. So was thinking that restoring the
> old behavior, except for returning EPROBE_DEFER was the better thing to do
> ?

We went from a situation where of_iommu_configure() could return either valid 
operations in the case the device was to be handled by the IOMMU or NULL 
otherwise, to a situation where we needed a third option for probe deferral. 
The way we've done this, through error pointers, allows lots of other errors 
to be returned as well from the of_xlate and add_device operations.

There is currently no use for returning error codes other than -EPROBE_DEFFER 
from of_iommu_configure(), so your proposal is to block errors returned from 
the of_xlate and add_device operations inside of_iommu_configure(). My point 
is that, by doing so, we allow IOMMU drivers to return random error codes that 
are then ignored. I believe this can cause problems in the future when we will 
need to extend the API and standardize more error codes, as by then IOMMU 
drivers will return random errors (they actually do so already to some 
extent).

For of_xlate I agree with you to some extent. v4.11 just checked whether 
of_xlate succeeded or not, and just didn't use the IOMMU in case it failed. 
The exact error value was ignored, and drivers already return random errors. 
Going back to the v4.11 behaviour is what we need to do in the short-term, 
even if I believe we should standardize the error values returned from 
of_xlate after v4.12.

For add_device, however, the situation is a bit different. The add_device 
operation is called from the IOMMU bus notifier, and the -ENODEV error is 
ignored by add_iommu_group(). Any other error will cause bus_set_iommu() to 
fail, which makes IOMMU probing fail for the drivers that check the return 
value of bus_set_iommu() (some of them don't :-/).

Fixing all this properly requires standardizing the error codes, and going 
through the existing IOMMU drivers to comply with the standardized behaviour. 
While this shouldn't be very difficult, it's likely not material for a v4.12-
rc fix. We will thus likely need to merge this patch (or something very 
similar to it), but I'd really like to see this fixed properly for v4.13.

> > At the very least, if you want to give a specific meaning to -ENODEV, you
> > should check for that value specifically and not ignore all errors other
> > than -EPROBE_DEFER. You also need to document the meaning of the error
> > value. This can be done in the documentation of the of_xlate operation in
> > include/linux/iommu.h:
> > 
> > diff --git a/include/linux/iommu.h b/include/linux/iommu.h
> > index 2cb54adc4a33..6ba553e7384a 100644
> > --- a/include/linux/iommu.h
> > +++ b/include/linux/iommu.h
> > @@ -181,7 +181,6 @@ struct iommu_resv_region {
> >   * @domain_window_disable: Disable a particular window for a domain
> >   * @domain_set_windows: Set the number of windows for a domain
> >   * @domain_get_windows: Return the number of windows for a domain
> > - * @of_xlate: add OF master IDs to iommu grouping
> >   * @pgsize_bitmap: bitmap of all possible supported page sizes
> >   */
> >  
> >  struct iommu_ops {
> > @@ -224,6 +223,11 @@ struct iommu_ops {
> >  	/* Get the number of windows per domain */
> >  	u32 (*domain_get_windows)(struct iommu_domain *domain);
> > 
> > +	/**
> > +	 * @of_xlate:
> > +	 *
> > +	 * Add OF master IDs to iommu grouping.
> > +	 */
> >  	int (*of_xlate)(struct device *dev, struct of_phandle_args *args);
> >  	
> >  	unsigned long pgsize_bitmap;
> > 
> > And add documentation for the error codes there.
> > 
> > If you want to ignore some errors returned from the add_device operation
> > you should document it similarly, and in particular document which error
> > check(s) need to be performed by of_xlate and which are the
> > responsibility of add_device.
> > 
> >> Fixes: 7b07cbefb68d ("iommu: of: Handle IOMMU lookup failure with
> >> deferred probing or error")
> >> Reported-by: Geert Uytterhoeven <geert@xxxxxxxxxxxxxx>
> >> Tested-by: Magnus Damn <magnus.damn@xxxxxxxxx>
> >> Signed-off-by: Sricharan R <sricharan@xxxxxxxxxxxxxx>
> >> ---
> >> [V2] Corrected spelling/case in commit log
> >> 
> >>  drivers/iommu/of_iommu.c | 6 ++++++
> >>  1 file changed, 6 insertions(+)
> >> 
> >> diff --git a/drivers/iommu/of_iommu.c b/drivers/iommu/of_iommu.c
> >> index e6e9bec..f0d22c0 100644
> >> --- a/drivers/iommu/of_iommu.c
> >> +++ b/drivers/iommu/of_iommu.c
> >> @@ -237,6 +237,12 @@ const struct iommu_ops *of_iommu_configure(struct
> >> device *dev, ops = ERR_PTR(err);
> >>  	}
> >> 
> >> +	/* Ignore all other errors apart from EPROBE_DEFER */
> >> +	if (IS_ERR(ops) && (PTR_ERR(ops) != -EPROBE_DEFER)) {
> >> +		dev_info(dev, "Adding to IOMMU failed: %ld\n", PTR_ERR(ops));
> >> +		ops = NULL;
> >> +	}
> >> +
> >>  	return ops;
> >>  }

-- 
Regards,

Laurent Pinchart