Re: [RFC PATCH 3/4] iommu: Preallocate iommu group when probing devices

[Lu Baolu]

Hi Robin,

On 1/21/20 8:45 PM, Robin Murphy wrote:
> On 19/01/2020 6:29 am, Lu Baolu wrote:
> > Hi Joerg,
> >
> > On 1/17/20 6:21 PM, Joerg Roedel wrote:
> > > On Wed, Jan 01, 2020 at 01:26:47PM +0800, Lu Baolu wrote:
> > > > This splits iommu group allocation from adding devices. This makes
> > > > it possible to determine the default domain type for each group as
> > > > all devices belonging to the group have been determined.
> > >
> > > I think its better to keep group allocation as it is and just defer
> > > default domain allocation after each device is in its group. But take
> >
> > I tried defering default domain allocation, but it seems not possible.
> >
> > The call path of adding devices into their groups:
> >
> > iommu_probe_device
> > -> ops->add_device(dev)
> > ÂÂÂ -> (iommu vendor driver) iommu_group_get_for_dev(dev)
> >
> > After doing this, the vendor driver will get the default domain and
> > apply dma_ops according to the domain type. If we defer the domain
> > allocation, they will get a NULL default domain and cause panic in
> > the vendor driver.
> >
> > Any suggestions?
>
> https://lore.kernel.org/linux-iommu/6dbbfc10-3247-744c-ae8d-443a336e0c50@xxxxxxxxxxxxxxx/ 
>
>
> Haven't we been here before? ;)
>
> Since we can't (safely or reasonably) change a group's default domain 
> after ops->add_device() has returned, and in general it gets impractical 
> to evaluate "all device in a group" once you look beyond &pci_bus_type 
> (or consider hotplug as mentioned), then AFAICS there's no reasonable 
> way to get away from the default domain type being defined by the first 
> device to attach.

Yes, agreed.

> But in practice it's hardly a problem anyway - if 
> every device in a given group requests the same domain type then it 
> doesn't matter which comes first, and if they don't then we ultimately 
> end up with an impossible set of constraints, so are doomed to do the 
> 'wrong' thing regardless.

The third case is, for example, three devices A, B, C in a group. The
first device A is neutral about which type of default domain type is
used. So the iommu framework will use a static default domain. But the
device B requires to use a specific one which is different from the
default. Currently, this is handled in the vendor iommu driver and one
motivation of this patch set is to handle this in the generic layer.

> Thus unless anyone wants to start redefining the whole group concept to 
> separate the notions of ID aliasing and peer-to-peer isolation (which 
> still wouldn't necessarily help), I think this user override effectively 
> boils down to just another flavour of iommu_request_*_for_dev(), and as 
> such comes right back to the "just pass the bloody device to 
> ops->domain_alloc() and resolve everything up-front" argument.
>
> Robin.

Best regards,
baolu