Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs

[Kirti Wankhede]

On 5/26/2021 4:22 AM, Alex Williamson wrote:
> On Wed, 26 May 2021 00:56:30 +0530
> Kirti Wankhede <kwankhede@xxxxxxxxxx> wrote:
>
> > On 5/25/2021 5:07 AM, Jason Gunthorpe wrote:
> > > On Mon, May 24, 2021 at 05:52:58PM +1000, David Gibson wrote:
> > >    
> > > > > > I don't really see a semantic distinction between "always one-device
> > > > > > groups" and "groups don't matter".  Really the only way you can afford
> > > > > > to not care about groups is if they're singletons.
> > > > >
> > > > > The kernel driver under the mdev may not be in an "always one-device"
> > > > > group.
> > > >
> > > > I don't really understand what you mean by that.
> > >
> > > I mean the group of the mdev's actual DMA device may have multiple
> > > things in it.
> > >      
> > > > > It is a kernel driver so the only thing we know and care about is that
> > > > > all devices in the HW group are bound to kernel drivers.
> > > > >
> > > > > The vfio device that spawns from this kernel driver is really a
> > > > > "groups don't matter" vfio device because at the IOMMU layer it should
> > > > > be riding on the physical group of the kernel driver.  At the VFIO
> > > > > layer we no longer care about the group abstraction because the system
> > > > > guarentees isolation in some other way.
> > > >
> > > > Uh.. I don't really know how mdevs are isolated from each other.  I
> > > > thought it was because the physical device providing the mdevs
> > > > effectively had an internal IOMMU (or at least DMA permissioning) to
> > > > isolate the mdevs, even though the physical device may not be fully
> > > > isolated.
> > > >
> > > > In that case the virtual mdev is effectively in a singleton group,
> > > > which is different from the group of its parent device.
> > >    
> >
> > That's correct.
> >
> > > That is one way to view it, but it means creating a whole group
> > > infrastructure and abusing the IOMMU stack just to create this
> > > nonsense fiction.
> >
> > I really didn't get how this abuse the IOMMU stack.
> > mdev can be used in 3 different ways:
> > 1. non-iommu backed mdev devices where mdev vendor driver takes care to
> > DMA map (iommu_map) and isolation is through device hardware internal
> > MMU. Here vfio_iommu_type1 module provides a way to validate and pin
> > pages required by mdev device for DMA mapping. Then IOMMU mapping is
> > done by mdev vendor driver which is owner driver of physical device.
> >
> > 2. iommu backed mdev devices for SRIOV where mdev device is created per
> > VF (mdev device == VF device) then that mdev device has same iommu
> > protection scope as VF associated to it. Here mdev device is virtual
> > device which uses features of mdev and represents underlying VF device,
> > same as vfio-pci but with additional mdev features.
>
> What features would those be?  There are no mdev specific parts of the
> vfio uAPI.
>
> The mdev device is a virtual device, by why it it virtual in this case?
> Aren't we effectively assigning the VF itself (mdev device == VF device)
> with a bunch of extra support code to fill in the gaps of the VF
> implementing the complete device model in hardware?
>
> We're effectively creating this virtual device, creating a fake IOMMU
> group, and trying to create this association of this virtual device to
> the real VF in order to shoehorn it into the mdev model.  What do we
> get from that model other than lifecycle management (ie. type selection)
> and re-use of a bunch of code from the driver supporting the 1) model
> above?

Yes, the lifecycle management which is in mdev is not in vfio-pci variant.

> This specific model seems better served by a device specific peer
> driver to vfio-pci (ie. a "vfio-pci variant").  You effectively already
> have the code for this driver, it's just in the format of an mdev
> driver rather than a vfio "bus driver".  The work Jason references
> relative to Max aims to make these kinds of drivers easier to implement
> through re-use of vfio-pci code.
>
> There are certainly other solutions we could come up with for selecting
> a specific device type for a vfio-pci variant driver to implement other
> than pretending this model actually belongs in mdev, right?  Thanks,

Sure and would like to see type selection mechanism to be implemented in 
vfio-pci variant.

Thanks,
Kirti