Re: [PATCH v3 0/8] vfio/mdev: IOMMU aware mediated device

[Lu Baolu]

Hi,

On 10/17/18 10:02 AM, Xu Zaibo wrote:
> Hi,
>
> On 2018/10/16 9:21, Lu Baolu wrote:
> > Hi,
> >
> > On 10/15/2018 04:50 PM, Xu Zaibo wrote:
> > > Hi,
> > >
> > > On 2018/10/15 10:48, Lu Baolu wrote:
> > > > Hi,
> > > >
> > > > On 10/13/2018 04:25 PM, Xu Zaibo wrote:
> > > > > Hi,
> > > > >
> > > > > On 2018/10/12 13:16, Lu Baolu wrote:
> > > > > > Hi,
> > > > > >
> > > > > > The Mediate Device is a framework for fine-grained physical device
> > > > > > sharing across the isolated domains. Currently the mdev framework
> > > > > > is designed to be independent of the platform IOMMU support. As the
> > > > > > result, the DMA isolation relies on the mdev parent device in a
> > > > > > vendor specific way.
> > > > > >
> > > > > > There are several cases where a mediated device could be protected
> > > > > > and isolated by the platform IOMMU. For example, Intel vt-d rev3.0
> > > > > > [1] introduces a new translation mode called 'scalable mode', which
> > > > > > enables PASID-granular translations. The vt-d scalable mode is the
> > > > > > key ingredient for Scalable I/O Virtualization [2] [3] which allows
> > > > > > sharing a device in minimal possible granularity (ADI - Assignable
> > > > > > Device Interface).
> > > > > >
> > > > > > A mediated device backed by an ADI could be protected and isolated
> > > > > > by the IOMMU since 1) the parent device supports tagging an unique
> > > > > > PASID to all DMA traffic out of the mediated device; and 2) the DMA
> > > > > > translation unit (IOMMU) supports the PASID granular translation.
> > > > > > We can apply IOMMU protection and isolation to this kind of devices
> > > > > > just as what we are doing with an assignable PCI device.
> > > > > >
> > > > > > In order to distinguish the IOMMU-capable mediated devices from those
> > > > > > which still need to rely on parent devices, this patch set adds two
> > > > > > new members in struct mdev_device.
> > > > > >
> > > > > > * iommu_device
> > > > > > ÂÂ - This, if set, indicates that the mediated device could
> > > > > > ÂÂÂÂ be fully isolated and protected by IOMMU via attaching
> > > > > > ÂÂÂÂ an iommu domain to this device. If empty, it indicates
> > > > > > ÂÂÂÂ using vendor defined isolation.
> > > > > >
> > > > > > * iommu_domain
> > > > > > ÂÂ - This is a place holder for an iommu domain. A domain
> > > > > > ÂÂÂÂ could be store here for later use once it has been
> > > > > > ÂÂÂÂ attached to the iommu_device of this mdev.
> > > > > >
> > > > > > Below helpers are added to set and get above iommu device
> > > > > > and iommu domain pointers in mdev core implementation.
> > > > > >
> > > > > > * mdev_set/get_iommu_device(dev, iommu_device)
> > > > > > ÂÂ - Set or get the iommu device which represents this mdev
> > > > > > ÂÂÂÂ in IOMMU's device scope. Drivers don't need to set the
> > > > > > ÂÂÂÂ iommu device if it uses vendor defined isolation.
> > > > > >
> > > > > > * mdev_set/get_iommu_domain(domain)
> > > > > > ÂÂ - A iommu domain which has been attached to the iommu
> > > > > > ÂÂÂÂ device in order to protect and isolate the mediated
> > > > > > ÂÂÂÂ device will be kept in the mdev data structure and
> > > > > > ÂÂÂÂ could be retrieved later.
> > > > > >
> > > > > > The mdev parent device driver could opt-in that the mdev could be
> > > > > > fully isolated and protected by the IOMMU when the mdev is being
> > > > > > created by invoking mdev_set_iommu_device() in its @create().
> > > > > I just cannot understand here, how to get an iommu_device while I 
> > > > > create mediated
> > > > > device in my parent device driver?
> > > >
> > > > When you are creating an mdev in your parent driver, you should know
> > > > which PCI device this mdev belonging to.
> > >
> > > So, generally, I can set the parent device as mdev's iommu_device?
> > > If that, however, Mdev already holds its parent device. So, I just 
> > > figure what
> > > differences between Mdev's parent device and iommu_device are.
> > > > > And why not reuse the device of MDEV instread of adding a new 
> > > > > device here?
> > > >
> > > > iommu_device in the mdev_device structure represents the PCI device
> > > > that represents this mdev in iommu's device scope. IOMMU is only aware
> > > > of pci devices, it's not aware of mdev device.
> > >
> > > Could I understand like that: IOMMU can be aware of the parent device 
> > > of Mdev?
> > > And more, I am doubting the necessary of iommu_device in Mdev.
> >
> > The "mdev parent device" and "mdev iommu device" are different although
> > they might be the same in practice. "mdev parent device" represents the
> > device who created the mdev. "mdev iommu device" represents the device
> > who shares the device context entry in iommu tables.
> >
> > "mdev iommu device" is always a PCI/PCIe device since IOMMU always use
> > source id (bus:dev:func) to walk the device context table. But there is
> > no limitation on who can create an mdev, right?
> Actually, I am not sure.
>
> My understanding:
> The DMA address will be issued by the parent device with PASID or 
> something like that to IOMMU
> facilities. However, the translation units such as iommu ïPASID/page 
> .etxïtables are from another
> device node. I cannot figure out how to control this in hardware level, 
> or whether there will be
> conflicts between the DMA transation of iommu_device and parent device.


Yes. That's the reason why these two devices are same in practice. But
conceptually, they might be different.

Best regards,
Lu Baolu