Re: [PATCH RFC 00/15] Add VFIO mediated device support and IMS support for the idxd driver.

[Jason Gunthorpe]

On Sun, Apr 26, 2020 at 05:18:59AM +0000, Tian, Kevin wrote:

> > > I think providing an unified abstraction to userspace is also important,
> > > which is what VFIO provides today. The merit of using one set of VFIO
> > > API to manage all kinds of mediated devices and VF devices is a major
> > > gain. Instead, inventing a new vDPA-like interface for every Scalable-IOV
> > > or equivalent device is just overkill and doesn't scale. Also the actual
> > > emulation code in idxd driver is actually small, if putting aside the PCI
> > > config space part for which I already explained most logic could be shared
> > > between mdev device drivers.
> > 
> > If it was just config space you might have an argument, VFIO already
> > does some config space mangling, but emulating BAR space is out of
> > scope of VFIO, IMHO.
> 
> out of scope of vfio-pci, but in scope of vfio-mdev. btw I feel that most
> of your objections are actually related to the general idea of
> vfio-mdev.

There have been several abusive proposals of vfio-mdev, everything
from a way to create device drivers to this kind of generic emulation
framework.

> Scalable IOV just uses PASID to harden DMA isolation in mediated
> pass-through usage which vfio-mdev enables. Then are you just opposing
> the whole vfio-mdev? If not, I'm curious about the criteria in your mind 
> about when using vfio-mdev is good...

It is appropriate when non-PCI standard techniques are needed to do
raw device assignment, just like VFIO.

Basically if vfio-pci is already doing it then it seems reasonable
that vfio-mdev should do the same. This mission creep where vfio-mdev
gains functionality far beyond VFIO is the problem.

> technically Scalable IOV is definitely different from SR-IOV. It's 
> simpler in hardware. And we're not emulating SR-IOV. The point
> is just in usage-wise we want to present a consistent user 
> experience just like passing through a PCI endpoint (PF or VF) device
> through vfio eco-system, including various userspace VMMs (Qemu,
> firecracker, rust-vmm, etc.), middleware (Libvirt), and higher level 
> management stacks. 

Yes, I understand your desire, but at the same time we have not been
doing device emulation in the kernel. You should at least be
forthwright about that major change in the cover letters/etc.
 
> > The only thing we get out of this is someone doesn't have to write a
> > idxd emulation driver in qemu, instead they have to write it in the
> > kernel. I don't see how that is a win for the ecosystem.
> 
> No. The clear win is on leveraging classic VFIO iommu and its eco-system
> as explained above.

vdpa had no problem implementing iommu support without VFIO. This was
their original argument too, it turned out to be erroneous.

Jason