Re: [PATCH v6 00/20] Add VFIO mediated device support and DEV-MSI support for the idxd driver

[Jason Gunthorpe]

On Mon, Jun 07, 2021 at 11:13:04AM -0700, Dave Jiang wrote:

> So in step 1, we 'tag' the wq to be dedicated to guest usage and put the
> hardware wq into enable state. For a dedicated mode wq, we can definitely
> just register directly and skip the mdev step. For a shared wq mode, we can
> have multiple mdev running on top of a single wq. So we need some way to
> create more mdevs. We can either go with the existing established creation
> path by mdev, or invent something custom for the driver as Jason suggested
> to accomodate additional virtual devices for guests. We implemented the mdev
> path originally with consideration of mdev is established and has a known
> interface already.

It sounds like you could just as easially have a 'create new vfio'
file under the idxd sysfs.. Especially since you already have a bus
and dynamic vfio specific things being created on this bus.

Have you gone over this with Dan?

> I think things become more complicated when we go from a dedicated wq to
> shared wq where the relationship of wq : mdev is 1 : 1 goes to 1 : N. Also
> needing to keep a consistent user config experience is desired, especially
> we already have such behavior since kernel 5.6 for host usages. So we really
> need try to avoid doing wq configuration differently just for "mdev" wqs. In
> the case suggested above, we basically just flipped the configuration steps.
> Mdev is first created through mdev sysfs interface. And then the device
> paramters are configured. Where for us, we configure the device parameter
> first, and then create the mdev. But in the end, it's still the hybrid mdev
> setup right?

So you don't even use mdev to configure anything? Yuk.

Jason