Re: [PATCH 00/21] Control VQ support in vDPA

[Jason Wang]

On 2020/12/18 上午6:28, Michael S. Tsirkin wrote:
> On Thu, Dec 17, 2020 at 05:02:49PM +0800, Jason Wang wrote:
> > On 2020/12/17 下午3:58, Michael S. Tsirkin wrote:
> > > On Thu, Dec 17, 2020 at 11:30:18AM +0800, Jason Wang wrote:
> > > > On 2020/12/16 下午5:47, Michael S. Tsirkin wrote:
> > > > > On Wed, Dec 16, 2020 at 02:47:57PM +0800, Jason Wang wrote:
> > > > > > Hi All:
> > > > > >
> > > > > > This series tries to add the support for control virtqueue in vDPA.
> > > > > >
> > > > > > Control virtqueue is used by networking device for accepting various
> > > > > > commands from the driver. It's a must to support multiqueue and other
> > > > > > configurations.
> > > > > >
> > > > > > When used by vhost-vDPA bus driver for VM, the control virtqueue
> > > > > > should be shadowed via userspace VMM (Qemu) instead of being assigned
> > > > > > directly to Guest. This is because Qemu needs to know the device state
> > > > > > in order to start and stop device correctly (e.g for Live Migration).
> > > > > >
> > > > > > This requies to isolate the memory mapping for control virtqueue
> > > > > > presented by vhost-vDPA to prevent guest from accesing it directly.
> > > > > > To achieve this, vDPA introduce two new abstractions:
> > > > > >
> > > > > > - address space: identified through address space id (ASID) and a set
> > > > > >                     of memory mapping in maintained
> > > > > > - virtqueue group: the minimal set of virtqueues that must share an
> > > > > >                     address space
> > > > > How will this support the pretty common case where control vq
> > > > > is programmed by the kernel through the PF, and others by the VFs?
> > > > In this case, the VF parent need to provide a software control vq and decode
> > > > the command then send them to VF.
> > > But how does that tie to the address space infrastructure?
> > In this case, address space is not a must.
> That's ok, problem is I don't see how address space is going
> to work in this case at all.
>
> There's no address space there that userspace/guest can control.

The virtqueue group is mandated by parent but the association between 
virtqueue group and address space is under the control of userspace (Qemu).

A simple but common case is that:

1) Device advertise two virtqueue groups: group 0 contains RX and TX, 
group 1 contains CVQ.
2) Device advertise two address spaces

Then, for vhost-vDPA using by VM:

1) associate group 0 with as 0, group 1 with as 1 (via vhost-vDPA 
VHOST_VDPA_SET_GROUP_ASID)
2) Publish guest memory mapping via IOTLB asid 0
3) Publish control virtqueue mapping via IOTLB asid 1

Then the DMA is totally isolated in this case.

For vhost-vDPA using by DPDK or virtio-vDPA

1) associate group 0 and group 1 with as 0

since we don't need DMA isolation in this case.

In order to let it be controlled by Guest, we need extend virtio spec to 
support those concepts.

Thanks