Re: [PATCH v2] vhost: introduce mdev based hardware backend

From: Jason Wang
Date: Sun Oct 27 2019 - 23:51:10 EST

On 2019/10/28 äå9:58, Tiwei Bie wrote:
On Fri, Oct 25, 2019 at 08:16:26AM -0400, Michael S. Tsirkin wrote:
On Fri, Oct 25, 2019 at 05:54:55PM +0800, Jason Wang wrote:
On 2019/10/24 äå6:42, Jason Wang wrote:

 And we should try to avoid
putting ctrl vq and Rx/Tx vqs in the same DMA space to prevent
guests having the chance to bypass the host (e.g. QEMU) to
setup the backend accelerator directly.

That's really good point. So when "vhost" type is created, parent
should assume addr of ctrl_vq is hva.


This works for vhost but not virtio since there's no way for virtio kernel
driver to differ ctrl_vq with the rest when doing DMA map. One possible
solution is to provide DMA domain isolation between virtqueues. Then ctrl vq
can use its dedicated DMA domain for the work.
It might not be a bad idea to let the parent drivers distinguish
between virtio-mdev mdevs and vhost-mdev mdevs in ctrl-vq handling
by mdev's class id.

Yes, that should work, I have something probable better, see below.

Anyway, this could be done in the future. We can have a version first that
doesn't support ctrl_vq.
+1, thanks

Well no ctrl_vq implies either no offloads, or no XDP (since XDP needs
to disable offloads dynamically).

if (!virtio_has_feature(vi->vdev, VIRTIO_NET_F_CTRL_GUEST_OFFLOADS)
&& (virtio_has_feature(vi->vdev, VIRTIO_NET_F_GUEST_TSO4) ||
virtio_has_feature(vi->vdev, VIRTIO_NET_F_GUEST_TSO6) ||
virtio_has_feature(vi->vdev, VIRTIO_NET_F_GUEST_ECN) ||
virtio_has_feature(vi->vdev, VIRTIO_NET_F_GUEST_UFO) ||
virtio_has_feature(vi->vdev, VIRTIO_NET_F_GUEST_CSUM))) {
NL_SET_ERR_MSG_MOD(extack, "Can't set XDP while host is implementing LRO/CSUM, disable LRO/CSUM first");

neither is very attractive.

So yes ok just for development but we do need to figure out how it will
work down the road in production.
Totally agree.

So really this specific virtio net device does not support control vq,
instead it supports a different transport specific way to send commands
to device.

Some kind of extension to the transport? Ideas?

So it's basically an issue of isolating DMA domains. Maybe we can start with transport API for querying per vq DMA domain/ASID?

- for vhost-mdev, userspace can query the DMA domain for each specific virtqueue. For control vq, mdev can return id for software domain, for the rest mdev will return id of VFIO domain. Then userspace know that it should use different API for preparing the virtqueue, e.g for vq other than control vq, it should use VFIO DMA API. The control vq it should use hva instead.

- for virito-mdev, we can introduce per-vq DMA device, and route DMA mapping request for control vq back to mdev instead of the hardware. (We can wrap them into library or helpers to ease the development of vendor physical drivers).