Re: [RFC v2 PATCH] vdpa_sim: implement .reset_map support

From: Si-Wei Liu
Date: Thu Oct 19 2023 - 21:09:08 EST




On 10/19/2023 2:29 AM, Stefano Garzarella wrote:
On Wed, Oct 18, 2023 at 04:47:48PM -0700, Si-Wei Liu wrote:


On 10/18/2023 1:05 AM, Stefano Garzarella wrote:
On Tue, Oct 17, 2023 at 10:11:33PM -0700, Si-Wei Liu wrote:
RFC only. Not tested on vdpa-sim-blk with user virtual address.
Works fine with vdpa-sim-net which uses physical address to map.

This patch is based on top of [1].

[1] https://lore.kernel.org/virtualization/1696928580-7520-1-git-send-email-si-wei.liu@xxxxxxxxxx/

Signed-off-by: Si-Wei Liu <si-wei.liu@xxxxxxxxxx>

---
RFC v2:
 - initialize iotlb to passthrough mode in device add

I tested this version and I didn't see any issue ;-)
Great, thank you so much for your help on testing my patch, Stefano!

You're welcome :-)

Just for my own interest/curiosity, currently there's no vhost-vdpa backend client implemented for vdpa-sim-blk

Yep, we developed libblkio [1]. libblkio exposes common API to access block devices in userspace. It supports several drivers.
The one useful for this use case is `virtio-blk-vhost-vdpa`. Here [2] some examples on how to use the libblkio test suite with the vdpa-sim-blk.

Since QEMU 7.2, it supports libblkio drivers, so you can use the following options to attach a vdpa-blk device to a VM:

  -blockdev node-name=drive_src1,driver=virtio-blk-vhost-vdpa,path=/dev/vhost-vdpa-0,cache.direct=on \
  -device virtio-blk-pci,id=src1,bootindex=2,drive=drive_src1 \

For now only what we called slow-path [3][4] is supported, since the VQs are not directly exposed to the guest, but QEMU allocates other VQs (similar to shadow VQs for net) to support live-migration and QEMU storage features. Fast-path is on the agenda, but on pause for now.

or any vdpa block device in userspace as yet, correct?

Do you mean with VDUSE?
In this case, yes, qemu-storage-daemon supports it, and can implement a virtio-blk in user space, exposing a disk image thorough VDUSE.

There is an example in libblkio as well [5] on how to start it.

So there was no test specific to vhost-vdpa that needs to be exercised, right?


I hope I answered above :-)
Definitely! This is exactly what I needed, it's really useful! Much appreciated for the detailed information!

I hadn't been aware of the latest status on libblkio drivers and qemu support since I last checked it (it was at some point right after KVM 2022, sorry my knowledge too outdated). I followed your links below and checked a few things, looks my change shouldn't affect anything. Good to see all the desired pieces landed to QEMU and libblkio already as planned, great job done!

Cheers,
-Siwei

This reminded me that I need to write a blog post with all this information, I hope to do that soon!

Stefano

[1] https://gitlab.com/libblkio/libblkio
[2] https://gitlab.com/libblkio/libblkio/-/blob/main/tests/meson.build?ref_type=heads#L42
[3] https://kvmforum2022.sched.com/event/15jK5/qemu-storage-daemon-and-libblkio-exploring-new-shores-for-the-qemu-block-layer-kevin-wolf-stefano-garzarella-red-hat
[4] https://kvmforum2021.sched.com/event/ke3a/vdpa-blk-unified-hardware-and-software-offload-for-virtio-blk-stefano-garzarella-red-hat
[5] https://gitlab.com/libblkio/libblkio/-/blob/main/tests/meson.build?ref_type=heads#L58