Re: [PATCH v3 00/11] VDUSE: Improve performance

[Jason Wang]

On Fri, Mar 10, 2023 at 4:50 PM Michael S. Tsirkin <mst@xxxxxxxxxx> wrote:
>
> On Tue, Feb 28, 2023 at 05:40:59PM +0800, Xie Yongji wrote:
> > Hi all,
> >
> > This series introduces some ways to improve VDUSE performance.
>
>
> Pls fix warnings reported by 0-day infra, dropping this for now.

Note that I plan to review this next week.

Thanks

>
>
> > Patch 1 ~ 6 bring current interrupt affinity spreading mechanism
> > to vduse device and make it possible for the virtio-blk driver
> > to build the blk-mq queues based on it. This would be useful to
> > mitigate the virtqueue lock contention in virtio-blk driver. In
> > our test, with those patches, we could get ~50% improvement (600k
> > iops -> 900k iops) when using per-cpu virtqueue.
> >
> > Patch 7 adds a sysfs interface for each vduse virtqueue to change
> > the affinity for IRQ callback. It would be helpful for performance
> > tuning when the affinity mask contains more than one CPU.
> >
> > Patch 8 ~ 9 associate an eventfd to the vdpa callback so that
> > we can signal it directly during irq injection without scheduling
> > an additional workqueue thread to do that.
> >
> > Patch 10, 11 add a sysfs interface to support specifying bounce
> > buffer size in virtio-vdpa case. The high throughput workloads
> > can benefit from it. And we can also use it to reduce the memory
> > overhead for small throughput workloads.
> >
> > Please review, thanks!
> >
> > V2 to V3:
> > - Rebased to newest kernel tree
> > - Export group_cpus_evenly() instead of irq_create_affinity_masks() [MST]
> > - Remove the sysfs for workqueue control [Jason]
> > - Associate an eventfd to the vdpa callback [Jason]
> > - Signal the eventfd directly in vhost-vdpa case [Jason]
> > - Use round-robin to spread IRQs between CPUs in the affinity mask [Jason]
> > - Handle the cpu hotplug case on IRQ injection [Jason]
> > - Remove effective IRQ affinity and balance mechanism for IRQ allocation
> >
> > V1 to V2:
> > - Export irq_create_affinity_masks()
> > - Add set/get_vq_affinity and set_irq_affinity callbacks in vDPA
> >   framework
> > - Add automatic irq callback affinity support in VDUSE driver [Jason]
> > - Add more backgrounds information in commit log [Jason]
> > - Only support changing effective affinity when the value is a subset
> >   of the IRQ callback affinity mask
> >
> > Xie Yongji (11):
> >   lib/group_cpus: Export group_cpus_evenly()
> >   vdpa: Add set/get_vq_affinity callbacks in vdpa_config_ops
> >   vdpa: Add set_irq_affinity callback in vdpa_config_ops
> >   vduse: Refactor allocation for vduse virtqueues
> >   vduse: Support automatic irq callback affinity
> >   vduse: Support set/get_vq_affinity callbacks
> >   vduse: Add sysfs interface for irq callback affinity
> >   vdpa: Add eventfd for the vdpa callback
> >   vduse: Signal interrupt's eventfd directly in vhost-vdpa case
> >   vduse: Delay iova domain creation
> >   vduse: Support specifying bounce buffer size via sysfs
> >
> >  drivers/vdpa/vdpa_user/vduse_dev.c | 490 +++++++++++++++++++++++++----
> >  drivers/vhost/vdpa.c               |   2 +
> >  drivers/virtio/virtio_vdpa.c       |  33 ++
> >  include/linux/vdpa.h               |  25 ++
> >  lib/group_cpus.c                   |   1 +
> >  5 files changed, 488 insertions(+), 63 deletions(-)
> >
> > --
> > 2.20.1
>