Re: [RFC v2 1/7] vhost: expose used buffers

[Guo Zhi]

----- Original Message -----
> From: "eperezma" <eperezma@xxxxxxxxxx>
> To: "Guo Zhi" <qtxuning1999@xxxxxxxxxxx>
> Cc: "jasowang" <jasowang@xxxxxxxxxx>, "sgarzare" <sgarzare@xxxxxxxxxx>, "Michael Tsirkin" <mst@xxxxxxxxxx>, "netdev"
> <netdev@xxxxxxxxxxxxxxx>, "linux-kernel" <linux-kernel@xxxxxxxxxxxxxxx>, "kvm list" <kvm@xxxxxxxxxxxxxxx>,
> "virtualization" <virtualization@xxxxxxxxxxxxxxxxxxxxxxxxxx>
> Sent: Thursday, August 18, 2022 2:16:40 PM
> Subject: Re: [RFC v2 1/7] vhost: expose used buffers

> On Wed, Aug 17, 2022 at 3:58 PM Guo Zhi <qtxuning1999@xxxxxxxxxxx> wrote:
>>
>> Follow VIRTIO 1.1 spec, only writing out a single used ring for a batch
>> of descriptors.
>>
>> Signed-off-by: Guo Zhi <qtxuning1999@xxxxxxxxxxx>
>> ---
>>  drivers/vhost/vhost.c | 14 ++++++++++++--
>>  drivers/vhost/vhost.h |  1 +
>>  2 files changed, 13 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
>> index 40097826cff0..7b20fa5a46c3 100644
>> --- a/drivers/vhost/vhost.c
>> +++ b/drivers/vhost/vhost.c
>> @@ -2376,10 +2376,20 @@ static int __vhost_add_used_n(struct vhost_virtqueue
>> *vq,
>>         vring_used_elem_t __user *used;
>>         u16 old, new;
>>         int start;
>> +       int copy_n = count;
>>
>> +       /**
>> +        * If in order feature negotiated, devices can notify the use of a batch
>> of buffers to
>> +        * the driver by only writing out a single used ring entry with the id
>> corresponding
>> +        * to the head entry of the descriptor chain describing the last buffer
>> in the batch.
>> +        */
>> +       if (vhost_has_feature(vq, VIRTIO_F_IN_ORDER)) {
>> +               copy_n = 1;
>> +               heads = &heads[count - 1];
>> +       }
>>         start = vq->last_used_idx & (vq->num - 1);
>>         used = vq->used->ring + start;
>> -       if (vhost_put_used(vq, heads, start, count)) {
>> +       if (vhost_put_used(vq, heads, start, copy_n)) {
>>                 vq_err(vq, "Failed to write used");
>>                 return -EFAULT;
>>         }
>> @@ -2410,7 +2420,7 @@ int vhost_add_used_n(struct vhost_virtqueue *vq, struct
>> vring_used_elem *heads,
>>
>>         start = vq->last_used_idx & (vq->num - 1);
>>         n = vq->num - start;
>> -       if (n < count) {
>> +       if (n < count && !vhost_has_feature(vq, VIRTIO_F_IN_ORDER)) {
>>                 r = __vhost_add_used_n(vq, heads, n);
>>                 if (r < 0)
>>                         return r;
> 
> It would be simpler to use vhost_add_used in vsock/vhost_test to add a
> batch of used descriptors, and leave vhost_add_used_n untouched.
> 
> Since it's the upper layer the one that manages the in_order in this
> version, we could:
> * Always call vhost_add_used(vq, last_head_of_batch, ...) for the tx
> queue, that does not need used length info.
> * Call vhost_add_used_n for the rx queue, since the driver needs the
> used length info.
 
Very insightful view!

At first, vhost_add_used will write used ring for skipped buffer in a batch,
so we can't let vhost unmodified to enable in order feature.

Secondly, Current changes to vhost_add_used_n will affect the rx queue,
as driver have to get each buffer's length from used ring.

So I would propose a flag parameter in vhost_add_used_n to decide
whether the batch for in order buffer is done or nor.
 
>> diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
>> index d9109107af08..0d5c49a30421 100644
>> --- a/drivers/vhost/vhost.h
>> +++ b/drivers/vhost/vhost.h
>> @@ -236,6 +236,7 @@ enum {
>>         VHOST_FEATURES = (1ULL << VIRTIO_F_NOTIFY_ON_EMPTY) |
>>                          (1ULL << VIRTIO_RING_F_INDIRECT_DESC) |
>>                          (1ULL << VIRTIO_RING_F_EVENT_IDX) |
>> +                        (1ULL << VIRTIO_F_IN_ORDER) |
>>                          (1ULL << VHOST_F_LOG_ALL) |
>>                          (1ULL << VIRTIO_F_ANY_LAYOUT) |
>>                          (1ULL << VIRTIO_F_VERSION_1)
>> --
>> 2.17.1
>>