Re: [PATCH net-next v6 3/4] virtio/vsock: fix logic which reduces credit update messages

[Stefano Garzarella]

On Tue, Dec 05, 2023 at 03:07:47PM +0300, Arseniy Krasnov wrote:
> On 05.12.2023 13:54, Stefano Garzarella wrote:
> > On Tue, Dec 05, 2023 at 09:48:05AM +0300, Arseniy Krasnov wrote:
> > > Add one more condition for sending credit update during dequeue from
> > > stream socket: when number of bytes in the rx queue is smaller than
> > > SO_RCVLOWAT value of the socket. This is actual for non-default value
> > > of SO_RCVLOWAT (e.g. not 1) - idea is to "kick" peer to continue data
> > > transmission, because we need at least SO_RCVLOWAT bytes in our rx
> > > queue to wake up user for reading data (in corner case it is also
> > > possible to stuck both tx and rx sides, this is why 'Fixes' is used).
> > >
> > > Fixes: b89d882dc9fc ("vsock/virtio: reduce credit update messages")
> > > Signed-off-by: Arseniy Krasnov <avkrasnov@xxxxxxxxxxxxxxxxx>
> > > ---
> > > net/vmw_vsock/virtio_transport_common.c | 9 +++++++--
> > > 1 file changed, 7 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c
> > > index e137d740804e..461c89882142 100644
> > > --- a/net/vmw_vsock/virtio_transport_common.c
> > > +++ b/net/vmw_vsock/virtio_transport_common.c
> > > @@ -558,6 +558,7 @@ virtio_transport_stream_do_dequeue(struct vsock_sock *vsk,
> > >     struct virtio_vsock_sock *vvs = vsk->trans;
> > >     size_t bytes, total = 0;
> > >     struct sk_buff *skb;
> > > +    bool low_rx_bytes;
> > >     int err = -EFAULT;
> > >     u32 free_space;
> > >
> > > @@ -602,6 +603,8 @@ virtio_transport_stream_do_dequeue(struct vsock_sock *vsk,
> > >     }
> > >
> > >     free_space = vvs->buf_alloc - (vvs->fwd_cnt - vvs->last_fwd_cnt);
> > > +    low_rx_bytes = (vvs->rx_bytes <
> > > +            sock_rcvlowat(sk_vsock(vsk), 0, INT_MAX));
> >
> > As in the previous patch, should we avoid the update it if `fwd_cnt` and `last_fwd_cnt` are the same?
> >
> > Now I'm thinking if it is better to add that check directly in virtio_transport_send_credit_update().
>
> Good point, but I think, that it is better to keep this check here, because access to 'fwd_cnt' and 'last_fwd_cnt'
> requires taking rx_lock - so I guess it is better to avoid taking this lock every time in 'virtio_transport_send_credit_update()'.

Yeah, I agree.

> So may be we can do something like:
>
>
> fwd_cnt_delta = vvs->fwd_cnt - vvs->last_fwd_cnt;
> free_space = vvs->buf_alloc - fwd_cnt_delta;

Pre-existing issue, but should we handle the wrap (e.g. fwd_cnt wrapped, 
but last_fwd_cnt not yet?). Maybe in that case we can foce the status
update.

> and then, after lock is released:
>
> if (fwd_cnt_delta && (free_space < VIRTIO_VSOCK_MAX_PKT_BUF_SIZE ||
>    low_rx_bytes))
>        virtio_transport_send_credit_update(vsk);
>
> WDYT?

Yep, I agree.

> Also, I guess that next idea to update this optimization(in next patchset), is to make
> threshold depends on vvs->buf_alloc. Because if someone changes minimum buffer size to
> for example 32KB, and then sets buffer size to 32KB, then free_space will be always
> non-zero, thus optimization is off now and credit update is sent on 
> every read.

But does it make sense to allow a buffer smaller than 
VIRTIO_VSOCK_MAX_PKT_BUF_SIZE?

Maybe we should fail in virtio_transport_notify_buffer_size() or use it 
as minimum.

Stefano