Re: [RFC] vdpa/mlx5: preserve CVQ vringh index

[Si-Wei Liu]

Steve, I think this is a loose end that I myself am not sure if worth 
fixing, copy Eugenio for his awareness. Reason is that when CVQ is in 
place it always has to cope with device state saving and restoration 
using shadowed virtqueue for a lot of cases not just migration, and 
that's the reason why SVQ is always enabled for CVQ in the latest QEMU. 
But I agree this is a nice to have, possibly there could be value to 
support vDPA VM instances without solely depending on SVQ for e.g. for 
use case like memory encrypted VM. Thanks for posting the fix and lets 
see what other people think about it.

-Siwei

On 10/26/2023 1:13 PM, Steven Sistare wrote:
> On 10/26/2023 4:11 PM, Steve Sistare wrote:
> > mlx5_vdpa does not preserve userland's view of vring base for the control
> > queue in the following sequence:
> >
> > ioctl VHOST_SET_VRING_BASE
> > ioctl VHOST_VDPA_SET_STATUS VIRTIO_CONFIG_S_DRIVER_OK
> >    mlx5_vdpa_set_status()
> >      setup_cvq_vring()
> >        vringh_init_iotlb()
> >          vringh_init_kern()
> >            vrh->last_avail_idx = 0;
> > ioctl VHOST_GET_VRING_BASE
> >
> > To fix, restore the value of cvq->vring.last_avail_idx after calling
> > vringh_init_iotlb.
> >
> > Signed-off-by: Steve Sistare <steven.sistare@xxxxxxxxxx>
> This is a resend, I forgot to cc myself the first time.
>
> I don't know if we expect vring_base to be preserved after reset, because the
> uapi comments say nothing about it.  mlx5 *does* preserve base across reset
> for the the data vq's, but perhaps that is an accident of the implementation.
>
> I posted this patch to perhaps avoid future problems. The bug(?) bit me while
> developing with an older version of qemu, and I can work around it in qemu
> code.  Further, the latest version of qemu always enables svq for the cvq
> and is not affected by this behavior AFAICT.
>
> - Steve