Re: [PATCH] virtio_ring: Fix the stale index in available ring

[Gavin Shan]

On 3/21/24 03:15, Keir Fraser wrote:
> On Wed, Mar 20, 2024 at 03:24:16PM +1000, Gavin Shan wrote:
> > Before this patch was posted, I had debugging code to record last 16 transactions
> > to the available and used queue from guest and host side. It did reveal the wrong
> > head was fetched from the available queue.
> >
> > [   11.785745] ================ virtqueue_get_buf_ctx_split ================
> > [   11.786238] virtio_net virtio0: output.0:id 74 is not a head!
> > [   11.786655] head to be released: 036 077
> > [   11.786952]
> > [   11.786952] avail_idx:
> > [   11.787234] 000  63985  <--
> > [   11.787237] 001  63986
> > [   11.787444] 002  63987
> > [   11.787632] 003  63988
> > [   11.787821] 004  63989
> > [   11.788006] 005  63990
> > [   11.788194] 006  63991
> > [   11.788381] 007  63992
> > [   11.788567] 008  63993
> > [   11.788772] 009  63994
> > [   11.788957] 010  63995
> > [   11.789141] 011  63996
> > [   11.789327] 012  63997
> > [   11.789515] 013  63998
> > [   11.789701] 014  63999
> > [   11.789886] 015  64000
>
> Does the error always occur at such a round idx value?
>
> Here, 64000 == 0xFA00. Maybe coincidence but it's improbable enough to be interesting.
>
> This debug code seems rather useful!

Keir, Nope, it's just coincidence. We don't have such kind of pattern.

Thanks,
Gavin

> > [   11.790068]
> > [   11.790068] avail_head:
> > [   11.790529] 000  075  <--
> > [   11.790718] 001  036
> > [   11.790890] 002  077
> > [   11.791061] 003  129
> > [   11.791231] 004  072
> > [   11.791400] 005  130
> > [   11.791574] 006  015
> > [   11.791748] 007  074
> > [   11.791918] 008  130
> > [   11.792094] 009  130
> > [   11.792263] 010  074
> > [   11.792437] 011  015
> > [   11.792617] 012  072
> > [   11.792788] 013  129
> > [   11.792961] 014  077    // The last two heads from guest to host: 077, 036
> > [   11.793134] 015  036
> >
> > [root@nvidia-grace-hopper-05 qemu.main]# cat /proc/vhost
> >
> > avail_idx
> > 000  63998
> > 001  64000
> > 002  63954  <---
> > 003  63955
> > 004  63956
> > 005  63974
> > 006  63981
> > 007  63984
> > 008  63986
> > 009  63987
> > 010  63988
> > 011  63989
> > 012  63992
> > 013  63993
> > 014  63995
> > 015  63997
> >
> > avail_head
> > 000  074
> > 001  015
> > 002  072
> > 003  129
> > 004  074            // The last two heads seen by vhost is: 074, 036
> > 005  036
> > 006  075  <---
> > 007  036
> > 008  077
> > 009  129
> > 010  072
> > 011  130
> > 012  015
> > 013  074
> > 014  130
> > 015  130
> >
> > used_idx
> > 000  64000
> > 001  63882  <---
> > 002  63889
> > 003  63891
> > 004  63898
> > 005  63936
> > 006  63942
> > 007  63946
> > 008  63949
> > 009  63953
> > 010  63957
> > 011  63981
> > 012  63990
> > 013  63992
> > 014  63993
> > 015  63999
> >
> > used_head
> > 000  072
> > 001  129
> > 002  074          // The last two heads published to guest is: 074, 036
> > 003  036
> > 004  075  <---
> > 005  036
> > 006  077
> > 007  129
> > 008  072
> > 009  130
> > 010  015
> > 011  074
> > 012  130
> > 013  130
> > 014  074
> > 015  015
> >
> > Thanks,
> > Gavin