Re: [PATCH netdev 0/5] virtio-net support xdp socket zero copy xmit

[Jason Wang]

On 2021/1/6 上午10:55, Xuan Zhuo wrote:
> On Wed, 6 Jan 2021 10:46:43 +0800, Jason Wang <jasowang@xxxxxxxxxx> wrote:
> > On 2021/1/5 下午8:42, Xuan Zhuo wrote:
> > > On Tue, 5 Jan 2021 17:32:19 +0800, Jason Wang <jasowang@xxxxxxxxxx> wrote:
> > > > On 2021/1/5 下午5:11, Xuan Zhuo wrote:
> > > > > The first patch made some adjustments to xsk.
> > > > Thanks a lot for the work. It's rather interesting.
> > > >
> > > >
> > > > > The second patch itself can be used as an independent patch to solve the problem
> > > > > that XDP may fail to load when the number of queues is insufficient.
> > > > It would be better to send this as a separated patch. Several people
> > > > asked for this before.
> > > >
> > > >
> > > > > The third to last patch implements support for xsk in virtio-net.
> > > > >
> > > > > A practical problem with virtio is that tx interrupts are not very reliable.
> > > > > There will always be some missing or delayed tx interrupts. So I specially added
> > > > > a point timer to solve this problem. Of course, considering performance issues,
> > > > > The timer only triggers when the ring of the network card is full.
> > > > This is sub-optimal. We need figure out the root cause. We don't meet
> > > > such issue before.
> > > >
> > > > Several questions:
> > > >
> > > > - is tx interrupt enabled?
> > > > - can you still see the issue if you disable event index?
> > > > - what's backend did you use? qemu or vhost(user)?
> > > Sorry, it may just be a problem with the backend I used here. I just tested the
> > > latest qemu and it did not have this problem. I think I should delete the
> > > timer-related code?
> >
> > Yes, please.
> >
> >
> > > > > Regarding the issue of virtio-net supporting xsk's zero copy rx, I am also
> > > > > developing it, but I found that the modification may be relatively large, so I
> > > > > consider this patch set to be separated from the code related to xsk zero copy
> > > > > rx.
> > > > That's fine, but a question here.
> > > >
> > > > How is the multieuque being handled here. I'm asking since there's no
> > > > programmable filters/directors support in virtio spec now.
> > > >
> > > > Thanks
> > > I don't really understand what you mean. In the case of multiple queues,
> > > there is no problem.
> >
> > So consider we bind xsk to queue 4, how can you make sure the traffic to
> > be directed queue 4? One possible solution is to use filters as what
> > suggested in af_xdp.rst:
> >
> >         ethtool -N p3p2 rx-flow-hash udp4 fn
> >         ethtool -N p3p2 flow-type udp4 src-port 4242 dst-port 4242 \
> >             action 16
> > ...
> >
> > But virtio-net doesn't have any filters that could be programmed from
> > the driver.
> >
> > Anything I missed here?
> >
> > Thanks
> I understand what you mean, this problem does exist, and I encountered it when I
> tested qemu.
>
> First of all, this is that the problem only affects recv. This patch is for
> xmit. Of course, our normal business must also have recv scenarios.
>
> My solution in developing the upper-level application is to bond all the queues
> to ensure that we can receive the packets we want.


I'm not sure I get you here. Note that. one advantage of AF_XDP is that 
is allows XSK to be bound to a specific queue and the rest could still 
be used by kernel.


>   And I think in the
> implementation of the use, even if the network card supports filters, we should
> also bond all the queues, because we don't know which queue the traffic we care
> about will arrive from.


With the help of filters the card can select a specific queue based on 
hash or n-tuple so it should work?


> Regarding the problem of virtio-net, I think our core question is whether we
> need to deal with this problem in the driver of virtio-net, I personally think
> that we should add the virtio specification to define this scenario.


Yes, so do you want to do that? It would make virtio-net more user 
friendly to AF_XDP. (Or if you wish I can post patch to extend the spec).


> When I tested it, I found that some cloud vendors' implementations guarantee
> this queue selection algorithm.


Right, though spec suggest a automatic steering algorithm but it's not 
mandatory. Vendor can implement their own.

But hash or ntuple filter should be still useful.

Thanks


> Thanks!!
>
> > > > > Xuan Zhuo (5):
> > > > >      xsk: support get page for drv
> > > > >      virtio-net: support XDP_TX when not more queues
> > > > >      virtio-net, xsk: distinguish XDP_TX and XSK XMIT ctx
> > > > >      xsk, virtio-net: prepare for support xsk
> > > > >      virtio-net, xsk: virtio-net support xsk zero copy tx
> > > > >
> > > > >     drivers/net/virtio_net.c    | 643 +++++++++++++++++++++++++++++++++++++++-----
> > > > >     include/linux/netdevice.h   |   1 +
> > > > >     include/net/xdp_sock_drv.h  |  10 +
> > > > >     include/net/xsk_buff_pool.h |   1 +
> > > > >     net/xdp/xsk_buff_pool.c     |  10 +-
> > > > >     5 files changed, 597 insertions(+), 68 deletions(-)
> > > > >
> > > > > --
> > > > > 1.8.3.1