Re: [PATCH 1/3] sfc: revert "reduce the number of requested xdp ev queues"

[Jesper Dangaard Brouer]

On 09/07/2021 16.07, Edward Cree wrote:
> On 08/07/2021 13:14, Íñigo Huguet wrote:
> > In my opinion, there is no reason to make that distinction between
> > normal traffic and XDP traffic.
> > [...]
> > If the user wants to prevent XDP from mixing with normal traffic, just
> > not attaching an XDP program to the interface, or not using
> > XDP_TX/REDIRECT in it would be enough. Probably I don't understand
> > what you want to say here.
>
> I think it's less about that and more about avoiding lock contention.
> If two sources (XDP and the regular stack) are both trying to use a TXQ,
>   and contending for a lock, it's possible that the resulting total
>   throughput could be far less than either source alone would get if it
>   had exclusive use of a queue.
> There don't really seem to be any good answers to this; any CPU in the
>   system can initiate an XDP_REDIRECT at any time and if they can't each
>   get a queue to themselves then I don't see how the arbitration can be
>   performant.  (There is the middle-ground possibility of TXQs shared by
>   multiple XDP CPUs but not shared with the regular stack, in which case
>   if only a subset of CPUs are actually handling RX on the device(s) with
>   an XDP_REDIRECTing program it may be possible to avoid contention if
>   the core-to-XDP-TXQ mapping can be carefully configured.)

Yes, I prefer the 'middle-ground' fallback you describe.  XDP gets it's 
own set of TXQ-queues, and when driver detect TXQ's are less than CPUs 
that can redirect packets it uses an ndo_xdp_xmit function that takes a 
(hashed) lock (happens per packet burst (max 16)).

--Jesper