Re: [PATCH 1/3] sfc: revert "reduce the number of requested xdp ev queues"

From: Jesper Dangaard Brouer
Date: Fri Jul 09 2021 - 11:07:14 EST

On 09/07/2021 16.07, Edward Cree wrote:
On 08/07/2021 13:14, Íñigo Huguet wrote:
In my opinion, there is no reason to make that distinction between
normal traffic and XDP traffic.
If the user wants to prevent XDP from mixing with normal traffic, just
not attaching an XDP program to the interface, or not using
XDP_TX/REDIRECT in it would be enough. Probably I don't understand
what you want to say here.

I think it's less about that and more about avoiding lock contention.
If two sources (XDP and the regular stack) are both trying to use a TXQ,
and contending for a lock, it's possible that the resulting total
throughput could be far less than either source alone would get if it
had exclusive use of a queue.
There don't really seem to be any good answers to this; any CPU in the
system can initiate an XDP_REDIRECT at any time and if they can't each
get a queue to themselves then I don't see how the arbitration can be
performant. (There is the middle-ground possibility of TXQs shared by
multiple XDP CPUs but not shared with the regular stack, in which case
if only a subset of CPUs are actually handling RX on the device(s) with
an XDP_REDIRECTing program it may be possible to avoid contention if
the core-to-XDP-TXQ mapping can be carefully configured.)

Yes, I prefer the 'middle-ground' fallback you describe. XDP gets it's own set of TXQ-queues, and when driver detect TXQ's are less than CPUs that can redirect packets it uses an ndo_xdp_xmit function that takes a (hashed) lock (happens per packet burst (max 16)).