Re: [RFC PATCH net-next 0/2] net: Use SMP threads for backlog NAPI.

[Yan Zhai]

On Thu, Aug 17, 2023 at 8:16 AM Sebastian Andrzej Siewior
<bigeasy@xxxxxxxxxxxxx> wrote:
>
> On 2023-08-14 11:24:21 [-0700], Jakub Kicinski wrote:
> > On Mon, 14 Aug 2023 11:35:26 +0200 Sebastian Andrzej Siewior wrote:
> > > The RPS code and "deferred skb free" both send IPI/ function call
> > > to a remote CPU in which a softirq is raised. This leads to a warning on
> > > PREEMPT_RT because raising softiqrs from function call led to undesired
> > > behaviour in the past. I had duct tape in RT for the "deferred skb free"
> > > and Wander Lairson Costa reported the RPS case.
> >
> > Could you find a less invasive solution?
> > backlog is used by veth == most containerized environments.
> > This change has a very high risk of regression for a lot of people.
>
> Looking at the cloudflare ppl here in the thread, I doubt they use
> backlog but have proper NAPI so they might not need this.
>
Cloudflare does have backlog usage. On some veths we have to turn GRO
off to cope with multi-layer encapsulation, and there is also no XDP
attached on these interfaces, thus the backlog is used. There are also
other usage of backlog, tuntap, loopback and bpf-redirect ingress.
Frankly speaking, making a NAPI instance "threaded" itself is not a
concern. We have threaded NAPI running on some veth for quite a while,
and it performs pretty well. The concern, if any, would be the
maturity of new code. I am happy to help derisk with some lab tests
and dogfooding if generic agreement is reached to proceed with this
idea.

Yan

> There is no threaded NAPI for backlog and RPS. This was suggested as the
> mitigation for the highload/ DoS case. Can this become a problem or
> - backlog is used only by old drivers so they can move to proper NAPI if
>   it becomes a problem.
> - RPS spreads the load across multiple CPUs so it unlikely to become a
>   problem.
>
> Making this either optional in general or mandatory for threaded
> interrupts or PREEMPT_RT will probably not make the maintenance of this
> code any simpler.
>
> I've been looking at veth. In the xdp case it has its own NAPI instance.
> In the non-xdp it uses backlog. This should be called from
> ndo_start_xmit and user's write() so BH is off and interrupts are
> enabled at this point and it should be kind of rate-limited. Couldn't we
> bypass backlog in this case and deliver the packet directly to the
> stack?
>
> Sebastian



-- 

Yan