Re: [PATCH net] net/sched: fix packet reordering on requeue

[Eric Dumazet]

Le mar. 27 oct. 2026 à 10:28, Jesper Dangaard Brouer <hawk@xxxxxxxxxx> a écrit :
>
>
>
> On 10/26/26 16:31, Antonio Quartulli wrote:
> > dev_requeue_skb() appends requeued skbs to the tail of q->gso_skb, while
> > dequeue_skb() takes them back one at a time from the head. If the driver
> > returns NETDEV_TX_BUSY again for the first of them, it moves behind the
> > skbs still parked there and is sent after them.
> >
> > This showed up as heavy TCP reordering while benchmarking over veth,
> > which returns NETDEV_TX_BUSY whenever its ring is full since
> > commit dc82a33297fc ("veth: apply qdisc backpressure on full ptr_ring to reduce TX drops").
> >
> > Requeue the skbs at the head of gso_skb, keeping their order. With 4 TCP
> > streams over a veth pair with a pfifo qdisc, this removes about 1.3M
> > out-of-order segments and 15-37k retransmits per 15 seconds, and raises
> > throughput from about 40 to 76 Gbit/s.
> >
> > Fixes: a53851e2c321 ("net: sched: explicit locking in gso_cpu fallback")
> > Fixes: 9540d977618c ("net: sched: fix skb leak in dev_requeue_skb()")
> > Cc: stable@xxxxxxxxxxxxxxx
> > Assisted-by: Claude:claude-opus-5-5
> > Signed-off-by: Antonio Quartulli <antonio@xxxxxxxxxxxxx>
> > ---
>
> Thanks for finding and fixing this!
>
> Acked-by: Jesper Dangaard Brouer <hawk@xxxxxxxxxx>
>
> > Notes:
> >
> > - Setup: 8-vCPU VM, veth pair; sender with a pfifo qdisc and TSO off,
> >    receiver with GRO and threaded NAPI; iperf3, 15 s runs.
> >
>
> For others, notice threaded NAPI an important setup parameter here.
> This makes it more likely and reliable for ptr_ring producer and
> consumer to run on different CPUs.  This situation can still occur
> without threaded NAPI, when multiple localhost userspace applications
> run producing packets into veth, as they will likely get be schedules on
> different CPUs.
>
> FYI, we also run veth with threaded NAPI in production, but used in a
> routing setup (no "localhost" apps).  We have configured veth to have
> many queues (64), and we use qdisc MQ on top to avoid serialization on
> the qdisc root-lock.  Thus, we should be less affected, as we spread
> flows more across qdisc.

And presumably you use TSO.

With TSO, chances of hitting the bug is probably quite small.