Re: [PATCH net-next v6 5/5] veth: time-based BQL completion coalescing via ethtool tx-usecs

[Jonas Köppeler]

On 6/1/26 2:00 PM, Simon Schippers wrote:
> On 5/28/26 09:46, Jonas Köppeler wrote:
> > On 5/27/26 3:54 PM, hawk@xxxxxxxxxx wrote:
> > > From: Simon Schippers <simon.schippers@xxxxxxxxxxxxxx>
> > >
> > > Per-packet BQL completion forces DQL to converge on limit=2, causing
> > > excessive NAPI scheduling overhead and qdisc requeues.
> > >
> > > Accumulate BQL completions and flush them when a configurable time
> > > threshold is exceeded, letting DQL discover a limit that bounds actual
> > > queuing delay to the configured interval. Coalescing state persists
> > > across NAPI polls in struct veth_rq so completions can accumulate
> > > beyond a single budget=64 cycle.
> > >
> > > Add ethtool tx-usecs support for runtime tuning. Default is 100 us;
> > > setting tx-usecs to 0 disables coalescing and falls back to per-packet
> > > completion.
> > >
> > >     ethtool -C <veth-dev> tx-usecs 500  # 500us coalescing
> > >     ethtool -C <veth-dev> tx-usecs 0    # per-packet (no coalescing)
> > >
> > > Co-developed-by: Jesper Dangaard Brouer <hawk@xxxxxxxxxx>
> > > Signed-off-by: Jesper Dangaard Brouer <hawk@xxxxxxxxxx>
> > > Signed-off-by: Simon Schippers <simon.schippers@xxxxxxxxxxxxxx>
> > Tested-by: Jonas Köppeler<j.koeppeler@xxxxxxxxxxxx>
> Thanks for your testing!
>
> However, I have issues reproducing.
> I run bare metal (without virtme) with v6 + your pktgen patch
> and I am on the branch pktgen-and-benchmark, commit
> "results: add veth-bql measurements":
>
> 1. ping fails with 100% packet loss ~20% of the times with --pktgen.
>     When this happens the avg ping of this run is mistakenly set
>     to 0.0 ms, which distorts the results.
>     I fixed it locally by rerunning when this happens.
>
> 2. pktgen runs with > 3 Mpps even with --nrules 10000, see log below.
>     I see that this is because of qdisc drops.
>     I also tried pfifo and sfq but with the same result.
>     I spent quite some time on it but I do not know a fix.
>
> Do you have an idea?
> Thanks!

Hi,
yes there are some changes missing in the test script.
I have pushed it now, sorry. This should fix 1.
Regarding 2.: do not look at the pktgen output, in the
new version you will see something like "goodput",
which is the number you should look for.
Pktgen will report at what speed it enqueued packets in
the qdisc.

Let me know if it worked.
Best,
Jonas

> The raw log:
>
> sudo ./veth_bql_test.sh --pktgen --duration 2 --qdisc fq_codel --no-bpftrace --tx-usecs 100 --nrules 10000
> INFO: Setting up veth pair with GRO
> INFO: Threaded NAPI enabled
> INFO: Installing qdisc: fq_codel
> INFO: Loaded 10000 iptables rules in consumer NS
> INFO: kernel: 7.1.0-rc4-patched-20260307+
> INFO: BQL sysfs found: /sys/class/net/veth_bql0/queues/tx-0/byte_queue_limits
> INFO: ethtool tx-usecs set to 100 on veth_bql1 (rx side)
> INFO: Starting ping to 10.99.0.2 (5/s) to measure latency under load
> INFO: Starting pktgen queue_xmit on veth_bql0 (threads=1 pkt_size=64)
>    [5s] BQL inflight=1 limit=17 watchdog=0
>    [5s] qdisc fq_codel pkts=27417 drops=6591520 requeues=14115 backlog=0 qlen=0 overlimits=0
>    [5s] softnet: processed=27960 time_squeeze=0 multi-CPU(6): cpu0(+5) cpu1(+121) cpu2(+33) cpu3(+116) cpu4(+27641) cpu5(+44)
> INFO: Ping loss: 0% packet loss
> INFO: Ping summary: rtt min/avg/max/mdev = 0.127/1.818/2.703/0.761 ms
> INFO: pktgen results (thread 0):
> Params: count 0  min_pkt_size: 64  max_pkt_size: 64
>       frags: 0  delay: 0  clone_skb: 0  ifname: veth_bql0@0
>       flows: 0 flowlen: 0
>       queue_map_min: 0  queue_map_max: 0
>       dst_min: 10.99.0.2  dst_max:
>       src_min:   src_max:
>       src_mac: 0e:aa:4e:05:95:89 dst_mac: c2:e5:a3:4c:2a:7f
>       udp_src_min: 9  udp_src_max: 9  udp_dst_min: 9999  udp_dst_max: 9999
>       src_mac_count: 0  dst_mac_count: 0
>       xmit_mode: xmit_queue
>       Flags: NO_TIMESTAMP  QUEUE_MAP_CPU  SHARED
> Current:
>       pkts-sofar: 6516031  errors: 102854
>       started: 12246566004us  stopped: 12248535234us idle: 0us
>       seq_num: 6516032  cur_dst_mac_offset: 0  cur_src_mac_offset: 0
>       cur_saddr: 10.99.0.1  cur_daddr: 10.99.0.2
>       cur_udp_dst: 9999  cur_udp_src: 9
>       cur_queue_map: 0
>       flows: 0
> Result: OK: 1969229(c1969229+d0) usec, 6516031 (64byte,0frags)
>    3308924pps 1694Mb/sec (1694169088bps) errors: 102854
> TEST: veth_bql                                                      [ OK ]
> INFO: Results: /home/simon/repos/veth-backpressure-performance-testing/results/selftests/2026-06-01T13-24-51