Re: [PATCH net-next 0/7] net: devmem: size TX net_iovs from the dmabuf layout

[Mina Almasry]

On Mon, Nov 2, 2026 at 11:14 AM Stanislav Fomichev <sdf.kernel@xxxxxxxxx> wrote:
>
> TX bindings use PAGE_SIZE net_iovs, so a frag covers at most a page.
> A 64KB skb needs 16 frags, and MAX_SKB_FRAGS caps devmem skbs at about
> 68KB, which makes BIG TCP ineffective for devmem TX.
>
> This series picks the TX net_iov size from the dmabuf sg table instead:
> the largest power of 2, up to 2MB, which aligns every DMA segment. With
> a 2MB hugepage-backed udmabuf a BIG TCP skb takes a few frags. There's
> no UAPI change since TX addresses are dmabuf offsets.
>
> TCP still bounds skbs by size_goal, so frags get larger, skbs don't.
> A single frag can now be as large as the skb, which matters for
> NETMEM_TX drivers with tso_max_size above 64KB:
> - mlx5: 32-bit length per data segment
> - gve: splits frags into 16KB descriptors
> - fbnic: keeps the default 64KB tso_max_size
> - bnxt: 16-bit BD length, devmem TX frags are capped at 32KB by a new
>   per-device limit
>
> Tested on two mlx5 400Gb/s hosts with kperf (udmabuf, 32KB RX
> net_iovs, 2MB hugepage-backed TX buffer), series backported to a 7.1
> kernel on the sender only. 4 flows, MTU 4200, 6 x 60s runs, medians
> (CPU over the 3 runs where every flow got its own CPUs):
>
>                               before    after
>   frags per skb (sender)       16-17      1-2
>   throughput (Gb/s)            389.6    389.6
>   sender sendmsg CPU (sys)     12.0%     6.7%   (-44%)
>
> The link is the bottleneck, so the gain is CPU, not throughput.
>

I mean, test single flow please so we see the throughput gain. But
CPU% reduction is good enough reason to merge I guess too.

-- 
Thanks,
Mina