[PATCH net-next] tcp: devmem: only pre-allocate tokens the receiver can consume

[Bobby Eshleman]

From: Bobby Eshleman <bobbyeshleman@xxxxxxxx>

tcp_recvmsg_dmabuf() derives its token allocation amount directly from
the skb's nr_frags. After filling the user's receive buffer
tcp_xa_pool_commit() erases any tokens that were not used. When the
receive buffer is significantly smaller than the skb size, much of the
token allocation work is wasted.

Bound the token allocation amount by the remaining user receive buffer
size, and consequently reduce wasted xarray work. Introduce the helper
tcp_xa_pool_max_frags() to compute the token amount based on number of
frags and their sizes but capped when the accumulated size exceeds the
user buffer.

Testing on a CX7 w/ GRO and a steady sendmsg() flow of 1MB per send, we
see a typical RX-side skb touch upwards of ~32KB. With a 4KB recvmsg
size, probing shows that ~75% of the allocated tokens are not used.

Mean +- stdev over 5 reps:

    read size    base Gbps        patched Gbps      delta
    ---------    -------------    -------------    ------
    4K             29.7 +- 0.5       42.4 +- 2.7    +42.9%
    8K             47.0 +- 3.1       61.9 +- 4.8    +31.8%
    16K            69.6 +- 3.4       80.2 +- 6.7    +15.3%
    32K            88.3 +- 0.8       87.1 +- 0.8     -1.4%
    64K            87.5 +- 1.2       87.0 +- 0.8     -0.6%
    256K           88.2 +- 0.7       86.5 +- 0.8     -1.9%
    1M             88.5 +- 0.8       87.5 +- 1.9     -1.1%
    4M             88.5 +- 0.8       88.0 +- 0.7     -0.6%
    8M             88.6 +- 0.9       88.7 +- 0.7     +0.1%
    16M            88.7 +- 0.6       88.0 +- 0.7     -0.8%
    32M            88.4 +- 1.1       88.3 +- 0.6     -0.1%

perf over the RX cores, at 4KB reads and 1MB writes (before left, after
right):

    10.94%  xas_store                5.06%  xas_store
     3.74%  xas_find_marked          1.49%  xas_find_marked
     2.33%  __xa_alloc               0.91%  __xa_alloc
     0.96%  __xa_erase               0.25%  __xa_erase
     0.80%  __xas_nomem              0.43%  __xas_nomem
     0.49%  xas_load                 0.74%  xas_load
     0.45%  __xa_cmpxchg_raw         0.74%  __xa_cmpxchg_raw
                                     0.28%  __xa_cmpxchg

The gain is only for reads below the typical GRO receive size for the
system (32KB on this system). Above that the user buffer covers the skb
size, so waste is already minimal.

Signed-off-by: Bobby Eshleman <bobbyeshleman@xxxxxxxx>
---
 net/ipv4/tcp.c | 31 ++++++++++++++++++++++++++++++-
 1 file changed, 30 insertions(+), 1 deletion(-)

diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index b4237d0e994d..c35277439386 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -2494,6 +2494,32 @@ static int tcp_xa_pool_refill(struct sock *sk, struct tcp_xa_pool *p,
 	return k ? 0 : err;
 }
 
+/* Return the number of fragments of @skb deliverable from byte @offset, capped
+ * by @remaining_len. Returns 0 only when no fragment holds @offset.
+ */
+static unsigned int tcp_xa_pool_max_frags(const struct sk_buff *skb,
+					  unsigned int offset,
+					  int remaining_len)
+{
+	unsigned int start = skb_headlen(skb);
+	unsigned int max_frags = 0;
+	int i;
+
+	for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) {
+		int end = start + skb_frag_size(&skb_shinfo(skb)->frags[i]);
+		int copy = end - offset;
+
+		if (copy > 0) {
+			max_frags++;
+			if (copy >= remaining_len)
+				break;
+		}
+		start = end;
+	}
+
+	return max_frags;
+}
+
 /* On error, returns the -errno. On success, returns number of bytes sent to the
  * user. May not consume all of @remaining_len.
  */
@@ -2503,6 +2529,7 @@ static int tcp_recvmsg_dmabuf(struct sock *sk, const struct sk_buff *skb,
 {
 	struct dmabuf_cmsg dmabuf_cmsg = { 0 };
 	struct tcp_xa_pool tcp_xa_pool;
+	unsigned int max_frags;
 	unsigned int start;
 	int i, copy, n;
 	int sent = 0;
@@ -2554,6 +2581,8 @@ static int tcp_recvmsg_dmabuf(struct sock *sk, const struct sk_buff *skb,
 		/* after that, send information of dmabuf pages through a
 		 * sequence of cmsg
 		 */
+		max_frags = tcp_xa_pool_max_frags(skb, offset, remaining_len);
+
 		for (i = 0; i < skb_shinfo(skb)->nr_frags; i++) {
 			skb_frag_t *frag = &skb_shinfo(skb)->frags[i];
 			struct net_iov *niov;
@@ -2591,7 +2620,7 @@ static int tcp_recvmsg_dmabuf(struct sock *sk, const struct sk_buff *skb,
 				dmabuf_cmsg.frag_offset = frag_offset;
 				dmabuf_cmsg.frag_size = copy;
 				err = tcp_xa_pool_refill(sk, &tcp_xa_pool,
-							 skb_shinfo(skb)->nr_frags - i);
+							 max_frags);
 				if (err)
 					goto out;
 

---
base-commit: 1bb784eb6e38fd73143f021608e4ef3095d0c0d7
change-id: 20260901-b4-net-next_devmem-token-refill-09ab9377ad58

Best regards,
-- 
Bobby Eshleman <bobbyeshleman@xxxxxxxx>