Re: [PATCH net] gve: fix checksum state and gso_segs of HW-GRO packets
From: Ankit Garg
Date: Sun Nov 15 2026 - 23:32:31 EST
On Sat, Nov 14, 2026 at 7:14 AM Eric Dumazet <edumazet@xxxxxxxxxx> wrote:
>
> Le ven. 13 nov. 2026 à 20:56, Joshua Washington <joshwash@xxxxxxxxxx> a écrit :
> >
> > From: Ankit Garg <nktgrg@xxxxxxxxxx>
> >
> > With HW-GRO enabled, gve_rx_complete_rsc() turns a coalesced packet into
> > a GSO skb by setting gso_size, gso_segs and gso_type, but leaves it
> > CHECKSUM_UNNECESSARY with th->check as the device wrote it. Unlike
> > tcp4_gro_complete() and tcp6_gro_complete() for a SW-GRO packet, it does
> > not seed th->check with the TCP pseudo-header checksum or set
> > CHECKSUM_PARTIAL.
> >
> > The GRO stack completes a HW-GRO skb itself only if it holds the skb for
> > merging (since commit 5eddb24901ee ("gro: add support of (hw)gro packets
> > to gro stack")). Otherwise the skb is delivered as is, e.g. with rx-gro
> > off or for a packet that is flushed immediately (e.g. with PSH set). If
> > such an skb is forwarded out of a TSO-capable device, netif_needs_gso()
> > accepts it as CHECKSUM_UNNECESSARY and the device segments it with a
> > bogus checksum seed, so every segment is dropped by the receiver. This
> > blackholes forwarded TCP traffic on VMs acting as routers or NAT
> > gateways and was observed on VMs in real use.
> >
> > Fix it in a manner consistent with other drivers with HW-GRO: set the
> > network and transport headers and, for packets whose checksum the device
> > verified, seed th->check with the pseudo-header checksum and call
> > tcp_gro_complete(), passing gso_segs through NAPI_GRO_CB(skb)->count.
> > skb->data points at the Ethernet header on the napi_gro_frags() path and
> > at the IP header on the header-split and copybreak paths, so account for
> > that when setting the network header. Since the header layout now
> > matters, tell IPv4 from IPv6 by the EtherType, as qede and bnx2x do,
> > rather than by the packet type.
> >
> > Also compute gso_segs from the TCP payload length. It was computed from
> > skb->data_len, which is 0 for a packet that rx_copybreak copied whole
> > into the linear part, leaving such packets with gso_segs 0 and a
> > non-zero gso_size.
> >
> > Fixes: e637c244b954 ("gve: Advertise NETIF_F_GRO_HW instead of NETIF_F_LRO")
> > Cc: stable@xxxxxxxxxxxxxxx
> > Reviewed-by: Joshua Washington <joshwash@xxxxxxxxxx>
> > Reviewed-by: Jordan Rhee <jordanrhee@xxxxxxxxxx>
> > Reviewed-by: Praveen Kaligineedi <pkaligineedi@xxxxxxxxxx>
> > Signed-off-by: Ankit Garg <nktgrg@xxxxxxxxxx>
> > ---
> > drivers/net/ethernet/google/gve/gve_rx_dqo.c | 69 +++++++++++++++-----
> > 1 file changed, 53 insertions(+), 16 deletions(-)
> >
> > diff --git a/drivers/net/ethernet/google/gve/gve_rx_dqo.c b/drivers/net/ethernet/google/gve/gve_rx_dqo.c
> > index 82ed5e48b00a..58696d26c698 100644
> > --- a/drivers/net/ethernet/google/gve/gve_rx_dqo.c
> > +++ b/drivers/net/ethernet/google/gve/gve_rx_dqo.c
> > @@ -13,6 +13,7 @@
> > #include <linux/ipv6.h>
> > #include <linux/skbuff.h>
> > #include <linux/slab.h>
> > +#include <net/gro.h>
> > #include <net/ip6_checksum.h>
> > #include <net/ipv6.h>
> > #include <net/tcp.h>
> > @@ -1000,13 +1001,34 @@ static int gve_rx_dqo(struct napi_struct *napi, struct gve_rx_ring *rx,
> > return -ENOMEM;
> > }
> >
> > +/* Prepare a HW-GRO packet the way tcp4/6_gro_complete() prepare a SW-GRO
> > + * packet, so that it can be resegmented by GSO or TSO if it is forwarded:
> > + * seed th->check with the TCP pseudo-header checksum and set CHECKSUM_PARTIAL.
> > + */
> > +static void gve_rx_fixup_gro_checksum(struct sk_buff *skb)
> > +{
> > + int len = skb->len - skb_transport_offset(skb);
> > + struct tcphdr *th = tcp_hdr(skb);
> > +
> > + if (skb_shinfo(skb)->gso_type & SKB_GSO_TCPV4)
> > + th->check = ~tcp_v4_check(len, ip_hdr(skb)->saddr,
> > + ip_hdr(skb)->daddr, 0);
> > + else
> > + th->check = ~tcp_v6_check(len, &ipv6_hdr(skb)->saddr,
> > + &ipv6_hdr(skb)->daddr, 0);
> > +
> > + NAPI_GRO_CB(skb)->count = skb_shinfo(skb)->gso_segs;
> > + tcp_gro_complete(skb);
> > +}
> > +
> > static int gve_rx_complete_rsc(struct sk_buff *skb,
> > const struct gve_rx_compl_desc_dqo *desc,
> > struct gve_ptype ptype)
> > {
> > struct skb_shared_info *shinfo = skb_shinfo(skb);
> > - int rsc_segments, rsc_seg_len, hdr_len;
> > + int rsc_seg_len, hdr_len, nhoff, thoff;
> > skb_frag_t *frag;
> > + __be16 proto;
> > void *va;
> >
> > /* HW-GRO only coalesces TCP. */
> > @@ -1017,20 +1039,11 @@ static int gve_rx_complete_rsc(struct sk_buff *skb,
> > if (!rsc_seg_len)
> > return 0;
> >
> > - switch (ptype.l3_type) {
> > - case GVE_L3_TYPE_IPV4:
> > - shinfo->gso_type = SKB_GSO_TCPV4;
> > - break;
> > - case GVE_L3_TYPE_IPV6:
> > - shinfo->gso_type = SKB_GSO_TCPV6;
> > - break;
> > - default:
> > - return -EINVAL;
>
> Before this patch the L3 type came from ptype.l3_type, now it comes from
> the EtherType. A VLAN tagged frame (gve does not strip tags) would have
> been accepted before, and is now dropped and accounted in
> rx_desc_err_dropped_pkt. nhoff = ETH_HLEN has the same assumption.
>
> Can the device coalesce VLAN tagged frames ?
>
> If not, please say so in the changelog. Otherwise this needs to be
> handled, like idpf_rx_rsc() does with eth_type_vlan().
>
GVE device does not coalesce VLAN tagged frames, so they are never
completed with desc->rsc set. I will explicitly mention that in v2.
> > - }
> > -
> > if (skb_headlen(skb)) {
> > - /* With header-split, payload is in the non-linear part */
> > - rsc_segments = DIV_ROUND_UP(skb->data_len, rsc_seg_len);
> > + /* With header-split or copybreak, eth_type_trans() has already
> > + * pulled the Ethernet header.
> > + */
> > + nhoff = 0;
> > } else {
> > /* HW-GRO packets are guaranteed to have complete TCP/IP
> > * headers in frag[0] when header-split is not enabled.
> > @@ -1039,7 +1052,6 @@ static int gve_rx_complete_rsc(struct sk_buff *skb,
> > va = skb_frag_address(frag);
> > hdr_len =
> > eth_get_headlen(skb->dev, va, skb_frag_size(frag));
> > - rsc_segments = DIV_ROUND_UP(skb->len - hdr_len, rsc_seg_len);
> > skb_copy_to_linear_data(skb, va, hdr_len);
> > skb_frag_size_sub(frag, hdr_len);
> > /* Verify we didn't empty the fragment completely as that could
> > @@ -1049,10 +1061,35 @@ static int gve_rx_complete_rsc(struct sk_buff *skb,
> > skb_frag_off_add(frag, hdr_len);
> > skb->data_len -= hdr_len;
> > skb->tail += hdr_len;
> > + /* napi_gro_frags() pulls the Ethernet header later. */
> > + nhoff = ETH_HLEN;
> > }
> > +
> > + skb_set_network_header(skb, nhoff);
> > + proto = nhoff ? ((struct ethhdr *)skb->data)->h_proto : skb->protocol;
> > + switch (proto) {
> > + case htons(ETH_P_IP):
> > + shinfo->gso_type = SKB_GSO_TCPV4;
> > + thoff = nhoff + ip_hdrlen(skb);
> > + break;
> > + case htons(ETH_P_IPV6):
> > + shinfo->gso_type = SKB_GSO_TCPV6;
> > + thoff = nhoff + sizeof(struct ipv6hdr);
> > + break;
> > + default:
> > + return -EINVAL;
> > + }
> > + skb_set_transport_header(skb, thoff);
> > +
> > shinfo->gso_size = rsc_seg_len;
> > - shinfo->gso_segs = rsc_segments;
> > + shinfo->gso_segs = DIV_ROUND_UP(skb->len - thoff - tcp_hdrlen(skb),
> > + rsc_seg_len);
> > +
> > + if (!IS_ENABLED(CONFIG_INET) ||
> > + unlikely(skb->ip_summed != CHECKSUM_UNNECESSARY))
> > + return 0;
> >
> > + gve_rx_fixup_gro_checksum(skb);
>
> thoff is derived from iph->ihl, and tcp_hdrlen() from th->doff, both read
> from the packet, and nothing checks that the TCP header is actually in
> the linear part before we read th->doff and write th->check.
>
> On the copybreak path the head was allocated by napi_alloc_skb(buf_len),
> so a short frame with desc->rsc set and a large ihl ends up reading and
> writing past the copied data. Same thing on the napi_get_frags() path if
> eth_get_headlen() returned less than the full headers.
>
> The old code was trusting the device as well, but it was not writing
> into the headers.
>
> I think we want something like this, before using tcp_hdr(skb):
>
> if (unlikely(thoff + sizeof(struct tcphdr) > skb_headlen(skb)))
> return -EINVAL;
>
> if (unlikely(skb->len < thoff + tcp_hdrlen(skb)))
> return -EINVAL;
>
> (skb_headlen() is hdr_len on the napi_get_frags() path at this point,
> so one check covers both cases.)
>
> This also makes sure gso_segs can not be computed from a negative length.
>
> Thanks.
Agreed, I'll add these in v2.
Thanks for the review!