RE: [PATCH net-next RFC] net: increase LL_MAX_HEADER for Hyper-V

[KY Srinivasan]

> -----Original Message-----
> From: David Laight [mailto:David.Laight@xxxxxxxxxx]
> Sent: Wednesday, September 16, 2015 9:25 AM
> To: Haiyang Zhang <haiyangz@xxxxxxxxxxxxx>; Vitaly Kuznetsov
> <vkuznets@xxxxxxxxxx>; netdev@xxxxxxxxxxxxxxx
> Cc: David S. Miller <davem@xxxxxxxxxxxxx>; linux-kernel@xxxxxxxxxxxxxxx;
> KY Srinivasan <kys@xxxxxxxxxxxxx>; Jason Wang <jasowang@xxxxxxxxxx>
> Subject: RE: [PATCH net-next RFC] net: increase LL_MAX_HEADER for Hyper-
> V
> 
> From: Haiyang Zhang
> > Sent: 16 September 2015 17:09
> > > -----Original Message-----
> > > From: Vitaly Kuznetsov [mailto:vkuznets@xxxxxxxxxx]
> > > Sent: Wednesday, September 16, 2015 11:50 AM
> > > To: netdev@xxxxxxxxxxxxxxx
> > > Cc: David S. Miller <davem@xxxxxxxxxxxxx>; linux-
> kernel@xxxxxxxxxxxxxxx;
> > > KY Srinivasan <kys@xxxxxxxxxxxxx>; Haiyang Zhang
> > > <haiyangz@xxxxxxxxxxxxx>; Jason Wang <jasowang@xxxxxxxxxx>
> > > Subject: [PATCH net-next RFC] net: increase LL_MAX_HEADER for Hyper-
> V
> > >
> > > Commit b08cc79155fc26d0d112b1470d1ece5034651a4b ("hv_netvsc:
> Eliminate
> > >  memory allocation in the packet send path") introduced skb headroom
> > > request for Hyper-V netvsc driver:
> > >
> > >        max_needed_headroom = sizeof(struct hv_netvsc_packet) +
> > >                                sizeof(struct rndis_message) +
> > >                                NDIS_VLAN_PPI_SIZE + NDIS_CSUM_PPI_SIZE +
> > >                                NDIS_LSO_PPI_SIZE + NDIS_HASH_PPI_SIZE;
> > >        ...
> > >        net->needed_headroom = max_needed_headroom;
> > >
> > > max_needed_headroom is 220 bytes, it significantly exceeds the
> > > LL_MAX_HEADER setting. This causes each skb to be cloned on send
> path,
> > > e.g. for IPv4 case we fall into the following clause
> > > (ip_finish_output2()):
> > >
> > > if (unlikely(skb_headroom(skb) < hh_len && dev->header_ops)) {
> > >     ...
> > >     skb2 = skb_realloc_headroom(skb, LL_RESERVED_SPACE(dev));
> > >     ...
> > > }
> > >
> > > leading to a significant performance regression. Increase
> LL_MAX_HEADER
> > > to make it suitable for netvsc, make it 224 to be 16-aligned.
> > > Alternatively we could (partially) revert the commit which introduced
> > > skb
> > > headroom request restoring manual memory allocation on transmit path.
> > >
> > > Signed-off-by: Vitaly Kuznetsov <vkuznets@xxxxxxxxxx>
> > > ---
> > >  include/linux/netdevice.h | 4 +++-
> > >  1 file changed, 3 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
> > > index 88a0069..7233790 100644
> > > --- a/include/linux/netdevice.h
> > > +++ b/include/linux/netdevice.h
> > > @@ -132,7 +132,9 @@ static inline bool dev_xmit_complete(int rc)
> > >   *	used.
> > >   */
> > >
> > > -#if defined(CONFIG_WLAN) || IS_ENABLED(CONFIG_AX25)
> > > +#if IS_ENABLED(CONFIG_HYPERV_NET)
> > > +# define LL_MAX_HEADER 224
> > > +#elif defined(CONFIG_WLAN) || IS_ENABLED(CONFIG_AX25)
> > >  # if defined(CONFIG_MAC80211_MESH)
> > >  #  define LL_MAX_HEADER 128
> > >  # else
> >
> > Thanks for the patch.
> > To avoid we forget to update that 224 number when we add more things
> > into netvsc header, I suggest that we define a macro in netdevice.h such
> > as:
> > #define HVNETVSC_MAX_HEADER 224
> > #define LL_MAX_HEADER HVNETVSC_MAX_HEADER
> >
> > And, put a note in netvsc code saying the header reservation shouldn't
> > exceed HVNETVSC_MAX_HEADER, or you need to update
> HVNETVSC_MAX_HEADER.
> 
> Am I right in thinking this is adding an extra 96 unused bytes to the front
> of almost all skb just so that hyper-v can make its link level header
> contiguous with whatever follows (IP header ?).
> 
> Doesn't sound ideal.

Remote NDIS is the protocol used to send packets from the guest to the host. Every packet
needs to be decorated with the RNDIS header and the maximum room needed for the RNDIS
header is the hreadroom we want.

K. Y
> 
> 	David

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/