Re: [PATCH] net: add per device sg_max_frags for skb

[Hans Westgaard Ry]

On 01/06/2016 02:59 PM, David Laight wrote:
> From: Hans Westgaard Ry
> > Sent: 06 January 2016 13:16
> > Devices may have limits on the number of fragments in an skb they
> > support. Current codebase uses a constant as maximum for number of
> > fragments (MAX_SKB_FRAGS) one skb can hold and use.
> >
> > When enabling scatter/gather and running traffic with many small
> > messages the codebase uses the maximum number of fragments and thereby
> > violates the max for certain devices.
> >
> > An example of such a violation is when running IPoIB on a HCA
> > supporting 16 SGE on an architecture with 4K pagesize. The
> > MAX_SKB_FRAGS will be 17 (64K/4K+1) and because IPoIB adds yet another
> > segment we end up with send_requests with 18 SGE resulting in
> > kernel-panic.
> >
> > The patch allows the device to limit the maximum number fragments used
> > in one skb.
> This doesn't seem to me to be the correct way to fix this.
> Anything that adds an extra fragment (in this case IPoIB) should allow
> for the skb already having the maximum number of fragments.
> Fully linearising the skb is overkill, but I think the first fragment
> can be added to the linear part of the skb.
>
> 	David
When IpoIB handles a skb-request it converts fragments to SGEs to
be handled by a HCA.
The problem arises when the HCA have a limited number of SGEs less than 
MAX_SKB_FRAGS.
(it gets a little worse since IPoIB need to yet another segment)
I have not found any easy way of fixing this with currenct codebase.

Hans