Re: cannot fill 100Mbps pipe (over 20ms rtt via netem)?

From: Bill Fink
Date: Thu May 01 2008 - 23:13:12 EST


On Thu, 1 May 2008, slashdev wrote:

> On Wed, Apr 30, 2008 at 11:43 PM, Bill Fink <billfink@xxxxxxxxxxxxxx> wrote:
> >
> > On Wed, 30 Apr 2008, slashdev wrote:
> >
> > > just a quick note. i turned off TSO on both ends and w/o any cable
> > > swap (or switch port change), the thruput jumped from 30Mbps to
> > > 60Mbps for the 20ms rtt case. not sure what would cause that.
> > >
> > > further, the 100 Mbps link that i've been referring to, is actually
> > > composed of GigE cards at the endpoints over a 100Mbps vlan.
> > >
> > > just wanted to point out these two facts while i get busy fixing
> > > up any potential cabling or switch problems. will also report back
> > > how does cross-over cable fair in this setup...
> >
> > The rate mismatch from GigE down to 100 Mbps can cause serious
> > performance degradation due to packet loss if there is insufficient
> > buffering in the switch. Rate limiting by judicious setting of the
> > window size might help (with nuttcp could also try rate limiting
> > using the "-Ri90m" parameter).
>
> thanks for reminding me of that option :-)
>
> but on my setup, i get maximum thruput (without changing any cables
> or switch ports) with TSO turned OFF. with it being ON, rate limiting
> nuttcp does not help. the thruput stays same low at around 30Mbps.
>
> because i guess rate-limiting in this case is at much higher layers,
> and does not prevent the card from injecting large # segments
> segments on the wire (when TSO is enabled) -- causing the switch
> to drop packets (due to large instantaneous incoming rate).
>
> does that logic make sense?

Yes, it makes perfect sense. I have seen similar effects with
other rate mismatch situations. And I see from your other reply
that when there is no rate mismatch you were able to fill a GigE
pipe as expected.

-Bill
--
To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html