Re: cannot fill 100Mbps pipe (over 20ms rtt via netem)?

From: slashdev
Date: Thu May 01 2008 - 12:12:10 EST


On Wed, Apr 30, 2008 at 11:43 PM, Bill Fink <billfink@xxxxxxxxxxxxxx> wrote:
>
> On Wed, 30 Apr 2008, slashdev wrote:
>
> > On Wed, Apr 30, 2008 at 5:43 PM, slashdev <slashdev@xxxxxxxxx> wrote:
> > >
> > > On Wed, Apr 30, 2008 at 5:27 PM, John Heffner <johnwheffner@xxxxxxxxx> wrote:
> >
> > [snip]
> >
> > > > Both traces show lots of packet loss -- enough to cause an equilibrium
> > > > cwnd of less than 100 KB. With a short RTT, this window is adequate
> > > > to fill a 100 Mbps pipe, but with even a 20 ms RTT, it's hurting you.
> > > > I'm not sure where the packet loss is coming from. Possibly an
> > > > overloaded switch or bad cable?
> > >
> > > thanks for the insight. will take a look at my setup and see why that
> > > should happen. will report back once i am done fixing things at my
> > > end.
> >
> > just a quick note. i turned off TSO on both ends and w/o any cable
> > swap (or switch port change), the thruput jumped from 30Mbps to
> > 60Mbps for the 20ms rtt case. not sure what would cause that.
> >
> > further, the 100 Mbps link that i've been referring to, is actually
> > composed of GigE cards at the endpoints over a 100Mbps vlan.
> >
> > just wanted to point out these two facts while i get busy fixing
> > up any potential cabling or switch problems. will also report back
> > how does cross-over cable fair in this setup...
>
> The rate mismatch from GigE down to 100 Mbps can cause serious
> performance degradation due to packet loss if there is insufficient
> buffering in the switch. Rate limiting by judicious setting of the
> window size might help (with nuttcp could also try rate limiting
> using the "-Ri90m" parameter).

thanks for reminding me of that option :-)

but on my setup, i get maximum thruput (without changing any cables
or switch ports) with TSO turned OFF. with it being ON, rate limiting
nuttcp does not help. the thruput stays same low at around 30Mbps.

because i guess rate-limiting in this case is at much higher layers,
and does not prevent the card from injecting large # segments
segments on the wire (when TSO is enabled) -- causing the switch
to drop packets (due to large instantaneous incoming rate).

does that logic make sense?
--
To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html