Re: cannot fill 100Mbps pipe (over 20ms rtt via netem)?

From: John Heffner
Date: Thu May 01 2008 - 11:39:06 EST


On Wed, Apr 30, 2008 at 11:43 PM, Bill Fink <billfink@xxxxxxxxxxxxxx> wrote:
>
> On Wed, 30 Apr 2008, slashdev wrote:
>
> > On Wed, Apr 30, 2008 at 5:43 PM, slashdev <slashdev@xxxxxxxxx> wrote:
> > >
> > > On Wed, Apr 30, 2008 at 5:27 PM, John Heffner <johnwheffner@xxxxxxxxx> wrote:
> >
> > [snip]
> >
> > > > Both traces show lots of packet loss -- enough to cause an equilibrium
> > > > cwnd of less than 100 KB. With a short RTT, this window is adequate
> > > > to fill a 100 Mbps pipe, but with even a 20 ms RTT, it's hurting you.
> > > > I'm not sure where the packet loss is coming from. Possibly an
> > > > overloaded switch or bad cable?
> > >
> > > thanks for the insight. will take a look at my setup and see why that
> > > should happen. will report back once i am done fixing things at my
> > > end.
> >
> > just a quick note. i turned off TSO on both ends and w/o any cable
> > swap (or switch port change), the thruput jumped from 30Mbps to
> > 60Mbps for the 20ms rtt case. not sure what would cause that.
> >
> > further, the 100 Mbps link that i've been referring to, is actually
> > composed of GigE cards at the endpoints over a 100Mbps vlan.
> >
> > just wanted to point out these two facts while i get busy fixing
> > up any potential cabling or switch problems. will also report back
> > how does cross-over cable fair in this setup...
>
> The rate mismatch from GigE down to 100 Mbps can cause serious
> performance degradation due to packet loss if there is insufficient
> buffering in the switch. Rate limiting by judicious setting of the
> window size might help (with nuttcp could also try rate limiting
> using the "-Ri90m" parameter).


This would be my number one guess. Certain policing algorithms don't
play well with TCP, either -- especially with a higher BDP.

<plug>This is the type of problem that 'pathdiag' is designed to find.
http://www.psc.edu/networking/projects/pathdiag/ </plug>

-John
--
To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html