Re: W/RTT verification, linux tcp buffers behaviour

From: Al Boldi
Date: Thu May 18 2006 - 23:35:55 EST

John Heffner wrote:
> Al Boldi wrote:
> > David S. Miller wrote:
> >> The tcp_adv_win_scale determines how much of the "socket buffer
> >> space" we advertise in the actual TCP window. It's mean to
> >> approximate the amount of structure and header overhead is
> >> assosciated with each packet.
> >>
> >> But again, it's an approximation and there are cases where it
> >> is suboptimal.
> >
> > Ok, so is this suboptimal approximation the reason why the kernel does
> > not reach full bandwidth?
> Not really.
> There are a few reasons why concurrent connections can get better
> performance than a single one. Each socket buffer has a per-connection
> size limit. If this size is too small to fill the bandwidth-delay
> product of your path, then using multiple connections will give you
> better performance up to n connections, where n = ceiling(BDP /
> (buffer_size * fudge_factor)). The fudge factor is determined by the
> overhead. Recent kernels do automatic tuning of the TCP buffers, so if
> your system limits (in /proc/sys/net/ipv4/tcp_{wr}mem) are set high
> enough, then this shouldn't be an issue. The 2.6.17 release will have
> significantly higher maximums by default -- 4 MB, up from ~100k.

Ok, but increasing them only has minimal impact.

> The other way concurrent connections can give you better performance is
> if congestion control is your limiting factor. Standard TCP uses an
> AIMD controller with an additive increase value a=1*MSS and a backoff
> factor of b=0.5. With n connections, you actually end up with an
> aggregate a=n*MSS, so it's more aggressive. If you're competing against
> other flows, your aggregate will get a higher fair share of the
> bottleneck. If you are not competing against any traffic, but you have
> a small bottleneck queue and unsynchronized losses, this can give you
> higher utilization. If congestion control is the issue, there are
> better alternatives than concurrent connections. Linux now has the
> ability to select from a variety of experimental congestion control
> algorithms loaded as modules.

Which module would you suggest for a non-congested network?

Or is there a module that allows for a=(n+x)*MSS where x is tunable?

> If you are actually CPU limited, on an SMP system you can also get
> better performance with multiple connections, since they can be handled
> concurrently by multiple CPU's.

It seems that CPU cycles are related to latency, which is related to
bandwidth-delay. So a faster CPU should yield higher thruput, which it
does, but this would mean, that thruput is dependent on system load as that
affects latency.

Is there a module that takes this fluctuation into account, to yield a
constant flow?

Thanks for your detailed response!


To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at