Re: W/RTT verification, linux tcp buffers behaviour

From: John Heffner
Date: Thu May 18 2006 - 14:26:09 EST

Al Boldi wrote:
David S. Miller wrote:
The tcp_adv_win_scale determines how much of the "socket buffer
space" we advertise in the actual TCP window. It's mean to
approximate the amount of structure and header overhead is
assosciated with each packet.

But again, it's an approximation and there are cases where it
is suboptimal.

Ok, so is this suboptimal approximation the reason why the kernel does not reach full bandwidth?

Not really.

There are a few reasons why concurrent connections can get better performance than a single one. Each socket buffer has a per-connection size limit. If this size is too small to fill the bandwidth-delay product of your path, then using multiple connections will give you better performance up to n connections, where n = ceiling(BDP / (buffer_size * fudge_factor)). The fudge factor is determined by the overhead. Recent kernels do automatic tuning of the TCP buffers, so if your system limits (in /proc/sys/net/ipv4/tcp_{wr}mem) are set high enough, then this shouldn't be an issue. The 2.6.17 release will have significantly higher maximums by default -- 4 MB, up from ~100k.

The other way concurrent connections can give you better performance is if congestion control is your limiting factor. Standard TCP uses an AIMD controller with an additive increase value a=1*MSS and a backoff factor of b=0.5. With n connections, you actually end up with an aggregate a=n*MSS, so it's more aggressive. If you're competing against other flows, your aggregate will get a higher fair share of the bottleneck. If you are not competing against any traffic, but you have a small bottleneck queue and unsynchronized losses, this can give you higher utilization. If congestion control is the issue, there are better alternatives than concurrent connections. Linux now has the ability to select from a variety of experimental congestion control algorithms loaded as modules.

If you are actually CPU limited, on an SMP system you can also get better performance with multiple connections, since they can be handled concurrently by multiple CPU's.


To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at