Tuning Linux for faster high-latency outbound transfers

From: Ketil Froyn
Date: Wed Jan 27 2010 - 05:45:48 EST


Hi,

I am based in Norway, and I'm benchmarking upload and download speeds
to and from a US site (Rackspace Cloud Files). I'm on a 1Gbit link
with a 100Mbit firewall limiting my speeds, but I routinely see both
uploads and downloads at 20Mbit, and spikes up to 50Mbit. Ping latency
between my site and the US site is about 170ms.

The speeds I'm seeing for single large file transfers when benchmarking are:

uploads: 90kb/s - 140kb/s
downloads: 450kb/s - 500kb/s

I also tested from a virtual (Xen-based) server I have access to in
the US, and there I see both uploads and downloads at 800-900kb/s. (I
also noted that copying my 100MB test file locally took about 1
minute, so this virtual server has some other limitations too.)

Since the network in Norway isn't asymmetric, and is quite a lot
slower than the US test speeds, I figure the slowness I see from
Norway could be down to TCP tuning on my end node. I have tried some
tips from these sites without much luck:

http://fasterdata.es.net/TCP-tuning/linux.html
http://www.psc.edu/networking/projects/tcptune/

Specifically, I did this:

echo 12582911 > /proc/sys/net/core/wmem_max
echo 12582911 > /proc/sys/net/core/rmem_max

and

echo '10240 87380 12582912' > /proc/sys/net/ipv4/tcp_wmem
echo '10240 87380 12582912' > /proc/sys/net/ipv4/tcp_rmem

but the initial transfer (before changing these) was fastest.

I'd appreciate any suggestions for what I could do to speed up
outbound transfers or diagnose the issue more accurately. My system is
a Debian 5.0.3 "lenny" system with a 2.6.26-2-xen-amd64 SMP kernel,
and it has a Broadcom NetXtreme II BCM5708 1000Base-T (B2) PCI-X
64-bit 133MHz NIC.

Thanks for your time.

Regards, Ketil
--
To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html