Poor NFS performance, kernel 2.6.6.
From: Harald Hannelius
Date: Thu May 27 2004 - 08:18:29 EST
[already posted this on the nfs-list too]
Hi there,
I'm running two servers, both Dual Opterons with Broadcom Gb NICs. I'm
using the bcm5700 driver and I see no errors whatsoever with 'ifconfig'.
These NICs are onboard PCIX:100MHz:64-bit.
I'm using nfs-utils-1.0.6 and the 2.6.6 kernel, compiled with both NFSv3
and NFSv4 support. The client and servers are identical except for the
server having a scsi-raid under /home and the client being and IDE-box.
Distribution slackware-9.1 on both computers.
/etc/exports;
/home 193.167.32.175(ro,no_root_squash,async)
/etc/fstab;
193.167.32.187:/home /home nfs \
defaults,noauto,rsize=32768,wsize=32768,nfsvers=3 0 0
(as you can see I have experimented with different rsize,wsize without any
noticeable effect, same goes for nfsvers 2 and 3. I even mounted the
filesystem as ext2 on the server and as ext3 with data=journal with no
effect)
The computers are connected through a HP 8-port Gb switch and are on the
same subnet.
rsync over ssh gives me roughly 200Mbps (37 GB dataset)
netcat over tcp with a 2GB file gives me 457 Mbps
netcat over udp with a 2GB file gives me 640 Mbps
But dd'ing the 2GB file over nfs as some NFS-HOWTO suggests takes over 5
minutes. That should equal to something around 49 Mbps, correct?
The netcat takes 25 secs to transfer 2GB file.
I installed kernel 2.4.26 and the netcat finished in 29seconds (550Mbps).
With kernel 2.4.26 the 'time cat largefile > /dev/null' over NFS took just
1min3s on the client. Which should equal to something like 254Mbps.
Is there some way I can look at the nfs-server what it's doing? Any
suggestions on solutions for this?
And what datarates over NFS could I expect on a setup like this?
Half-wirespeed maybe?
Thanks in advance,
Harald
--
A: Top Posters
Q: What is the most annoying thing on mailing lists?
Harald H Hannelius | harald/a\arcada.fi | GSM +358 50 594 1020
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/