Re: Ext3 meta-data performance
From: Nick Piggin (piggin@cyberone.com.au)
Date: Thu May 29 2003 - 08:08:25 EST
Kevin Jacobs wrote:
I've recently upgraded our company rsync/backup server and have been running
into performance slowdowns. The old system was a dual processor Pentium III
(Coppermine) 866MHz running Redhat 7.3 with IDE disks (ext2 filesystems).
We have since upgraded it to Redhat 9, added a 3Ware 7500-8 RAID controller
and more disks (w/ ext3 filesystems + external journal).
The primary use for this system is to provide live rsync snapshots of
several of our primary servers. For each system we maintain a "current"
snapshot, from which a hard-linked image is taken after each rsync update.
i.e., we rsync and then 'cp -Rl current snapshot-$DATE'. After the update
to Redhat 9, the rsync itself was faster, but the time to make the
hard-links became an order of magnitude slower (~4min -> ~50min for a tree
with 500,000 files). Not only was it slower, but it destroyed system
interactivity for minutes at a time.
Since these rsync backups are done in addition to traditional daily tape
backups, we've taken the system out of production use and opened the door
for experimentation. So, the next logical step was to try a 2.5 kernel.
After some work, I've gotten 2.5.70-mm2 booting and it is _much_ better than
the Redhat 2.4 kernels, and the system interactivity is flawless. However,
the speed of creating hard-links is still three and a half times slower than
with the old 2.2 kernel. It now takes ~14 minutes to create the links, and
from what I can tell, the bottlenecks is not the CPU or the disk-throughput.
Its probably seek bound.
Provide some more information about your disk/partition setup, and external
journals, and data= mode. Remember ext3 will generally always have to do
more work than ext2.
If you want to play with the scheduler, try set
/sys/block/blockdev*/queue/nr_requests = 8192
then try
/sys/block/blockdev*/queue/iosched/antic_expire = 0
Try the above combinations with and without a big TCQ depth. You should
be able to set them on the fly and see what happens to throughput during
the operation. Let me know what you see.
Nice to see interactivity is good though.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/