benchmark: write barrier/write cache on XFS

From: Martin Steigerwald
Date: Mon Dec 08 2008 - 15:40:20 EST


Hi!

I got curious about the recent discussions about the write barrier
feature[1][2].

Thus I did my own benchmark this evening. Since I use XFS and tested with
XFS for now only. Write barrier + write cache, no barrier + write cache,
no write barrier + no write cache. I just did tar -xf
linux-2.6.27.tar.bz2 and rm -rf linux-2.6.27.

My conclusion is: At least for this metadata intensive workload enabling
write barries is complete nonsense for XFS filesystems, cause it runs way
faster without write cache and without barriers. I am completely puzzled
about this, cause I always thought that barriers where meant to provide a
performance improvement versus disabling write cache. I actually
advertised them as such in my Linux magazine article, should have benched
them before it seems. Write barrier seem to be a slow down feature for
XFS.

See for yourself. I am interested in other benchmarks like this.
Automating this would be good and running it for different filesystems.
Maybe adding some different workloads as these are highly selective
tests, testing with RAID systems as Justin did and of course testing with
different filesystems.

I think I will disable write cache and disable barriers on that ThinkPad
T42 and I think also other machines.



IBM ThinkPad T42 with 160 GB IDE hitachi drive via libata:

shambhala:~> grep "model name" /proc/cpuinfo
model name : Intel(R) Pentium(R) M processor 1.80GHz
shambhala:~> cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
ondemand
shambhala:~> cat /proc/version
Linux version 2.6.27.7-tp42-toi-3.0-2008-11-25 (martin@shambhala) (gcc
version 4.3.2 (Debian 4.3.2-1) ) #1 PREEMPT Sun Nov 30 10:29:09 CET 2008

shambhala:~> hdparm -I /dev/sda | egrep "(Model Num|device size|power
manage|acoustic manage|DMA:|120 ns)"
Model Number: Hitachi HTS541616J9AT00
device size with M = 1024*1024: 152627 MBytes
device size with M = 1000*1000: 160041 MBytes (160 GB)
Advanced power management level: 254
Recommended acoustic management value: 128, current value: 128
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 *udma5



The default - write barrier and write cache enabled:

shambhala:~> hdparm -W /dev/sda

/dev/sda:
write-caching = 1 (on)
shambhala:~> hdparm -I /dev/sda | grep cache
* Write cache
shambhala:~> mkfs.xfs -f -L xfs -l lazy-count=1 /dev/sda6
meta-data=/dev/sda6 isize=256 agcount=4, agsize=1344188
blks
= sectsz=512 attr=2
data = bsize=4096 blocks=5376750, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096
log =internal log bsize=4096 blocks=2625, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
shambhala:~> mount /dev/sda6 /mnt/zeit
shambhala:~> grep zeit /proc/mounts
/dev/sda6 /mnt/zeit xfs rw,attr2,noquota 0 0
shambhala:~> cd /mnt/zeit
shambhala:/mnt/zeit> sync ; time
tar -xf /home/martin/Linux/Kernel/Mainline/linux-2.6.27.tar.bz2; time
sync
tar -xf /home/martin/Linux/Kernel/Mainline/linux-2.6.27.tar.bz2 46,01s
user 5,59s system 41% cpu 2:03,01 total
sync 0,00s user 0,18s system 3% cpu 5,303 total
shambhala:/mnt/zeit> sync ; time rm -rf linux-2.6.27 ; time sync
rm -rf linux-2.6.27 0,08s user 3,49s system 6% cpu 51,482 total
sync 0,00s user 0,15s system 45% cpu 0,320 total
shambhala:/mnt/zeit> cd
shambhala:~> umount /mnt/zeit



Write barriers disabled, write cache enabled:

shambhala:~> hdparm -W /dev/sda

/dev/sda:
write-caching = 1 (on)
shambhala:~> hdparm -I /dev/sda | grep cache
* Write cache
shambhala:~> mkfs.xfs -f -L xfs -l lazy-count=1 /dev/sda6
meta-data=/dev/sda6 isize=256 agcount=4, agsize=1344188
blks
= sectsz=512 attr=2
data = bsize=4096 blocks=5376750, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096
log =internal log bsize=4096 blocks=2625, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
shambhala:~> mount -o nobarrier /dev/sda6 /mnt/zeit
shambhala:~> cd /mnt/zeit
shambhala:/mnt/zeit> sync ; time
tar -xf /home/martin/Linux/Kernel/Mainline/linux-2.6.27.tar.bz2; time
sync
tar -xf /home/martin/Linux/Kernel/Mainline/linux-2.6.27.tar.bz2 26,77s
user 3,20s system 67% cpu 44,236 total
sync 0,00s user 0,17s system 5% cpu 3,224 total
shambhala:/mnt/zeit> sync ; time rm -rf linux-2.6.27 ; time sync
rm -rf linux-2.6.27 0,05s user 3,20s system 43% cpu 7,442 total
sync 0,00s user 0,14s system 44% cpu 0,309 total
shambhala:/mnt/zeit> cd
shambhala:~> umount /mnt/zeit



Write barriers and write cache disabled:

shambhala:~> hdparm -W0 /dev/sda

/dev/sda:
setting drive write-caching to 0 (off)
write-caching = 0 (off)
shambhala:~> hdparm -I /dev/sda | grep cache
Write cache
shambhala:~> mkfs.xfs -f -L xfs -l lazy-count=1 /dev/sda6
meta-data=/dev/sda6 isize=256 agcount=4, agsize=1344188
blks
= sectsz=512 attr=2
data = bsize=4096 blocks=5376750, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096
log =internal log bsize=4096 blocks=2625, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
shambhala:~> mount -o nobarrier /dev/sda6 /mnt/zeit
shambhala:~> cd /mnt/zeit
shambhala:/mnt/zeit> sync ; time
tar -xf /home/martin/Linux/Kernel/Mainline/linux-2.6.27.tar.bz2; time
sync
tar -xf /home/martin/Linux/Kernel/Mainline/linux-2.6.27.tar.bz2 35,22s
user 3,89s system 53% cpu 1:12,71 total
sync 0,00s user 0,17s system 7% cpu 2,355 total
shambhala:/mnt/zeit> sync ; time rm -rf linux-2.6.27 ; time sync
rm -rf linux-2.6.27 0,07s user 2,82s system 12% cpu 23,391 total
sync 0,00s user 0,14s system 44% cpu 0,329 total
shambhala:/mnt/zeit> cd
shambhala:~> umount /mnt/zeit


shambhala:~> date
Mo 8. Dez 21:24:42 CET 2008





[1] http://oss.sgi.com/archives/xfs/2008-12/msg00219.html
[2] http://oss.sgi.com/archives/xfs/2008-12/msg00161.html or
http://lkml.org/lkml/2008/12/4/169

Ciao,
--
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7

Attachment: signature.asc
Description: This is a digitally signed message part.