Hi there,
Firstly, IANAKH (I am not a kernel-hacker), so please CC: everything to
linux-raid. And be gentle with me :)
I have been experimenting with RAID for some time, and found that the big
read-access slowdown between kernels 2.2.15 and 2.2.16 on RAID-1 mirrors
was because of the raw disk accesses (on IDE) rather than a RAID problem.
I benchmarked single IDE disk performance on the following setup:
Intel 810E Chipset Motherboard (CA810EAL), Pentium III-667, 32M RAM,
Maxtor DiamondMax Plus 40 40.9GB UDMA66 Disk, Model 54098U8
I have attached the (edited) kernel config I used for all 2.2 kernels FYI.
I used tiotest to benchmark, using a file size of 256MB, block size of 4K,
and with 1, 2, 4, 16, 32 threads. The performance starts to get hit as
soon as you have multiple threads reading simultaneously, and degrades
much more quickly on 2.2.16. Strangely, writes do not degrade.
Size is MB, BlkSz is Bytes, Read and Write are MB/sec, Seeks are Seeks/sec
==> 2.2.15 <==
Dir Size BlkSz Thr# Read (CPU%) Write (CPU%) Seeks (CPU%)
----- ------ ------- ---- ------------- -------------- --------------
/mnt/ 256 4096 1 27.1371 10.3% 26.7979 23.0% 146.187 0.95%
/mnt/ 256 4096 2 27.1219 10.7% 26.6606 23.2% 142.233 0.60%
/mnt/ 256 4096 4 26.9915 10.6% 26.4289 22.9% 142.789 0.50%
/mnt/ 256 4096 16 26.4320 10.5% 26.1310 23.0% 147.424 0.52%
/mnt/ 256 4096 32 25.3407 10.1% 25.6822 22.7% 150.750 0.57%
==> 2.2.16 <==
Dir Size BlkSz Thr# Read (CPU%) Write (CPU%) Seeks (CPU%)
----- ------ ------- ---- ------------- -------------- --------------
/mnt/ 256 4096 1 27.0948 10.5% 25.9272 22.5% 147.264 0.83%
/mnt/ 256 4096 2 20.5753 8.42% 25.9316 22.7% 143.113 0.56%
/mnt/ 256 4096 4 13.0397 6.09% 25.7723 22.7% 143.619 0.51%
/mnt/ 256 4096 16 9.38573 5.40% 25.6285 23.1% 147.815 0.51%
/mnt/ 256 4096 32 6.96358 5.90% 25.2889 22.9% 151.182 0.57%
The performance of 2.2.17-pre9 is basically exactly the same as 2.2.16,
except that it died with do_vm_free_pages() when trying to run it with 64
threads.
I had a look in drivers/block/ide.c, and the only big change to seems to
be adding a new element 'expiry' to ide_hwgroup_t. ide_set_handler() has
been changed to take it as an extra argument, however, the only place that
I can find calls to ide_set_handler() with a non-NULL expiry is in
ide-cd.c, (I haven't tested CD-ROMs to see if there's a slow-down there as
well). All calls to ide_set_handler() in ide.c and ide-disk.c specify
expiry as NULL.
The only place where hwgroup->expiry is set is in ide_set_handler (always
passed NULL by the HD routines). The only place that it is examined is in
ide_timer_expiry() which should not execute any more code if expiry is
NULL.
I also tested 2.4.0-test5, and performance there sucks with one thread,
but doesn't degrade any more quickly than 2.2.15
==> 2.4.0-test5 <==
Dir Size BlkSz Thr# Read (CPU%) Write (CPU%) Seeks (CPU%)
----- ------ ------- ---- ------------- -------------- --------------
/mnt/ 256 4096 1 8.15398 49.6% 7.64104 71.2% 125.675 5.90%
/mnt/ 256 4096 2 8.01528 56.1% 7.63085 69.8% 123.076 5.59%
/mnt/ 256 4096 4 7.79720 61.6% 7.62060 68.1% 125.437 5.75%
/mnt/ 256 4096 16 7.64617 65.2% 7.60466 62.5% 129.431 5.83%
/mnt/ 256 4096 32 7.60580 67.6% 7.58702 51.7% 132.791 5.95%
I haven't tried any of this with SCSI disks, so the problem may well lie
in VFS or the VM subsystem (I know there are problems there as well). I
don't know anything about those, so I guess I should just leave it to the
professionals now :)
Thanks,
Corin
/------------------------+-------------------------------------\
| Corin Hartland-Swann | Direct: +44 (0) 20 7544 4676 |
| Commerce Internet Ltd | Mobile: +44 (0) 79 5854 0027 |
| 22 Cavendish Buildings | Tel: +44 (0) 20 7491 2000 |
| Gilbert Street | Fax: +44 (0) 20 7491 2010 |
| Mayfair | Web: http://www.commerce.uk.net/ |
| London W1K 5HJ | E-Mail: cdhs@commerce.uk.net |
\------------------------+-------------------------------------/
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/
This archive was generated by hypermail 2b29 : Tue Aug 15 2000 - 21:00:21 EST