Performance problems when writing large files on CCISS hardware

From: Martin Knoblauch
Date: Wed Jan 23 2008 - 09:47:03 EST

Next message: Alasdair G Kergon: "Re: [dm-devel] [PATCH 0/2] dm-band: The I/O bandwidth controller: Overview"
Previous message: Robert Hancock: "Re: sata_nv and 2.6.24 (was Re: fixed a bug of adma in rhel4u5 withHDS7250SASUN500G.)"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

Please CC me on replies, as I am not subscribed.

Hi,

for a while now I am having problems writing large files sequentially to EXT2 filesystems on CCISS based boxes. The problem is that writing multiple files in parallel is extremely slow compared to a single file in non-DIO mode. When using DIO, the scaling is almost "perfect". The problem manifests itself in RHEL4 kernels (2.6.9-X) and any mainline kernel up to 2.6.24-rc8.

The systems in question are HP/DL380G4 with 2 cpus, 8 GB memory, SmartArray6i (CCISS) with BBWC and 4x72GB@10krpm disks in RAID5 configuration. Environment is 64-bit RHEL4.3.

The problem can be reproduced by running 1, 2 or 3 parallel "dd" processes, or "iozone" with 1, 2 or 3 threads. Curiously, there was a period from 2.6.24-rc1 until 2.6.24-rc5 where the problem went away. It turned out that this was due to a "regression" that was "fixed" by below commit. Unfortunatelly this is not good for my systems, but it might shed some light on the underlying problem:

> #commit 81eabcbe0b991ddef5216f30ae91c4b226d54b6d
> #Author: Mel Gorman <mel@xxxxxxxxx>
> #Date: Mon Dec 17 16:20:05 2007 -0800
> #
> # mm: fix page allocation for larger I/O segments
> #
> # In some cases the IO subsystem is able to merge requests if the
pages are
> # adjacent in physical memory. This was achieved in the allocator
by having
> # expand() return pages in physically contiguous order in
situations were a
> # large buddy was split. However, list-based anti-fragmentation
changed the
> # order pages were returned in to avoid searching in
buffered_rmqueue() for a
> # page of the appropriate migrate type.
> #
> # This patch restores behaviour of rmqueue_bulk() preserving the
physical
> # order of pages returned by the allocator without incurring
increased search
> # costs for anti-fragmentation.
> #
> # Signed-off-by: Mel Gorman <mel@xxxxxxxxx>
> # Cc: James Bottomley <James.Bottomley@xxxxxxxxxxxx>
> # Cc: Jens Axboe <jens.axboe@xxxxxxxxxx>
> # Cc: Mark Lord <mlord@xxxxxxxxx
> # Signed-off-by: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
> # Signed-off-by: Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx>
> diff -urN linux-2.6.24-rc5/mm/page_alloc.c
linux-2.6.24-rc6/mm/page_alloc.c
> --- linux-2.6.24-rc5/mm/page_alloc.c 2007-12-21 04:14:11.305633890
+0000
> +++ linux-2.6.24-rc6/mm/page_alloc.c 2007-12-21 04:14:17.746985697
+0000
> @@ -847,8 +847,19 @@
> struct page *page = __rmqueue(zone, order,
migratetype);
> if (unlikely(page == NULL))
> break;
> +
> + /*
> + * Split buddy pages returned by expand() are
received here
> + * in physical page order. The page is added to the
callers and
> + * list and the list head then moves forward. From
the callers
> + * perspective, the linked list is ordered by page
number in
> + * some conditions. This is useful for IO devices
that can
> + * merge IO requests if the physical pages are
ordered
> + * properly.
> + */
> list_add(&page->lru, list);
> set_page_private(page, migratetype);
> + list = &page->lru;
> }
> spin_unlock(&zone->lock);
> return i;
>

Reverting this patch from 2.6.24-rc8 gives the good performance reported below (rc8*). So, apparently CCISS is very sensitive to the page ordering.

Here are the numbers (MB/sec) including sync-time. I compare 2.6.24-rc8 (rc8) and 2.6.24-rc8 with abore commit reverted (rc8*). Reported is the combined throughput for 1,2,3 iozone threads, for reference also the DIO numbers. Raw numbers are attached.

Test rc8 rc8*
----------------------------------------
1x3GB 56 90
1x3GB-DIO 86 86
2x1.5GB 9.5 87
2x1.5GB-DIO 80 85
3x1GB 16.5 85
3x1GB-DIO 85 85

One can see that in mainline/rc8 all non-DIO numbers are smaller than the corresponding DIO numbers, or the non-DIO numbers from rc8*. The performance for 2 and 3 threads in mainline/rc8 is just bad.

Of course I have the option to revert commit ....54b6d for my systems, but I think a more general solution would be better. If I can help tracking the real problem down, I am open for suggestions.

Cheers
Martin

------------------------------------------------------
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www: http://www.knobisoft.de

Iozone: Performance Test of File I/O
Version $Revision: 3.283 $
Compiled for 64 bit mode.
Build: linux-AMD64

Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
Erik Habbinga, Kris Strecker, Walter Wong.

Run began: Wed Jan 23 14:53:26 2008

Record Size 1024 KB
File size set to 3072000 KB
Include close in write timing
Include fsync in write timing
Command line used: ./iozone -r 1m -s 3000m -c -e -t 1 -F /scratch/X1 -i 0
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 1 process
Each process writes a 3072000 Kbyte file in 1024 Kbyte records

Children see throughput for 1 initial writers = 55847.39 KB/sec
Parent sees throughput for 1 initial writers = 55844.87 KB/sec
Min throughput per process = 55847.39 KB/sec
Max throughput per process = 55847.39 KB/sec
Avg throughput per process = 55847.39 KB/sec
Min xfer = 3072000.00 KB

Children see throughput for 1 rewriters = 60760.14 KB/sec
Parent sees throughput for 1 rewriters = 60757.17 KB/sec
Min throughput per process = 60760.14 KB/sec
Max throughput per process = 60760.14 KB/sec
Avg throughput per process = 60760.14 KB/sec
Min xfer = 3072000.00 KB

iozone test complete.
Iozone: Performance Test of File I/O
Version $Revision: 3.283 $
Compiled for 64 bit mode.
Build: linux-AMD64

Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
Erik Habbinga, Kris Strecker, Walter Wong.

Run began: Wed Jan 23 14:55:22 2008

Record Size 1024 KB
File size set to 3072000 KB
Include close in write timing
Include fsync in write timing
O_DIRECT feature enabled
Command line used: ./iozone -r 1m -s 3000m -c -e -I -t 1 -F /scratch/X1 -i 0
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 1 process
Each process writes a 3072000 Kbyte file in 1024 Kbyte records

Children see throughput for 1 initial writers = 86304.83 KB/sec
Parent sees throughput for 1 initial writers = 86300.38 KB/sec
Min throughput per process = 86304.83 KB/sec
Max throughput per process = 86304.83 KB/sec
Avg throughput per process = 86304.83 KB/sec
Min xfer = 3072000.00 KB

Children see throughput for 1 rewriters = 85388.78 KB/sec
Parent sees throughput for 1 rewriters = 85379.37 KB/sec
Min throughput per process = 85388.78 KB/sec
Max throughput per process = 85388.78 KB/sec
Avg throughput per process = 85388.78 KB/sec
Min xfer = 3072000.00 KB

iozone test complete.
Iozone: Performance Test of File I/O
Version $Revision: 3.283 $
Compiled for 64 bit mode.
Build: linux-AMD64

Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
Erik Habbinga, Kris Strecker, Walter Wong.

Run began: Wed Jan 23 14:56:41 2008

Record Size 1024 KB
File size set to 1536000 KB
Include close in write timing
Include fsync in write timing
Command line used: ./iozone -r 1m -s 1500m -c -e -t 2 -F /scratch/X1 /scratch/X2 -i 0
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 2 processes
Each process writes a 1536000 Kbyte file in 1024 Kbyte records

Children see throughput for 2 initial writers = 9461.15 KB/sec
Parent sees throughput for 2 initial writers = 9455.69 KB/sec
Min throughput per process = 4726.85 KB/sec
Max throughput per process = 4734.29 KB/sec
Avg throughput per process = 4730.57 KB/sec
Min xfer = 1533952.00 KB

Children see throughput for 2 rewriters = 9780.23 KB/sec
Parent sees throughput for 2 rewriters = 9771.18 KB/sec
Min throughput per process = 4885.66 KB/sec
Max throughput per process = 4894.57 KB/sec
Avg throughput per process = 4890.12 KB/sec
Min xfer = 1536000.00 KB

iozone test complete.
Iozone: Performance Test of File I/O
Version $Revision: 3.283 $
Compiled for 64 bit mode.
Build: linux-AMD64

Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
Erik Habbinga, Kris Strecker, Walter Wong.

Run began: Wed Jan 23 15:07:30 2008

Record Size 1024 KB
File size set to 1536000 KB
Include close in write timing
Include fsync in write timing
O_DIRECT feature enabled
Command line used: ./iozone -r 1m -s 1500m -c -e -I -t 2 -F /scratch/X1 /scratch/X2 -i 0
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 2 processes
Each process writes a 1536000 Kbyte file in 1024 Kbyte records

Children see throughput for 2 initial writers = 79399.89 KB/sec
Parent sees throughput for 2 initial writers = 48569.16 KB/sec
Min throughput per process = 32239.45 KB/sec
Max throughput per process = 47160.45 KB/sec
Avg throughput per process = 39699.95 KB/sec
Min xfer = 1050624.00 KB

Children see throughput for 2 rewriters = 80684.21 KB/sec
Parent sees throughput for 2 rewriters = 80342.38 KB/sec
Min throughput per process = 27509.90 KB/sec
Max throughput per process = 53174.30 KB/sec
Avg throughput per process = 40342.10 KB/sec
Min xfer = 799744.00 KB

iozone test complete.
Iozone: Performance Test of File I/O
Version $Revision: 3.283 $
Compiled for 64 bit mode.
Build: linux-AMD64

Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
Erik Habbinga, Kris Strecker, Walter Wong.

Run began: Wed Jan 23 15:09:00 2008

Record Size 1024 KB
File size set to 1024000 KB
Include close in write timing
Include fsync in write timing
Command line used: ./iozone -r 1m -s 1000m -c -e -t 3 -F /scratch/X1 /scratch/X2 /scratch/X3 -i 0
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 3 processes
Each process writes a 1024000 Kbyte file in 1024 Kbyte records

Children see throughput for 3 initial writers = 16572.10 KB/sec
Parent sees throughput for 3 initial writers = 16228.59 KB/sec
Min throughput per process = 4908.56 KB/sec
Max throughput per process = 5942.16 KB/sec
Avg throughput per process = 5524.03 KB/sec
Min xfer = 815104.00 KB

Children see throughput for 3 rewriters = 17564.65 KB/sec
Parent sees throughput for 3 rewriters = 17144.66 KB/sec
Min throughput per process = 5643.31 KB/sec
Max throughput per process = 6190.94 KB/sec
Avg throughput per process = 5854.88 KB/sec
Min xfer = 995328.00 KB

iozone test complete.
Iozone: Performance Test of File I/O
Version $Revision: 3.283 $
Compiled for 64 bit mode.
Build: linux-AMD64

Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
Erik Habbinga, Kris Strecker, Walter Wong.

Run began: Wed Jan 23 15:15:00 2008

Record Size 1024 KB
File size set to 1024000 KB
Include close in write timing
Include fsync in write timing
O_DIRECT feature enabled
Command line used: ./iozone -r 1m -s 1000m -c -e -I -t 3 -F /scratch/X1 /scratch/X2 /scratch/X3 -i 0
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 3 processes
Each process writes a 1024000 Kbyte file in 1024 Kbyte records

Children see throughput for 3 initial writers = 85558.32 KB/sec
Parent sees throughput for 3 initial writers = 85553.00 KB/sec
Min throughput per process = 28509.68 KB/sec
Max throughput per process = 28538.28 KB/sec
Avg throughput per process = 28519.44 KB/sec
Min xfer = 1022976.00 KB

Children see throughput for 3 rewriters = 85329.49 KB/sec
Parent sees throughput for 3 rewriters = 85320.34 KB/sec
Min throughput per process = 28441.95 KB/sec
Max throughput per process = 28444.73 KB/sec
Avg throughput per process = 28443.16 KB/sec
Min xfer = 1024000.00 KB

iozone test complete.
Iozone: Performance Test of File I/O
Version $Revision: 3.283 $
Compiled for 64 bit mode.
Build: linux-AMD64

Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
Erik Habbinga, Kris Strecker, Walter Wong.

Run began: Wed Jan 23 14:12:42 2008

Record Size 1024 KB
File size set to 3072000 KB
Include close in write timing
Include fsync in write timing
Command line used: ./iozone -r 1m -s 3000m -c -e -t 1 -F /scratch/X1 -i 0
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 1 process
Each process writes a 3072000 Kbyte file in 1024 Kbyte records

Children see throughput for 1 initial writers = 90735.91 KB/sec
Parent sees throughput for 1 initial writers = 90727.50 KB/sec
Min throughput per process = 90735.91 KB/sec
Max throughput per process = 90735.91 KB/sec
Avg throughput per process = 90735.91 KB/sec
Min xfer = 3072000.00 KB

Children see throughput for 1 rewriters = 86449.82 KB/sec
Parent sees throughput for 1 rewriters = 86444.85 KB/sec
Min throughput per process = 86449.82 KB/sec
Max throughput per process = 86449.82 KB/sec
Avg throughput per process = 86449.82 KB/sec
Min xfer = 3072000.00 KB

iozone test complete.
Iozone: Performance Test of File I/O
Version $Revision: 3.283 $
Compiled for 64 bit mode.
Build: linux-AMD64

Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
Erik Habbinga, Kris Strecker, Walter Wong.

Run began: Wed Jan 23 14:36:53 2008

Record Size 1024 KB
File size set to 1536000 KB
Include close in write timing
Include fsync in write timing
Command line used: ./iozone -r 1m -s 1500m -c -e -t 2 -F /scratch/X1 /scratch/X2 -i 0
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 2 processes
Each process writes a 1536000 Kbyte file in 1024 Kbyte records

Children see throughput for 2 initial writers = 87352.61 KB/sec
Parent sees throughput for 2 initial writers = 84191.30 KB/sec
Min throughput per process = 42023.50 KB/sec
Max throughput per process = 45329.11 KB/sec
Avg throughput per process = 43676.30 KB/sec
Min xfer = 1497088.00 KB

Children see throughput for 2 rewriters = 83801.86 KB/sec
Parent sees throughput for 2 rewriters = 83674.05 KB/sec
Min throughput per process = 41841.26 KB/sec
Max throughput per process = 41960.60 KB/sec
Avg throughput per process = 41900.93 KB/sec
Min xfer = 1536000.00 KB

iozone test complete.
Iozone: Performance Test of File I/O
Version $Revision: 3.283 $
Compiled for 64 bit mode.
Build: linux-AMD64

Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
Erik Habbinga, Kris Strecker, Walter Wong.

Run began: Wed Jan 23 14:38:15 2008

Record Size 1024 KB
File size set to 1024000 KB
Include close in write timing
Include fsync in write timing
Command line used: ./iozone -r 1m -s 1000m -c -e -t 3 -F /scratch/X1 /scratch/X2 /scratch/X3 -i 0
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 3 processes
Each process writes a 1024000 Kbyte file in 1024 Kbyte records

Children see throughput for 3 initial writers = 85182.48 KB/sec
Parent sees throughput for 3 initial writers = 73354.87 KB/sec
Min throughput per process = 26752.83 KB/sec
Max throughput per process = 30931.31 KB/sec
Avg throughput per process = 28394.16 KB/sec
Min xfer = 802816.00 KB

Children see throughput for 3 rewriters = 86484.59 KB/sec
Parent sees throughput for 3 rewriters = 81781.53 KB/sec
Min throughput per process = 27262.20 KB/sec
Max throughput per process = 31317.65 KB/sec
Avg throughput per process = 28828.20 KB/sec
Min xfer = 1024000.00 KB

iozone test complete.
Iozone: Performance Test of File I/O
Version $Revision: 3.283 $
Compiled for 64 bit mode.
Build: linux-AMD64

Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
Erik Habbinga, Kris Strecker, Walter Wong.

Run began: Wed Jan 23 14:40:04 2008

Record Size 1024 KB
File size set to 3072000 KB
Include close in write timing
Include fsync in write timing
O_DIRECT feature enabled
Command line used: ./iozone -r 1m -s 3000m -c -e -I -t 1 -F /scratch/X1 -i 0
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 1 process
Each process writes a 3072000 Kbyte file in 1024 Kbyte records

Children see throughput for 1 initial writers = 86219.96 KB/sec
Parent sees throughput for 1 initial writers = 86214.89 KB/sec
Min throughput per process = 86219.96 KB/sec
Max throughput per process = 86219.96 KB/sec
Avg throughput per process = 86219.96 KB/sec
Min xfer = 3072000.00 KB

Children see throughput for 1 rewriters = 85718.92 KB/sec
Parent sees throughput for 1 rewriters = 85714.79 KB/sec
Min throughput per process = 85718.92 KB/sec
Max throughput per process = 85718.92 KB/sec
Avg throughput per process = 85718.92 KB/sec
Min xfer = 3072000.00 KB

iozone test complete.
Iozone: Performance Test of File I/O
Version $Revision: 3.283 $
Compiled for 64 bit mode.
Build: linux-AMD64

Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
Erik Habbinga, Kris Strecker, Walter Wong.

Run began: Wed Jan 23 14:42:42 2008

Record Size 1024 KB
File size set to 1536000 KB
Include close in write timing
Include fsync in write timing
O_DIRECT feature enabled
Command line used: ./iozone -r 1m -s 1500m -c -e -I -t 2 -F /scratch/X1 /scratch/X2 -i 0
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 2 processes
Each process writes a 1536000 Kbyte file in 1024 Kbyte records

Children see throughput for 2 initial writers = 85696.29 KB/sec
Parent sees throughput for 2 initial writers = 85689.20 KB/sec
Min throughput per process = 42834.30 KB/sec
Max throughput per process = 42861.98 KB/sec
Avg throughput per process = 42848.14 KB/sec
Min xfer = 1534976.00 KB

Children see throughput for 2 rewriters = 85508.10 KB/sec
Parent sees throughput for 2 rewriters = 85495.34 KB/sec
Min throughput per process = 42738.68 KB/sec
Max throughput per process = 42769.42 KB/sec
Avg throughput per process = 42754.05 KB/sec
Min xfer = 1534976.00 KB

iozone test complete.
Iozone: Performance Test of File I/O
Version $Revision: 3.283 $
Compiled for 64 bit mode.
Build: linux-AMD64

Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
Erik Habbinga, Kris Strecker, Walter Wong.

Run began: Wed Jan 23 14:44:15 2008

Record Size 1024 KB
File size set to 1024000 KB
Include close in write timing
Include fsync in write timing
O_DIRECT feature enabled
Command line used: ./iozone -r 1m -s 1000m -c -e -I -t 3 -F /scratch/X1 /scratch/X2 /scratch/X3 -i 0
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
Throughput test with 3 processes
Each process writes a 1024000 Kbyte file in 1024 Kbyte records

Children see throughput for 3 initial writers = 85836.74 KB/sec
Parent sees throughput for 3 initial writers = 85828.38 KB/sec
Min throughput per process = 28602.75 KB/sec
Max throughput per process = 28630.92 KB/sec
Avg throughput per process = 28612.25 KB/sec
Min xfer = 1022976.00 KB

Children see throughput for 3 rewriters = 85296.01 KB/sec
Parent sees throughput for 3 rewriters = 85285.05 KB/sec
Min throughput per process = 28431.54 KB/sec
Max throughput per process = 28432.71 KB/sec
Avg throughput per process = 28432.00 KB/sec
Min xfer = 1024000.00 KB

iozone test complete.

Attachment: config-2.6.24-rc8
Description: Binary data

Next message: Alasdair G Kergon: "Re: [dm-devel] [PATCH 0/2] dm-band: The I/O bandwidth controller: Overview"
Previous message: Robert Hancock: "Re: sata_nv and 2.6.24 (was Re: fixed a bug of adma in rhel4u5 withHDS7250SASUN500G.)"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]