2009/7/3 Vladislav Bolkhovitin <vst@xxxxxxxx>:Ronald Moesbergen, on 07/03/2009 01:14 PM wrote:Ok, done. Performance is pretty bad overall :(OK, now I tend to agree on decreasing max_sectors_kb and increasingBecause, as we found out, without your http://lkml.org/lkml/2009/5/21/319
read_ahead_kb. But before actually trying to push that idea I'd like
to
- do more benchmarks
- figure out why context readahead didn't help SCST performance
(previous traces show that context readahead is submitting perfect
large io requests, so I wonder if it's some io scheduler bug)
patch read-ahead was nearly disabled, hence there were no difference
which
algorithm was used?
Ronald, can you run the following tests, please? This time with 2 hosts,
initiator (client) and target (server) connected using 1 Gbps iSCSI. It
would be the best if on the client vanilla 2.6.29 will be ran, but any
other
kernel will be fine as well, only specify which. Blockdev-perftest should
be
ran as before in buffered mode, i.e. with "-a" switch.
1. All defaults on the client, on the server vanilla 2.6.29 with
Fengguang's
http://lkml.org/lkml/2009/5/21/319 patch with all default settings.
2. All defaults on the client, on the server vanilla 2.6.29 with
Fengguang's
http://lkml.org/lkml/2009/5/21/319 patch with default RA size and 64KB
max_sectors_kb.
3. All defaults on the client, on the server vanilla 2.6.29 with
Fengguang's
http://lkml.org/lkml/2009/5/21/319 patch with 2MB RA size and default
max_sectors_kb.
4. All defaults on the client, on the server vanilla 2.6.29 with
Fengguang's
http://lkml.org/lkml/2009/5/21/319 patch with 2MB RA size and 64KB
max_sectors_kb.
5. All defaults on the client, on the server vanilla 2.6.29 with
Fengguang's
http://lkml.org/lkml/2009/5/21/319 patch and with context RA patch. RA
size
and max_sectors_kb are default. For your convenience I committed the
backported context RA patches into the SCST SVN repository.
6. All defaults on the client, on the server vanilla 2.6.29 with
Fengguang's
http://lkml.org/lkml/2009/5/21/319 and context RA patches with default RA
size and 64KB max_sectors_kb.
7. All defaults on the client, on the server vanilla 2.6.29 with
Fengguang's
http://lkml.org/lkml/2009/5/21/319 and context RA patches with 2MB RA
size
and default max_sectors_kb.
8. All defaults on the client, on the server vanilla 2.6.29 with
Fengguang's
http://lkml.org/lkml/2009/5/21/319 and context RA patches with 2MB RA
size
and 64KB max_sectors_kb.
9. On the client default RA size and 64KB max_sectors_kb. On the server
vanilla 2.6.29 with Fengguang's http://lkml.org/lkml/2009/5/21/319 and
context RA patches with 2MB RA size and 64KB max_sectors_kb.
10. On the client 2MB RA size and default max_sectors_kb. On the server
vanilla 2.6.29 with Fengguang's http://lkml.org/lkml/2009/5/21/319 and
context RA patches with 2MB RA size and 64KB max_sectors_kb.
11. On the client 2MB RA size and 64KB max_sectors_kb. On the server
vanilla
2.6.29 with Fengguang's http://lkml.org/lkml/2009/5/21/319 and context RA
patches with 2MB RA size and 64KB max_sectors_kb.
The kernels I used:
client kernel: 2.6.26-15lenny3 (debian)
server kernel: 2.6.29.5 with blk_dev_run patch
And I adjusted the blockdev-perftest script to drop caches on both the
server (via ssh) and the client.
The results:
... results ...
Those are on the server without io_context-2.6.29 and readahead-2.6.29
patches applied and with CFQ scheduler, correct?
No. It was done with the readahead patch
(http://lkml.org/lkml/2009/5/21/319) and the context RA patch
(starting at test 5) as you requested.
Then we see how reorder of requests caused by many I/O threads submitting
I/O in separate I/O contexts badly affect performance and no RA, especially
with default 128KB RA size, can solve it. Less max_sectors_kb on the client
=> more requests it sends at once => more reorder on the server => worse
throughput. Although, Fengguang, in theory, context RA with 2MB RA size
should considerably help it, no?
Wouldn't setting scst_threads to 1 help also in this case?
Ronald, can you perform those tests again with both io_context-2.6.29 and
readahead-2.6.29 patches applied on the server, please?
Ok. I only have access to the test systems during the week, so results
might not be ready before Monday. Are there tests that we can exclude
to speed things up?