Re: [PATCH] cfq-iosched: reduce write depth only if sync was delayed

[Corrado Zoccolo]

Thanks Jeff.
Your tests show that with the patch, low_latency doesn't penalize
writeback any more on fast hardware, so the goal of the patch is
fulfilled. Lowering slice_async doesn't change the picture (I was
expecting an improvement in seq write bandwidth).
I'm still puzzled of why with 2.6.29 your SAN could achieve 300 MB/s
sequential writes, and it can't achieve it any more.
Can you try lowering slice_idle, trying e.g. 2ms?

Thanks
Corrado

On Fri, Dec 18, 2009 at 4:32 PM, Jeff Moyer <jmoyer@xxxxxxxxxx> wrote:
> Corrado Zoccolo <czoccolo@xxxxxxxxx> writes:
>
>> Hi Jeff,
>> On Wed, Dec 9, 2009 at 7:09 PM, Jeff Moyer <jmoyer@xxxxxxxxxx> wrote:
>>> Corrado Zoccolo <czoccolo@xxxxxxxxx> writes:
>>>
>>>> The numbers look good. Now, there is no penalty in having low_latency
>>>> set for sequential writes, and just a small penalty for random ones.
>>>> The fact that random reads are faster with low_latency set is interesting.
>>>> Is the test is running with your patched tiobench (so that the number
>>>> of random operations is comparable with sequential ones)?
>>>
>>> No, I forgot all about that. ÂThe number of random operations defaults
>>> to 4000, which is pretty low. ÂI'll re-run the tests with a number
>>> comparable to the sequential runs. ÂSorry about that.
>>>
>> N.P.
>> if you have time, can you also re-run the test changing:
>> iosched/fifo_expire_async to 8 ?
>> I hope that reducing the expire_async, will make cfq quicker at switching
>> between the different threads, allowing more parallelism for seq
>> writers on your hw.
>> If this is the case, I think I can try to estimate the
>> fifo_expire_async in the autotuning patch.
>
> Sorry this took so long. ÂI've been rather busy of late.
>
> Cheers,
> Jeff
>
> low_latency=1, fifo_expire_async=8
>
> Unit information
> ================
> File size = megabytes
> Blk Size Â= bytes
> Rate   Â= megabytes per second
> CPU% Â Â Â= percentage of CPU used during the test
> Latency  = milliseconds
> Lat% Â Â Â= percent of requests that took longer than X seconds
> CPU Eff  = Rate divided by CPU% - throughput per cpu load
>
> Sequential Reads
>               ÂFile ÂBlk  Num          Avg   ÂMaximum   ÂLat%   Lat%  ÂCPU
> Identifier          ÂSize ÂSize ÂThr  Rate Â(CPU%) ÂLatency  ÂLatency   Â>2s   Â>10s  ÂEff
> ---------------------------- ------ ----- --- Â------ ------ --------- ----------- Â-------- -------- -----
> 2.6.32 Â Â Â Â Â Â Â Â Â Â Â Â8192 Â65536 Â 8 Â 88.39 89.74% Â Â16.388 Â Â 2032.62 Â 0.00000 Â0.00000 Â Â98
> 2.6.32 Â Â Â Â Â Â Â Â Â Â Â Â8192 Â65536 Â16 Â 90.77 185.3% Â Â32.213 Â Â 2175.99 Â 0.00000 Â0.00000 Â Â49
>
> Random Reads
>               ÂFile ÂBlk  Num          Avg   ÂMaximum   ÂLat%   Lat%  ÂCPU
> Identifier          ÂSize ÂSize ÂThr  Rate Â(CPU%) ÂLatency  ÂLatency   Â>2s   Â>10s  ÂEff
> ---------------------------- ------ ----- --- Â------ ------ --------- ----------- Â-------- -------- -----
> 2.6.32 Â Â Â Â Â Â Â Â Â Â Â Â8192 Â65536 Â 8 Â 19.62 25.74% Â Â71.827 Â Â 3397.26 Â 0.00000 Â0.00000 Â Â76
> 2.6.32 Â Â Â Â Â Â Â Â Â Â Â Â8192 Â65536 Â16 Â 23.82 55.01% Â 103.361 Â Â 4075.53 Â 0.00000 Â0.00000 Â Â43
>
> Sequential Writes
>               ÂFile ÂBlk  Num          Avg   ÂMaximum   ÂLat%   Lat%  ÂCPU
> Identifier          ÂSize ÂSize ÂThr  Rate Â(CPU%) ÂLatency  ÂLatency   Â>2s   Â>10s  ÂEff
> ---------------------------- ------ ----- --- Â------ ------ --------- ----------- Â-------- -------- -----
> 2.6.32 Â Â Â Â Â Â Â Â Â Â Â Â8192 Â65536 Â 8 Â108.28 1007.% Â Â12.984 Â Â 5643.55 Â 0.00076 Â0.00000 Â Â11
> 2.6.32 Â Â Â Â Â Â Â Â Â Â Â Â8192 Â65536 Â16 Â112.40 2014.% Â Â25.430 Â Â 8592.98 Â 0.00839 Â0.00000 Â Â 6
>
> Random Writes
>               ÂFile ÂBlk  Num          Avg   ÂMaximum   ÂLat%   Lat%  ÂCPU
> Identifier          ÂSize ÂSize ÂThr  Rate Â(CPU%) ÂLatency  ÂLatency   Â>2s   Â>10s  ÂEff
> ---------------------------- ------ ----- --- Â------ ------ --------- ----------- Â-------- -------- -----
> 2.6.32 Â Â Â Â Â Â Â Â Â Â Â Â8192 Â65536 Â 8 Â 63.94 337.7% Â Â22.885 Â Â 6047.22 Â 0.00076 Â0.00000 Â Â19
> 2.6.32 Â Â Â Â Â Â Â Â Â Â Â Â8192 Â65536 Â16 Â 61.94 662.5% Â Â46.997 Â Â12759.69 Â 0.15411 Â0.00000 Â Â 9
>
>
> low_latency=0, fifo_expire_async=8
>
> Unit information
> ================
> File size = megabytes
> Blk Size Â= bytes
> Rate   Â= megabytes per second
> CPU% Â Â Â= percentage of CPU used during the test
> Latency  = milliseconds
> Lat% Â Â Â= percent of requests that took longer than X seconds
> CPU Eff  = Rate divided by CPU% - throughput per cpu load
>
> Sequential Reads
>               ÂFile ÂBlk  Num          Avg   ÂMaximum   ÂLat%   Lat%  ÂCPU
> Identifier          ÂSize ÂSize ÂThr  Rate Â(CPU%) ÂLatency  ÂLatency   Â>2s   Â>10s  ÂEff
> ---------------------------- ------ ----- --- Â------ ------ --------- ----------- Â-------- -------- -----
> 2.6.32 Â Â Â Â Â Â Â Â Â Â Â Â8192 Â65536 Â 8 Â 89.13 88.30% Â Â15.872 Â Â 3101.39 Â 0.00000 Â0.00000 Â 101
> 2.6.32 Â Â Â Â Â Â Â Â Â Â Â Â8192 Â65536 Â16 Â 86.78 161.7% Â Â30.794 Â Â 4909.02 Â 0.00000 Â0.00000 Â Â54
>
> Random Reads
>               ÂFile ÂBlk  Num          Avg   ÂMaximum   ÂLat%   Lat%  ÂCPU
> Identifier          ÂSize ÂSize ÂThr  Rate Â(CPU%) ÂLatency  ÂLatency   Â>2s   Â>10s  ÂEff
> ---------------------------- ------ ----- --- Â------ ------ --------- ----------- Â-------- -------- -----
> 2.6.32 Â Â Â Â Â Â Â Â Â Â Â Â8192 Â65536 Â 8 Â 20.21 26.64% Â Â69.863 Â Â 4285.42 Â 0.00000 Â0.00000 Â Â76
> 2.6.32 Â Â Â Â Â Â Â Â Â Â Â Â8192 Â65536 Â16 Â 20.10 52.75% Â 139.761 Â Â 5986.94 Â 0.00076 Â0.00000 Â Â38
>
> Sequential Writes
>               ÂFile ÂBlk  Num          Avg   ÂMaximum   ÂLat%   Lat%  ÂCPU
> Identifier          ÂSize ÂSize ÂThr  Rate Â(CPU%) ÂLatency  ÂLatency   Â>2s   Â>10s  ÂEff
> ---------------------------- ------ ----- --- Â------ ------ --------- ----------- Â-------- -------- -----
> 2.6.32 Â Â Â Â Â Â Â Â Â Â Â Â8192 Â65536 Â 8 Â108.74 1020.% Â Â13.070 Â Â 5331.78 Â 0.00076 Â0.00000 Â Â11
> 2.6.32 Â Â Â Â Â Â Â Â Â Â Â Â8192 Â65536 Â16 Â112.18 2020.% Â Â25.559 Â Â 7903.16 Â 0.00992 Â0.00000 Â Â 6
>
> Random Writes
>               ÂFile ÂBlk  Num          Avg   ÂMaximum   ÂLat%   Lat%  ÂCPU
> Identifier          ÂSize ÂSize ÂThr  Rate Â(CPU%) ÂLatency  ÂLatency   Â>2s   Â>10s  ÂEff
> ---------------------------- ------ ----- --- Â------ ------ --------- ----------- Â-------- -------- -----
> 2.6.32 Â Â Â Â Â Â Â Â Â Â Â Â8192 Â65536 Â 8 Â 64.53 337.8% Â Â22.671 Â Â 5388.77 Â 0.00000 Â0.00000 Â Â19
> 2.6.32 Â Â Â Â Â Â Â Â Â Â Â Â8192 Â65536 Â16 Â 61.75 668.9% Â Â47.265 Â Â13271.37 Â 0.12894 Â0.00000 Â Â 9
>



-- 
__________________________________________________________________________

dott. Corrado Zoccolo                          mailto:czoccolo@xxxxxxxxx
PhD - Department of Computer Science - University of Pisa, Italy
--------------------------------------------------------------------------
The self-confidence of a warrior is not the self-confidence of the average
man. The average man seeks certainty in the eyes of the onlooker and calls
that self-confidence. The warrior seeks impeccability in his own eyes and
calls that humbleness.
                               Tales of Power - C. Castaneda
èº{.nÇ+‰·Ÿ®‰­†+%ŠËlzwm…ébëæìr¸›zX§»®w¥Š{ayºÊÚë,j­¢f£¢·hš‹àz¹®w¥¢¸¢·¦j:+v‰¨ŠwèjØm¶Ÿÿ¾«‘êçzZ+ƒùšŽŠÝj"ú!¶iO•æ¬z·švØ^¶m§ÿðÃnÆàþY&—