Re: [RFC PATCH v2] blk-mq: Introduce per sw queue time-slice

From: Andreas Herrmann
Date: Mon Feb 01 2016 - 17:47:05 EST

Following benchmark results for the patch using fio.

- kernel version 4.5.0-rc2-0001 (ie. with time-slice patch)
- Intel Core i7-3770S CPU @ 3.10GHz system, 4 cores, 8 threads/CPUs
- fio version as of 2.2.9-37-g0e1c4
- results were gathered iterating using rw and numjobs parameter, e.g.:

fio --directory=$1 --rw=$j --name=fio-scaling --size=5G --group_reporting \
--ioengine=libaio --direct=1 --iodepth=1 --runtime=$2 --numjobs=$i

- rotational device was a 1500GB Samsung HD155UI

ata3.00: ATA-8: SAMSUNG HD155UI, 1AQ10001, max UDMA/133
scsi 2:0:0:0: Direct-Access ATA SAMSUNG HD155UI 0001 PQ: 0 ANSI: 5

- no regression between blk-mq using vanilla v4.5-rc2 kernel and new
blk-mq w/ time_slice_us=0
- I omitted random read/write/rw tests as there was no significant
difference
- time_slice_us=250 seems to be optimal choice
(I compared 1000 Âs, 500 Âs, 250 Âs and 50 Âs)

(1) cfq, slice_idle=8 (default)
(2) cfq, slice_idle=0
(3) blk-mq, time_slice_us=0 (behaviour matches unmodified mainline)
(4) blk-mq, time_slice_us=250 (250 Âs) (time slice incremented)

---------------------------------------------------------------------
n cfq cfq blk-mq blk-mq
u (1) (2) (3) (4)
m
j 0 250
o Âs Âs
b
s
---------------------------------------------------------------------
read iops [mean (stdev)]
---------------------------------------------------------------------
1 17394.6 (470.48) 17336.4 (382.88) 20324.5 ( 72.75) 18914.5 (324.23)
2 15862.1 (416.38) 8427.8 (104.45) 8373.2 ( 71.97) 13216.6 (964.83)
3 16597.1 (246.77) 7078.0 (107.29) 8671.3 (238.12) 12370.2 (276.76)
4 15078.6 (132.34) 8589.8 (169.00) 9107.5 (328.00) 11582.0 (225.09)
5 14392.2 ( 75.55) 5908.5 (120.76) 6052.0 (102.01) 10906.3 (435.44)
6 13540.2 (142.83) 5902.9 (122.41) 5944.5 ( 90.23) 9758.9 (374.65)
7 13112.4 (175.02) 5938.7 ( 99.70) 5982.2 (104.44) 9233.0 (616.16)
8 12651.0 (187.11) 5955.9 (181.99) 6010.2 (105.62) 8908.5 (399.25)
---------------------------------------------------------------------
write iops [mean (stdev)]
---------------------------------------------------------------------
1 15233.6 (236.28) 15062.4 (176.56) 17141.0 (107.48 16236.8 (128.94)
2 15333.5 (474.39) 9860.1 ( 58.42) 10020.0 ( 38.67 15027.4 (288.07)
3 15753.6 (459.48) 9552.5 (153.48) 9523.0 (111.82 14021.5 (551.12)
4 15932.2 (328.94) 9949.8 ( 97.03) 9946.0 (110.38 13166.0 (289.70)
5 15840.5 (407.32) 9886.9 ( 43.84) 9814.0 ( 60.92 12401.2 (706.76)
6 16071.2 (349.90) 9822.1 ( 88.81) 9796.0 ( 83.29 11916.1 (904.20)
7 15864.1 (353.14) 9684.6 ( 63.48) 9628.0 ( 35.54 12194.0 (292.25)
8 15901.2 (308.38) 9474.3 ( 86.85) 9447.0 ( 40.46 11933.3 (633.21)
---------------------------------------------------------------------
rw (read) iops [mean (stdev)]
---------------------------------------------------------------------
1 4034.5 ( 76.29) 4238.7 ( 65.71) 4305.9 (107.76) 4069.8 (115.45)
2 3727.7 (108.46) 3860.5 (274.02) 4194.4 (132.19) 3909.3 (202.47)
3 3837.6 ( 57.84) 3533.6 (206.67) 3560.5 (145.49) 3625.9 (170.29)
4 3650.8 ( 53.90) 3201.1 ( 72.26) 3123.0 (154.18) 3506.5 (142.67)
5 3638.8 (102.67) 3043.1 (122.89) 3210.6 ( 89.05) 3301.1 (194.63)
6 3592.9 ( 93.41) 3002.3 (114.94) 3180.3 ( 42.36) 3297.5 (200.37)
7 3666.1 ( 66.77) 3081.3 ( 92.76) 3120.7 (127.99) 3210.4 (139.00)
8 3787.5 ( 46.90) 2859.6 (220.61) 2978.1 (119.55) 3215.4 (166.90)
---------------------------------------------------------------------
rw (write) iops [mean (stdev)]
---------------------------------------------------------------------
1 4037.4 ( 74.32) 4241.1 ( 66.48) 4306.4 (105.00) 4073.3 (115.22)
2 3740.1 (102.04) 3868.2 (265.28) 4193.9 (128.23) 3914.6 (195.13)
3 3832.9 ( 58.52) 3526.3 (209.27) 3550.3 (148.23) 3622.4 (169.52)
4 3647.1 ( 53.76) 3211.4 ( 68.84) 3133.7 (152.57) 3504.2 (138.19)
5 3642.5 (102.65) 3051.0 (114.87) 3206.6 ( 84.58) 3303.5 (193.98)
6 3610.2 ( 93.48) 3013.7 (124.22) 3208.8 ( 45.25) 3291.2 (197.91)
7 3681.6 ( 66.38) 3077.4 ( 91.72) 3118.5 (130.27) 3184.9 (140.03)
8 3792.6 ( 49.89) 2816.7 (235.14) 2943.7 (133.80) 3176.4 (164.24)
---------------------------------------------------------------------