2.6.0-test8/test9 io scheduler needs tuning?

From: Michael Frank
Date: Mon Oct 27 2003 - 17:15:39 EST


To my surprise 2.6 - which used to do better then 2.4 - does no longer
handle these test that well.

Generally, IDE IO throughput is _very_ uneven and IO _stops_ at times with the
system cpu load very high (and the disk LED off).

IMHO the CPU scheduling is OK but the IO scheduling acts up here.

The test system is a 2.4GHz P4 with 512M RAM and a 55MB/s udma IDE harddisk.

The tests load the system to loadavg > 30. IO should be about 20MB/s on avg.

Enclosed are vmstat -1 logs for 2.6-test9-Vanilla, followed by 2.6-test8-Vanilla
(-mm1 behaves similar), 2.4.22-Vanilla and 2.4.21+swsusp all compiled wo preempt.

IO on 2.6 stops now for seconds at a time. -test8 is worse than -test9

2.6-test9-Vanilla:

procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
11 10 2 0 17164 60408 96292 0 0 0 5756 1147 30803 21 79 0
13 7 1 0 22968 60600 83876 0 0 0 30100 1079 6475 19 81 0
16 6 4 0 4728 59900 106028 0 0 4 29556 1122 71394 23 77 0
12 7 3 0 27128 60056 70588 0 0 68 6344 1212 111635 19 81 0
13 8 2 0 45176 60200 51656 0 0 0 22868 1238 20471 13 87 0
12 9 2 0 18544 60324 76668 0 0 0 14720 1267 36875 22 78 0
15 5 1 0 4784 60352 95196 0 0 184 37448 1191 144856 32 68 0
3 15 1 0 14768 59980 91368 0 0 180 12376 1112 133689 41 59 0
4 15 1 0 40816 60112 59320 0 0 164 188 1092 75854 31 69 0
6 14 1 0 4976 60164 102640 0 0 16 56904 1122 85364 27 73 0
11 8 1 0 25648 60188 79316 0 0 4 3784 1100 75808 26 74 0
9 8 3 0 8896 60280 103316 0 0 0 24772 1154 4031 23 77 0
5 14 2 0 23744 60408 75220 0 0 232 108 1157 86855 38 62 0
10 8 1 0 12032 60028 91796 0 0 8 27048 1272 79355 24 76 0
10 10 2 0 21120 60292 80144 0 0 0 49700 1092 3015 10 90 0
13 8 2 0 34064 60488 66528 0 0 0 172 1061 3871 10 90 0
14 7 2 0 40200 60540 79824 0 0 8 36864 1121 3138 12 88 0
20 2 2 0 42304 60544 78452 0 0 4 504 1103 281659 23 77 0
12 8 1 0 59024 60656 61688 0 0 28 26080 1156 158476 22 78 0
14 4 1 0 59536 60924 54128 0 0 0 9132 1251 3056 9 91 0
15 2 3 0 46528 61108 65628 0 0 0 23092 1360 58596 11 89 0
8 10 4 0 4288 61244 103836 0 0 56 25904 1135 33984 21 79 0
14 3 2 0 48640 61340 67512 0 0 56 18156 1252 53401 24 76 0
13 1 2 0 30336 61572 80856 0 0 4 15192 1365 1880 20 80 0
10 7 1 0 13376 61752 93496 0 0 176 2320 1225 21294 30 70 0
12 3 2 0 22272 62012 81292 0 0 112 17808 1238 38432 19 81 0
10 3 2 0 12736 62288 91356 0 0 192 21840 1121 2531 21 79 0
16 2 3 0 26752 62228 67660 0 0 56 32500 1745 2100 21 79 0
15 8 2 0 4208 62292 74004 0 0 20 108 1152 4642 37 63 0
17 7 4 0 25200 62444 68544 0 0 60 19184 1384 2337 20 80 0
22 4 4 0 45224 62540 59164 0 0 64 24852 1309 451 14 86 0
22 4 4 0 27240 62684 77048 0 0 0 7480 1280 323 15 85 0
24 3 3 0 26528 62960 66448 0 0 0 24788 1240 837 9 91 0
23 3 2 0 26656 63208 62952 0 0 4 204 1068 616 8 92 0
24 3 2 0 20704 63500 69628 0 0 40 0 1029 440 7 93 0
24 3 2 0 13400 63796 76536 0 0 4 0 1020 426 8 92 0
25 3 2 0 13400 64032 76340 0 0 0 0 1019 371 10 90 0
26 3 2 0 23688 64048 81792 0 0 124 31828 1098 2308 22 78 0
18 5 2 0 30664 64356 69520 0 0 304 2828 1099 4965 32 68 0
22 0 2 0 26120 64424 68936 0 0 40 31328 1076 1737 29 71 0
19 2 3 0 21960 64580 71500 0 0 36 7136 1167 1952 23 77 0
21 2 2 0 15496 64772 87160 0 0 136 30928 1143 2391 19 81 0
23 1 3 0 21720 64904 96284 0 0 304 19200 1118 3474 19 81 0
15 5 3 0 18008 65232 91716 0 0 164 17032 1153 1545 19 81 0
19 0 3 0 18376 64508 80988 0 0 68 0 1145 2052 11 89 0
19 2 3 0 6088 64696 95896 0 0 140 21112 1241 3761 11 89 0
15 7 2 0 16408 64184 89024 0 0 600 28172 1141 1273 25 75 0
15 10 0 0 54024 64420 50096 0 0 1544 0 1182 5076 35 65 0
24 1 2 0 8520 64660 92148 0 0 268 32496 1136 695 14 86 0
11 9 1 0 47560 64908 54868 0 0 436 864 1199 7414 49 51 0
18 0 2 0 14424 65260 86804 0 0 160 46684 1179 2291 16 84 0
15 8 4 0 16728 65552 86248 0 0 288 7612 1159 6312 27 73 0
21 1 4 0 14488 65868 93388 0 0 296 4 1188 11041 20 80 0
18 7 4 0 41752 66116 52124 0 0 196 880 1115 80040 16 84 0

2.6-test8-Vanilla:

This one is terrible. The disk led goes off for many seconds.

procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
29 1 1 0 313908 9272 89024 0 0 0 0 2572 17213 82 18 0
29 2 1 0 313924 9272 89024 0 0 0 0 2216 90773 64 36 0
31 2 1 0 313660 9272 89024 0 0 0 0 1203 286591 29 71 0
30 2 1 0 313276 9280 89024 0 0 0 44 1294 283827 33 67 0
27 3 1 0 313756 9280 89024 0 0 0 0 2311 64008 78 22 0
29 3 1 0 314316 9280 89028 0 0 0 0 2513 2516 82 18 0
23 2 2 0 317620 9304 84124 0 0 28 6348 1865 86787 55 45 0
19 0 0 0 318148 9448 81264 0 0 140 540 2235 2050 78 22 0
14 3 2 0 317884 9524 81288 0 0 100 36 2460 2556 77 23 0
19 0 1 0 317476 9564 81300 0 0 32 160 2310 2426 76 24 0
24 0 1 0 307748 9584 90536 0 0 4 0 2314 2270 71 29 0
26 0 1 0 314036 9596 84132 0 0 0 0 2208 2606 68 32 0
24 0 2 0 304356 9616 93268 0 0 36 0 2486 2650 80 20 0
18 0 2 0 284724 9676 111564 0 0 24 12048 2296 2230 69 31 0
18 3 2 0 277100 9724 118888 0 0 0 21120 2690 2464 74 26 0
22 1 4 0 295068 9756 100984 0 0 0 9828 2584 2549 76 24 0
20 0 4 0 288220 9796 107940 0 0 4 288 2104 1927 60 40 0
22 0 4 0 287236 9852 108188 0 0 8 0 2293 2094 70 30 0
20 0 5 0 291212 9884 104208 0 0 12 0 2317 2212 79 21 0
22 0 5 0 295756 9904 99764 0 0 0 0 2395 2178 79 21 0
21 0 5 0 294740 9932 100796 0 0 4 0 2398 2441 79 21 0
23 0 5 0 287444 9952 107812 0 0 0 10620 2456 1828 83 17 0
21 0 5 0 296580 9956 98816 0 0 0 0 2413 1852 81 19 0
24 2 5 0 296564 9956 98816 0 0 0 0 2107 1114 83 17 0
26 0 5 0 296596 9956 98816 0 0 0 0 1920 1232 73 27 0
29 0 5 0 296380 9956 98820 0 0 4 0 2192 1731 77 23 0
28 0 5 0 296324 9968 98820 0 0 0 5956 2561 2448 81 19 0
32 0 5 0 296300 9968 98808 0 0 0 92 2019 1247 79 21 0
32 0 5 0 296308 9968 98808 0 0 0 0 2053 1444 83 17 0
32 0 5 0 296316 9968 98808 0 0 0 0 2070 1390 82 18 0
31 0 5 0 296324 9968 98808 0 0 0 0 2267 1900 81 19 0
32 0 5 0 296316 9976 98808 0 0 0 112 2292 1971 85 15 0
31 0 5 0 296324 9976 98808 0 0 0 0 2127 1392 84 16 0
32 0 5 0 296308 9976 98808 0 0 0 0 2284 1902 84 16 0
31 0 5 0 296308 9976 98808 0 0 0 0 2112 1617 81 19 0
31 0 5 0 296308 9976 98808 0 0 0 0 2160 1645 81 19 0
32 0 5 0 296236 9976 98808 0 0 0 0 2698 2965 81 19 0
32 0 5 0 296380 9976 98808 0 0 0 0 2151 1548 82 18 0
31 0 5 0 296364 9976 98808 0 0 0 0 2068 1309 83 17 0
32 0 5 0 296364 9976 98808 0 0 0 0 2337 2073 79 21 0
38 0 5 0 296212 9976 98808 0 0 0 0 2294 2077 81 19 0
36 0 5 0 296220 9976 98808 0 0 0 0 2294 1933 84 16 0
35 0 5 0 296236 9976 98808 0 0 0 0 2577 2775 84 16 0
35 0 5 0 296140 9976 98808 0 0 0 0 2565 2671 77 23 0
37 0 5 0 296236 9984 98808 0 0 0 68 1844 908 85 15 0
38 0 5 0 296236 9984 98808 0 0 0 0 1892 943 84 16 0
29 5 5 0 293348 10004 98884 0 0 100 220 1950 1235 81 19 0
25 10 4 0 267476 10080 123724 0 0 464 24416 2020 1288 72 28 0
26 10 4 0 267996 10084 124488 0 0 768 0 2382 1824 79 21 0
25 10 4 0 267684 10124 124948 0 0 500 0 2053 1389 83 17 0
25 10 4 0 266908 10168 125012 0 0 108 0 2179 1654 85 15 0
23 10 4 0 266468 10236 125148 0 0 204 0 2182 1673 82 18 0
27 10 4 0 266428 10296 125248 0 0 160 0 2496 2356 85 15 0
25 11 5 0 266340 10308 125248 0 0 12 0 2781 3085 84 16 0
22 13 5 0 266348 10316 125252 0 0 12 0 2828 3210 86 14 0
21 13 5 0 266364 10320 125256 0 0 8 0 2777 3052 86 14 0
22 13 5 0 266300 10324 125268 0 0 16 0 2769 3124 85 15 0
23 13 5 0 266148 10324 125268 0 0 0 0 2789 3229 83 17 0
33 4 2 0 266692 10380 124560 0 0 16 4744 2560 2557 77 23 0
31 5 3 0 276908 10400 114340 0 0 12 56 2676 2859 82 18 0
33 5 3 0 268884 10408 122176 0 0 0 0 2855 3324 83 17 0
34 5 3 0 265684 10412 125284 0 0 0 0 2947 3480 84 16 0

If you compare this with 2.4.22, it's io load is comparatively even
and it never stops:

procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
15 9 1 0 198856 19864 185180 0 0 244 18060 659 8954 34 66 0
11 13 1 0 178604 19896 179860 0 0 52 19120 692 87760 37 63 0
9 15 1 0 192816 19940 159160 0 0 52 20720 685 94859 31 69 0
8 15 1 0 201836 19948 150144 0 0 0 17604 650 113990 34 66 0
10 15 1 0 198572 20004 149444 0 0 72 21172 728 100160 30 70 0
12 11 1 0 129312 20156 176044 0 0 140 8940 584 54396 30 70 0
16 8 1 0 84604 20288 185412 0 0 32 25852 746 48955 33 67 0
18 8 1 0 86116 20384 183840 0 0 96 13404 662 51891 27 73 0
19 7 1 0 88336 20508 182492 0 0 48 14340 697 40311 19 81 0
17 9 1 0 82640 20608 183760 0 0 100 23088 735 62380 39 61 0
2 21 1 0 61484 20628 188908 0 0 44 18112 674 45239 36 64 0
2 21 1 0 37096 20684 196092 0 0 40 24252 729 572 25 75 0
2 21 1 0 47580 20684 182996 0 0 0 13464 653 363 43 57 0
3 21 1 0 59624 20712 171620 0 0 8 22988 646 448 39 61 0
2 21 1 0 50584 20744 178204 0 0 16 23008 805 657 31 69 0
2 21 1 0 59244 20744 169552 0 0 24 17672 670 408 44 56 0
14 2 1 0 36096 20836 175416 0 0 4 19148 730 1044 40 60 0
15 7 1 76 35792 19384 150752 0 0 52 14492 658 843 22 78 0
11 15 1 76 34736 19540 153128 0 0 60 17396 745 1346 23 77 0
2 21 1 76 27060 19548 159248 0 0 44 21008 703 543 36 64 0
2 21 1 76 38796 19548 147504 0 0 0 18528 712 495 43 57 0
2 23 1 76 39448 19592 148160 0 0 56 23328 725 625 33 67 0
22 8 1 76 30200 19616 157304 0 0 0 20876 680 499 29 71 0
12 3 1 76 4420 19712 151292 0 0 64 23968 675 2676 24 76 0
15 5 1 76 4632 19764 116028 0 0 60 13824 695 1015 27 73 0
24 3 1 244 4356 19432 128800 0 0 24 23752 636 1174 28 72 0
23 3 1 244 29684 19532 113968 0 0 40 26124 724 18608 25 75 0
21 2 1 244 17316 19592 124904 0 0 88 18596 660 51583 38 62 0
17 9 1 244 20728 19660 107408 0 0 52 22244 710 55023 27 73 0
21 5 1 244 24172 19748 115872 0 0 108 22824 680 44490 25 75 0
23 4 1 244 4460 19832 113824 0 0 240 15136 691 44779 25 75 0
14 12 1 244 5588 19884 131408 0 0 72 20208 719 69408 25 75 0
23 4 1 244 4404 19964 133028 0 0 128 18564 659 51254 24 76 0
22 5 1 244 9084 20068 129856 0 0 208 16824 631 22191 28 72 0
7 17 1 244 35000 20100 104024 0 0 28 18816 708 106858 38 62 0

Also 2.4.21, it's behavior is similar to 2.4.22.

procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
3 20 2 22480 5472 2032 82500 8 0 128 14716 3279 4137 71 26 3
8 19 1 22480 5068 2044 76988 56 0 96 15016 2704 104072 56 25 19
2 24 1 22480 4976 2044 77008 8 0 28 12836 3521 167313 52 48 0
3 24 1 22480 4788 2044 77036 72 0 100 16684 2886 155685 60 40 0
5 23 2 22480 12092 1876 83660 96 4 304 12556 2953 68285 39 61 0
4 27 2 22480 10796 1892 79136 56 0 144 8544 2373 139592 50 50 0
6 30 1 22480 20548 1912 72924 76 0 124 17428 2858 69541 39 61 0
6 26 1 22480 9284 1952 89440 36 0 152 17128 2334 65785 40 60 0
7 26 1 22480 9056 1952 89552 24 0 136 10440 2475 72782 42 58 0
6 26 2 22480 8428 1956 89732 128 0 312 22380 2704 79724 50 50 0
6 25 1 22340 29768 2016 67656 104 0 216 17048 2198 71301 42 58 0
6 31 1 21552 39852 2060 57028 124 0 288 16376 2845 39256 51 49 0
6 29 1 21552 39708 2064 57084 20 0 80 17832 2381 5321 50 50 0
7 21 1 21464 27672 2192 74152 184 0 356 6808 2006 5181 32 68 0
5 23 1 21400 27032 2196 62272 180 0 284 14552 2903 7905 40 60 0
6 23 1 21340 26540 2196 62312 204 0 244 15556 2279 6140 53 47 0
1 26 1 21284 25952 2212 62416 124 0 244 19804 3322 8600 54 46 0

Regards
Michael


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/