Re: sluggish system responsiveness under higher IO load

From: Matthias Dahl
Date: Tue Aug 08 2006 - 11:55:12 EST

On Sunday 06 August 2006 21:37, Andrew Morton wrote:

> hm. Do you have any reason for thinking that it's specific to any
> particular device driver or hardware setup?

Even though this is subjective, I suspect the sata subsystem or something in
that area. My old P3 500 system (IDE based) performed better under heavy IO
than my current setup which is sata based. But like I said, this is highly
subjective and I cannot give you any reasonable facts.

> How much memory does the machine have?

1 gb of ram. Usually swap is rarely or not much touched except when I stress
the system with eclipse and other things that consume too much memory. :)

> It'd be interesting if you could gather a `vmstat 1' trace during the
> sluggish period.

No problem at all. It's just not as easy to trigger as one might think.
Sometimes every high io load causes sluggish performance, while on a
different occassion, things run almost fine even though the system is under
io pressure...

procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 1 0 203172 30616 495628 0 0 26 14 472 278 3 1 95 1
1 0 0 190092 30628 508736 0 0 13120 0 675 572 37 3 0 60
1 0 0 191712 30628 508728 0 0 0 0 389 97 97 3 0 0
1 0 0 191592 30628 508836 0 0 108 0 396 186 74 7 0 19
0 1 0 190772 30640 508872 0 0 44 388 464 127 84 7 0 9
0 1 0 184732 30736 514152 0 0 5360 172 587 379 2 4 0 94
1 0 0 179080 30788 519664 0 0 5564 0 514 387 13 7 0 80
0 1 0 177024 30836 520008 0 0 260 0 457 304 31 49 0 20
1 0 0 165084 31844 525608 0 0 1212 0 680 1825 42 10 0 48
1 1 0 153924 32784 533796 0 0 692 4 641 3119 61 6 0 33
0 2 0 151072 33124 535380 0 0 268 9640 793 661 10 7 0 83
0 2 0 149212 33288 536752 0 0 128 3828 710 565 13 5 0 82
2 1 0 148344 33408 537484 0 0 80 2408 683 998 23 7 0 70
1 1 0 144252 33944 540648 0 0 360 0 620 2245 39 8 0 53
2 1 0 131968 34560 550612 0 0 188 2672 665 2039 81 11 0 8
2 1 0 123908 35104 556448 0 0 156 5168 909 1765 45 14 0 41
0 2 0 115352 35520 563024 0 0 128 8492 1135 2256 51 14 0 35
1 0 0 107852 35876 568948 0 0 232 2428 751 972 41 9 0 50
3 0 0 90368 36812 582488 0 0 220 0 550 1248 77 16 0 7
0 2 0 74248 37228 596256 0 0 16 5568 584 3173 87 10 0 3
2 1 0 64576 37544 604380 0 0 44 14572 1242 1329 56 9 0 35
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 2 0 62096 37768 606216 0 0 148 4212 890 902 13 5 0 82
0 2 0 58624 37892 609120 0 0 72 7676 906 2011 44 10 0 45
1 1 0 57012 37984 610028 0 0 60 5768 893 1451 20 6 0 74
0 3 0 56704 38076 610352 0 0 100 7908 796 1092 18 8 0 75
1 2 0 55340 38200 611292 0 0 100 9664 806 1236 20 9 0 71
0 2 0 51992 38416 613904 0 0 136 4420 910 1320 27 7 0 66
1 2 0 49512 38600 615976 0 0 116 3132 896 1412 25 5 0 70
2 1 0 45544 38788 619324 0 0 84 1004 863 4514 68 13 0 19
3 0 0 30540 39392 629544 0 0 304 0 667 1737 73 5 0 22
0 2 0 15396 39768 637920 0 0 160 12044 1174 1975 63 10 0 27
0 4 0 9332 40048 643272 0 0 80 6804 1813 1470 52 6 0 42
2 3 0 10232 40236 641768 0 0 88 5372 1729 1392 45 6 0 49
1 1 0 10008 40464 641096 0 0 136 13176 1795 1364 43 6 0 51
2 1 0 10116 40700 640608 0 0 156 6248 1739 1597 46 5 0 50
2 0 0 9504 40852 640964 0 0 56 432 1581 2085 50 5 0 46
2 0 0 9364 41284 639996 0 0 176 0 1554 1471 86 6 0 8
2 0 0 9212 41748 638880 0 0 252 0 1596 1521 86 5 0 9
0 1 0 9148 42160 637656 0 0 228 0 1560 1477 82 6 0 12
1 2 0 9876 42792 635208 0 0 356 10160 1662 1574 66 7 0 27
0 3 0 9180 42996 635276 0 0 124 11028 1880 1377 36 3 0 61
0 4 0 9848 43148 634336 0 0 32 9900 1762 1345 42 5 0 53
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 4 0 9260 43312 634376 0 0 76 3316 1758 1427 48 6 0 47
0 4 0 9480 43528 633752 0 0 92 3336 1843 1434 45 7 0 48
0 2 0 9272 43724 633500 0 0 108 1584 1714 1364 35 5 0 60
2 3 0 16220 43856 634752 0 0 68 5400 959 1540 36 6 0 58
0 4 0 13988 43956 636588 0 0 44 3752 942 4695 43 20 0 36
0 5 0 12004 44140 637956 0 0 124 3740 915 2949 14 11 0 75
0 5 0 10020 44340 639328 0 0 152 3104 740 455 11 4 0 85
0 5 0 9044 44588 639500 0 0 172 3660 740 513 15 9 0 76
0 5 0 9984 44752 638000 0 0 128 1952 754 460 13 8 0 79
0 2 0 9588 44992 637556 0 0 172 1108 769 1255 15 11 0 74
1 4 0 9140 45164 637340 0 0 112 4392 718 1256 23 8 0 69
2 3 0 9520 45444 636112 0 0 248 1964 738 769 15 7 0 78
1 5 0 9544 45628 635748 0 0 136 1424 656 604 6 3 0 91
0 5 0 10096 45812 634900 0 0 168 2172 682 585 5 6 0 89
0 5 0 9500 45988 634940 0 0 128 1588 652 601 11 4 0 85
0 6 0 9924 46156 633736 0 0 108 3132 707 462 17 8 0 75
0 2 0 10156 46416 633064 0 0 208 4320 731 381 18 7 0 75
0 3 0 9568 46552 633400 0 0 96 1252 619 249 1 5 0 94
0 3 0 9568 46712 633680 0 0 116 1496 614 310 1 1 0 98
0 4 0 9284 46880 633188 0 0 108 2448 681 505 24 8 0 68
2 3 0 9696 47112 631712 0 0 136 1968 735 535 26 10 0 64
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 1 0 9528 47272 631492 0 0 84 1288 689 462 19 6 0 75
0 3 0 9488 47396 631216 0 0 76 7448 785 416 8 8 0 84
1 4 0 9328 47576 630700 0 0 124 3436 827 583 25 11 0 64
0 3 0 9692 47792 629884 0 0 148 2292 665 481 15 6 0 79
0 1 0 9812 48032 629272 0 0 180 1196 631 489 13 8 0 79
1 1 0 10376 48560 627012 0 0 344 0 517 581 56 6 0 38
2 3 0 10028 48748 627580 0 0 132 7832 875 518 16 6 0 78
2 5 0 10148 48908 627044 0 0 84 3828 935 1475 39 8 0 53
1 3 0 9728 49048 627040 0 0 84 2604 1024 3977 31 8 0 61
0 3 0 9372 49120 627136 0 0 60 1680 905 2990 34 7 0 59
0 3 0 9168 49272 626992 0 0 92 1428 734 2410 27 4 0 69
3 2 0 9124 49368 626952 0 0 52 5332 833 2286 17 3 0 80
1 3 0 9192 49596 626320 0 0 100 1784 992 3304 35 9 0 56
0 3 0 10324 49772 624704 0 0 60 3444 906 3163 45 6 0 48
0 3 0 9424 49984 624956 0 0 56 2020 857 1386 35 4 0 61
1 1 0 9836 50176 624192 0 0 48 2584 859 3261 33 7 0 60
0 3 0 10136 50288 623588 0 0 36 1860 632 410 14 3 0 83
0 2 0 17204 50328 621736 0 0 52 20828 5602 7789 8 7 0 86
1 0 0 17212 50328 621736 0 0 0 380 720 1035 14 17 0 69
0 2 0 17088 50328 621740 0 0 0 2764 667 830 17 20 0 63
2 0 0 10512 50356 624032 0 0 500 5696 890 1517 33 24 0 43
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 0 0 9868 49312 625308 0 0 0 0 443 642 94 6 0 0
2 0 0 13080 49004 625656 0 0 188 0 489 610 82 5 0 13
0 2 0 10228 49104 627156 0 0 4 29100 501 630 47 29 1 23
2 0 0 9484 49220 628400 0 0 0 3564 601 418 43 29 0 27
3 0 0 9848 49488 627496 0 0 128 2132 565 771 55 41 0 4
1 0 0 9372 49632 624564 0 0 288 524 642 1623 57 31 0 12
1 0 12 10024 49356 619120 0 0 396 0 912 2909 62 26 0 12
2 0 12 9764 49660 619056 0 0 0 0 830 2940 66 34 0 0
2 0 12 14004 50432 615020 0 0 84 468 634 1506 58 37 0 5
2 0 12 12504 51084 616064 0 0 24 0 552 718 50 50 0 0
0 2 12 10016 51696 616112 0 0 0 17752 687 1288 55 28 0 17
2 3 12 10536 51936 614960 0 0 16 12460 1155 1088 17 22 0 61
1 3 12 11048 52880 611944 0 0 0 21716 972 428 52 48 0 0
0 3 12 9596 53260 612508 0 0 0 15400 1003 682 40 57 0 3
0 2 12 10348 53284 612516 0 0 8 15628 847 709 5 5 0 90
0 2 12 11132 53832 613036 0 0 56 91444 6355 6600 9 11 0 80
2 0 12 41160 53932 585872 0 0 288 3516 782 1821 32 30 0 38
0 1 12 40688 53948 586020 0 0 0 3584 722 532 17 10 0 73
3 0 12 36956 53952 585952 0 0 0 4 1136 1878 38 28 0 33
0 1 12 35064 53968 585952 0 0 0 11160 1052 1081 15 15 0 70
4 0 12 32416 53968 585960 0 0 0 4336 2840 5072 19 8 0 73
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
3 1 12 31616 54272 586864 0 0 1168 18540 5110 4204 8 6 0 86
6 2 12 30560 54276 587020 0 0 128 784 761 1957 26 14 0 60
2 1 12 32908 54292 587452 0 0 448 916 903 1486 21 29 0 50
1 1 12 35660 54304 587452 0 0 12 1096 645 422 46 33 0 20
1 1 12 35252 54304 587452 0 0 0 1384 594 292 84 16 0 0
1 0 12 33888 54312 587452 0 0 0 696 707 2809 85 15 0 0
1 0 12 33764 54312 587452 0 0 0 0 420 208 97 3 0 0
1 0 12 33796 54312 587452 0 0 0 0 468 231 98 2 0 0
1 0 12 33672 54312 587452 0 0 0 0 414 193 97 3 0 0
1 0 12 34920 54320 587452 0 0 0 16 470 224 98 2 0 0
1 0 12 34920 54320 587452 0 0 0 648 521 236 92 8 0 0
2 0 12 34944 54320 587452 0 0 0 0 471 286 98 2 0 0
1 0 12 34944 54320 587452 0 0 0 0 403 190 87 13 0 0
1 0 12 35084 54320 587452 0 0 0 0 408 245 99 1 0 0
1 0 12 35332 54328 587452 0 0 0 16 354 240 93 7 0 0
1 0 12 35364 54328 587452 0 0 0 0 408 292 93 7 0 0
1 0 12 35364 54328 587452 0 0 0 0 353 161 93 7 0 0
1 0 12 35380 54328 587452 0 0 0 0 406 212 91 9 0 0
1 0 12 35380 54328 587452 0 0 0 0 352 170 96 4 0 0
1 0 12 35660 54332 587452 0 0 0 16 409 216 96 4 0 0
1 0 12 34420 54332 587452 0 0 0 0 352 161 96 4 0 0
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 0 12 34164 54332 587452 0 0 0 0 409 239 88 12 0 0
0 1 12 33916 54344 587588 0 0 12 0 358 292 75 20 0 5
2 0 12 33800 54432 587628 0 0 16 0 414 860 29 69 0 2
2 0 12 33552 54532 587656 0 0 12 1192 394 870 25 74 0 1
4 0 12 33452 54596 587684 0 0 4 0 501 852 20 80 0 0
2 0 12 33164 54676 587712 0 0 4 0 432 670 22 78 0 0
1 0 12 33008 54716 587740 0 0 0 0 450 989 25 75 0 0
2 0 12 32860 54784 587768 0 0 4 0 526 1212 26 74 0 0
1 0 12 32612 54868 587796 0 0 4 1692 471 980 23 77 0 0
2 0 12 32472 54940 587824 0 0 4 0 432 838 26 74 0 0
2 0 12 32364 55008 587860 0 0 12 0 455 858 28 71 0 1
1 0 12 32232 55060 587892 0 0 0 0 432 847 22 78 0 0
2 0 12 32004 55112 588012 0 0 92 0 468 881 22 74 0 4
3 0 12 31888 55156 588080 0 0 44 1680 446 853 27 71 0 2
1 0 12 31624 55240 588108 0 0 0 0 451 865 25 75 0 0
1 0 12 31456 55288 588184 0 0 48 0 443 869 26 70 0 4
1 0 12 31456 55368 588212 0 0 4 0 454 915 23 77 0 0
2 0 12 31340 55436 588244 0 0 0 0 446 1158 23 77 0 0
2 0 12 31456 55488 588276 0 0 0 1768 485 1098 23 68 0 9
0 1 12 31348 55580 588312 0 0 8 0 440 1000 24 75 0 1
1 0 12 31008 55644 588392 0 0 64 0 465 997 23 77 0 0
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
2 0 12 31024 55720 588452 0 0 32 0 445 983 26 72 0 2
2 0 12 30940 55748 588484 0 0 0 0 450 1131 23 77 0 0
1 0 12 30684 55824 588512 0 0 4 1624 461 1051 26 70 0 4
2 0 12 30552 55900 588544 0 0 4 0 454 941 25 74 0 1
2 0 12 30436 55968 588576 0 0 4 0 452 970 32 66 0 2
1 0 12 30320 56040 588608 0 0 4 0 486 997 34 66 0 0
2 0 12 30320 56092 588636 0 0 0 0 400 796 19 81 0 0
1 1 12 30200 56204 588668 0 0 0 4576 548 899 25 75 0 1
3 1 12 30084 56300 588684 0 0 0 1444 588 611 13 50 0 37
5 1 12 29892 56444 588712 0 0 0 2304 755 737 17 83 0 0
1 0 12 29720 56520 588740 0 0 0 564 563 648 22 78 0 0
2 0 12 29540 56600 588772 0 0 0 0 449 1009 28 72 0 0
1 0 12 29416 56700 588804 0 0 0 1968 557 1315 26 73 0 1
1 0 12 29292 56748 588836 0 0 0 0 532 1336 23 77 0 0
2 0 12 28060 56888 589236 0 0 232 0 515 1350 27 73 0 0
2 0 12 28008 57008 589144 0 0 32 0 521 1281 24 76 0 0
2 0 12 27760 57180 589180 0 0 0 0 530 1392 25 75 0 0
2 0 12 27380 57384 589208 0 0 0 1992 557 1549 29 65 0 6
1 0 12 27248 57484 589228 0 0 0 0 541 51688 45 55 0 0
2 0 12 26620 57568 589260 0 0 0 0 509 2092 31 69 0 0
2 0 12 26480 57680 589292 0 0 0 0 451 913 25 75 0 0
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 0 12 26240 57868 589328 0 0 0 0 471 1509 27 73 0 0
2 0 12 26000 57972 589364 0 0 0 2108 487 1409 24 75 0 1
3 0 12 25892 58052 589396 0 0 0 0 528 1346 26 74 0 0
2 0 12 22088 58120 589416 0 0 0 1292 663 10366 69 31 0 0
3 0 12 21864 58160 589460 0 0 0 0 774 5945 64 36 0 0
2 0 12 20896 58296 589488 0 0 0 0 563 2229 32 68 0 0
2 0 12 20764 58380 589520 0 0 4 0 554 1464 24 74 0 2
2 0 12 20632 58432 589540 0 0 0 0 528 1136 51 49 0 0
2 0 12 20640 58460 589556 0 0 0 1824 509 972 60 40 0 0
3 0 12 19708 58480 589568 0 0 0 0 520 1006 68 32 0 0
2 0 12 20252 58544 589588 0 0 0 0 637 2646 44 56 0 0
1 0 12 20308 58604 589616 0 0 0 4 620 4560 33 67 0 0
2 0 12 20076 58752 589648 0 0 0 0 553 1438 27 73 0 0
2 0 12 19828 58820 589672 0 0 0 1804 553 52245 43 56 0 1
2 0 12 19820 58892 589700 0 0 0 0 513 1193 24 76 0 0
3 0 12 19860 58908 589732 0 0 0 0 428 996 22 78 0 0
2 0 12 19720 58924 589760 0 0 0 0 476 1118 25 75 0 0
4 0 12 19628 58980 589796 0 0 0 4 414 1006 27 73 0 0
2 0 12 19488 59040 589828 0 0 0 1392 486 1324 26 73 0 1
1 0 12 19412 59092 589860 0 0 0 0 424 843 18 82 0 0
2 0 12 19396 59120 589892 0 0 0 0 548 1191 21 79 0 0
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
1 1 12 19272 59136 589920 0 0 0 3900 713 1290 27 73 0 0
2 1 12 19272 59144 589948 0 0 0 1880 753 1075 27 73 0 0
3 1 12 19296 59184 589984 0 0 0 3740 772 880 25 75 0 0
2 0 12 19156 59220 590016 0 0 0 1076 512 889 20 80 0 0
2 0 12 19164 59248 590044 0 0 0 0 353 848 21 79 0 0
2 0 12 19048 59280 590076 0 0 4 84 415 935 26 73 0 1
2 0 12 19072 59304 590104 0 0 0 0 358 850 27 73 0 0
2 0 12 18956 59404 590140 0 0 4 1128 422 919 25 74 0 1
2 0 12 18824 59532 590172 0 0 0 0 357 839 26 74 0 0
2 0 12 18840 59540 590204 0 0 0 0 393 886 22 78 0 0
2 0 12 18740 59640 590232 0 0 0 0 367 837 22 78 0 0
0 1 12 18648 59700 590268 0 0 16 0 391 774 34 62 0 4
0 2 12 18388 59800 590432 0 0 260 1876 470 388 2 2 0 96
0 1 12 17188 59856 591608 0 0 1232 8 603 333 0 1 0 99
1 1 12 14268 60000 594212 0 0 2748 0 522 425 0 2 0 98
0 1 12 10840 60116 597404 0 0 3308 0 608 395 0 3 0 97
4 1 12 10128 59624 598320 0 0 1704 0 509 244 1 35 0 64
0 1 20 10124 58976 598716 0 0 3660 24 664 312 1 11 0 88
1 1 20 9416 58484 599856 0 0 3176 0 633 622 5 1 0 94
0 1 20 9560 57596 600128 0 0 3528 0 1068 3061 17 6 0 77
3 1 20 9560 57240 600636 0 0 2708 0 959 4046 18 6 0 76
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 1 20 10096 56316 600856 0 0 3240 0 804 3018 17 6 0 77
0 2 20 9868 54828 602856 0 0 3156 8 620 390 2 12 0 86
0 1 20 10044 52652 604952 0 0 3916 20 634 406 0 1 0 99
0 1 20 9108 50216 608528 0 0 3700 0 574 453 2 2 0 96
1 1 20 9408 47736 610748 0 0 3684 4 625 448 0 2 0 98
0 1 20 9304 47752 611008 0 0 2980 0 558 425 1 3 0 96
0 2 20 10008 45744 612276 0 0 4408 16 596 436 0 1 0 99
0 1 20 9376 45704 613136 0 0 3560 0 573 377 1 3 0 96
0 1 20 9608 45488 613104 0 0 3720 0 649 405 1 2 0 97
0 1 20 9264 45580 613204 0 0 2728 0 536 417 3 1 0 96

All I did was emerging the kernel sources and doing some surfing with
konqueror (which stalled quite a few times without even repainting itself) to
test the system behaviour.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/