Okay...I'm still seeing the problem on the mail server that I saw before,
but the example I had with the floppy has been fixed, so it seems like
there might be something more that we're missing.
With the pre-8 buffer.c patches, a few minutes after boot:
procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
2 0 0 0 366880 32508 77188 0 0 1 0 250 254 18 4 78
0 0 0 0 366328 32512 77192 0 0 1 0 240 254 17 6 77
0 0 0 0 366788 32552 77192 0 0 2 0 250 251 16 6 78
0 0 0 0 367180 32560 77192 0 0 2 0 300 353 30 13 57
1 0 0 0 367144 32560 77192 0 0 0 0 262 244 3 4 93
0 1 0 0 367640 32576 77312 0 0 121 0 370 358 20 7 73
0 0 0 0 367320 32576 77412 0 0 98 0 287 201 1 2 97
0 0 0 0 366788 32600 77412 0 0 7 0 366 382 13 5 82
0 0 0 0 367752 32644 77416 0 0 2 0 328 333 15 5 80
1 0 0 0 367028 32644 77412 0 0 0 0 304 307 17 6 76
<- I ran sync ->
0 1 0 0 363852 32652 77384 0 0 14 1851 734 913 30 10 60
3 0 0 0 365864 32696 77368 0 0 4 1014 516 714 11 10 79
3 2 0 0 365268 32704 77376 0 0 20 742 386 472 11 4 84
2 8 0 0 364252 32712 77348 0 0 1 218 469 608 24 5 71
0 12 0 0 363516 32716 77352 0 0 16 284 468 644 7 3 90
0 15 0 0 362880 32716 77352 0 0 0 334 399 534 12 4 84
0 18 0 0 362600 32928 77132 0 0 1 354 393 527 5 3 92
0 21 0 0 361848 32928 77132 0 0 0 380 386 506 9 2 88
0 24 0 0 361296 32928 77132 0 0 0 754 383 502 9 4 87
1 24 0 0 361432 32928 77120 0 0 0 520 419 534 0 3 96
0 27 0 0 361008 32928 77120 0 0 0 646 389 514 7 4 89
0 29 0 0 360552 32928 77120 0 0 0 641 412 541 6 4 90
0 31 0 0 360296 32928 77120 0 0 0 853 350 456 3 2 94
0 31 0 0 360412 32928 77088 0 0 0 1413 351 457 0 3 96
0 32 0 0 359964 32928 77088 0 0 0 1081 403 536 8 4 88
1 30 0 0 358752 33148 77248 0 0 10 215 389 605 11 8 82
0 29 0 0 359252 33172 77256 0 0 12 242 429 806 12 8 81
0 29 0 0 360164 33192 77228 0 0 2 309 445 878 11 7 83
0 1 2 0 365184 33204 77024 0 0 48 156 528 822 13 27 60
0 0 0 0 366576 33208 77476 0 0 77 0 418 462 9 5 86
1 0 0 0 366396 33208 77480 0 0 0 0 245 222 9 6 85
It looks to me like there still is some sort of problem, but it now seems
partially gone; the floppy example I mentioned before seems to be fixed.
It looks like it was doing a lot of seeking when it was flushing out, but
for a good deal of the time all reads just totally stopped. Is this
intended? I can duplicate this quite easily on my computer by running the
following (with "vmstat 1" in another console):
# Replace 256 below with the number of megabytes of RAM you have (so it
# doesn't fit in cache)
head -c256m /dev/zero > foo1
cp foo1 /dev/null & sleep 2 && head -c64m /dev/zero > foo2 && sync
It seems that as soon as "sync" is called, all reads totally stop and
freeze until all buffers have been syncd out. "ps axl" shows
"wait_on_buffer" for the WCHAN of the processes that are trying to read.
This wouldn't be too bad for me except for the fact that bdflush() (called
by update) seems to flush by file instead of by a small chunk of
blocks...Eg: try creating a big file and call bdflush(1,0) a few times --
it might sync out a few other small things the first few times, but then it
will hit the file and sync out the whole 32MB in one call (and block all
reads while it does it). So, on a server where big files are occasionally
written, performance can become very jerky.
Hmm...Killing update altogether makes the system run very smoothly. Not
very safe, though. Well, it seems to flush out things occasionally when it
gets low on memory or when diry buffers exceeds the max percentage, but
that's probably not often enough. Hmmmm.
Here's a quick bdflush.c just for testing:
---#include <stdlib.h> #include <unistd.h> #include <asm/unistd.h>
_syscall2(int, bdflush, int, func, int, data);
int main(){ bdflush(1,0); }
---Simon-
| Simon Kirby | Systems Administration | | mailto:sim@netnation.com | NetNation Communications | | http://www.netnation.com/ | Tech: (604) 684-6892 |
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/