Re: [PATCH 0/4] 2.6.21-rc7 NFS writes: fix a series of issues

From: Rogier Wolff
Date: Sun Apr 29 2007 - 15:41:49 EST

On Tue, Apr 17, 2007 at 10:37:38PM -0700, Andrew Morton wrote:
> Florin, can we please see /proc/meminfo as well?
>
> Also the result of `echo m > /proc/sysrq-trigger'

Hi,

It's been a while since this thread died out, but maybe I'm
having the same problem. Networking, large part of memory is
buffering writes.....

In my case I'm using NBD.

Oh,

/sys/block/nbd0/stat gives:
636 88 5353 1700 991 19554 162272 63156 43 1452000 61802352
I put some debugging stuff in nbd, and it DOES NOT KNOW about the
43 requests that the io scheduler claims are in flight at the
driver....

Those requests start a couple of seconds AFTER the whole thing
grinds to a halt.

I switched from crashing my 512Mb-ram-workstation to my development
machine, which has only 64M of RAM. (I got the development machine
back up and running after some effort).

My rsync (and also "sync" if I call it, or reboot without -n) also
gets stuck in D state:

<4>[ 622.364000] rsync D 0019C170 0 2456 2455 (NOTLB)
<4>[ 622.364000] c04d7c80 00000086 c1f61ba8 0019c170 00000000 00000008 c31048a0 0003382e
<4>[ 622.364000] 00000000 c24c9908 00000286 c1092740 c3e12590 c2330a50 c2330b5c 00061a80
<4>[ 622.364000] 8639a400 00000078 c04d7cd0 00000000 c10801b8 c04d7c88 c03082af c0176c48
<4>[ 622.364000] Call Trace:
<4>[ 622.364000] [<c03082af>] io_schedule+0xe/0x16
<4>[ 622.364000] [<c0176c48>] sync_buffer+0x0/0x2e
<4>[ 622.364000] [<c0176c73>] sync_buffer+0x2b/0x2e
<4>[ 622.364000] [<c03083b9>] __wait_on_bit+0x2c/0x51
<4>[ 622.364000] [<c0176c48>] sync_buffer+0x0/0x2e
<4>[ 622.364000] [<c0308451>] out_of_line_wait_on_bit+0x73/0x7b
<4>[ 622.364000] [<c012970e>] wake_bit_function+0x0/0x3c
<4>[ 622.364000] [<c012970e>] wake_bit_function+0x0/0x3c
<4>[ 622.364000] [<c0176cce>] __wait_on_buffer+0x22/0x25
<4>[ 622.364000] [<c0198cf0>] ext3_find_entry+0x1aa/0x36f
<4>[ 622.364000] [<c01a2324>] journal_dirty_metadata+0x1b6/0x1d3
<4>[ 622.364000] [<c01990e7>] ext3_lookup+0x28/0xc6
<4>[ 622.364000] [<c0161611>] real_lookup+0x53/0xc2
<4>[ 622.364000] [<c0161881>] do_lookup+0x57/0x9d
<4>[ 622.364000] [<c0162075>] __link_path_walk+0x7ae/0xb81
<4>[ 622.364000] [<c011cb77>] __do_softirq+0x57/0x83
<4>[ 622.364000] [<c0162485>] link_path_walk+0x3d/0xa0
<4>[ 622.364000] [<c015a4e7>] sys_lchown+0x3c/0x44
<4>[ 622.364000] [<c015a8c7>] get_unused_fd+0xa0/0xbc
<4>[ 622.364000] [<c0162840>] do_path_lookup+0x1b7/0x200
<4>[ 622.364000] [<c01628e1>] __path_lookup_intent_open+0x42/0x72
<4>[ 622.364000] [<c0162931>] path_lookup_open+0x20/0x25
<4>[ 622.364000] [<c0163026>] open_namei+0x8c/0x532
<4>[ 622.364000] [<c015a328>] sys_fchmodat+0xac/0xb9
<4>[ 622.364000] [<c015a71b>] do_filp_open+0x25/0x39
<4>[ 622.364000] [<c015a4e7>] sys_lchown+0x3c/0x44
<4>[ 622.364000] [<c015a8c7>] get_unused_fd+0xa0/0xbc
<4>[ 622.364000] [<c015a9d5>] do_sys_open+0x42/0xbe
<4>[ 622.364000] [<c015aa6b>] sys_open+0x1a/0x1c
<4>[ 622.364000] [<c0103dbc>] syscall_call+0x7/0xb
<4>[ 622.364000] =======================

----------------------
<6>[ 871.520000] SysRq : Show Memory
<6>[ 871.520000] Mem-info:
<4>[ 871.520000] DMA per-cpu:
<4>[ 871.520000] CPU 0: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
<4>[ 871.520000] Normal per-cpu:
<4>[ 871.520000] CPU 0: Hot: hi: 6, btch: 1 usd: 0 Cold: hi: 2, btch: 1 usd: 0
<4>[ 871.520000] Active:5632 inactive:6764 dirty:0 writeback:302 unstable:0
<4>[ 871.520000] free:717 slab:2024 mapped:926 pagetables:135 bounce:0
<4>[ 871.520000] DMA free:1104kB min:252kB low:312kB high:376kB active:3600kB inactive:6820kB present:16256kB pages_scanned:0 all_unreclaimable? no
<4>[ 871.520000] lowmem_reserve[]: 0 47
<4>[ 871.520000] Normal free:1764kB min:760kB low:948kB high:1140kB active:18928kB inactive:20236kB present:48708kB pages_scanned:0 all_unreclaimable? no
<4>[ 871.520000] lowmem_reserve[]: 0 0
<4>[ 871.520000] DMA: 118*4kB 19*8kB 2*16kB 2*32kB 0*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1104kB
<4>[ 871.520000] Normal: 171*4kB 23*8kB 0*16kB 0*32kB 4*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 1764kB
<4>[ 871.520000] Swap cache: add 0, delete 0, find 0/0, race 0+0
<4>[ 871.520000] Free swap = 0kB
<4>[ 871.520000] Total swap = 0kB
<6>[ 871.520000] Free swap: 0kB
<6>[ 871.520000] 16368 pages of RAM
<6>[ 871.520000] 0 pages of HIGHMEM
<6>[ 871.520000] 1044 reserved pages
<6>[ 871.520000] 13456 pages shared
<6>[ 871.520000] 0 pages swap cached
<6>[ 871.520000] 0 pages dirty
<6>[ 871.520000] 302 pages writeback
<6>[ 871.520000] 926 pages mapped
<6>[ 871.520000] 2024 pages slab
<6>[ 871.520000] 135 pages pagetables

----------------------
ozon:/home/wolff# cat /proc/meminfo
MemTotal: 61296 kB
MemFree: 2752 kB
Buffers: 2228 kB
Cached: 29968 kB
SwapCached: 0 kB
Active: 22632 kB
Inactive: 27056 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 0 kB
Writeback: 1208 kB
AnonPages: 17512 kB
Mapped: 3704 kB
Slab: 8088 kB
SReclaimable: 3656 kB
SUnreclaim: 4432 kB
PageTables: 552 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
CommitLimit: 30648 kB
Committed_AS: 50092 kB
VmallocTotal: 974548 kB
VmallocUsed: 208 kB
VmallocChunk: 974160 kB



--
** R.E.Wolff@xxxxxxxxxxxx ** http://www.BitWizard.nl/ ** +31-15-2600998 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement.
Does it sit on the couch all day? Is it unemployed? Please be specific!
Define 'it' and what it isn't doing. --------- Adapted from lxrbot FAQ
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/