I *really* need suggestions here. This is a system which is supposed to
go production on Monday and I'm not feeling too good about this.
Here's some more output:
blah:/usr/src/linux/scripts/ksymoops# cat /proc/mdstat
Personalities : [raid0] [raid1]
read_ahead 1024 sectors
md2 : active raid1 md1[1](F) md0[0](F) 26876288 blocks [2/1] [_U]
md0 : active raid0 sdc1[2] sdb1[1] sda1[0] 26876352 blocks 16k chunks
md1 : active raid0 sdf1[2] sde1[1] sdd1[0] 26876352 blocks 16k chunks
unused devices: <none>
raid0_map bug: hash->zone0==NULL for block 151322631
attempt to access beyond end of device
09:00: rw=0, want=151322632, limit=26876352
dev 09:02 blksize=1024 blocknr=151322631 sector=302645262 size=1024
count=1
raid1: Disk failure on md0, disabling device.
Operation continuing on 1 devices
raid1: md2: rescheduling block 151322631
raid0_map bug: hash->zone0==NULL for block 587268096
attempt to access beyond end of device
09:01: rw=0, want=587268097, limit=26876352
dev 09:02 blksize=1024 blocknr=587268096 sector=1174536192 size=1024
count=1
raid1: only one disk left and IO error.
raid1: md2: rescheduling block 587268096
md: recovery thread got woken up ...
md2: no spare disk to reconstruct array! -- continuing in degraded mode
md: recovery thread finished ...
dirty sb detected, updating.
md: updating md2 RAID superblock on device
(skipping faulty md1 )
(skipping faulty md0 )
.
raid1: md2: unrecoverable I/O read error for block 587268096
raid1: md2: redirecting sector 151322631 to another mirror
raid0_map bug: hash->zone0==NULL for block 151322631
attempt to access beyond end of device
09:01: rw=0, want=151322632, limit=26876352
dev 09:02 blksize=1024 blocknr=151322631 sector=302645262 size=1024
count=1
raid1: only one disk left and IO error.
raid1: md2: rescheduling block 151322631
raid1: md2: unrecoverable I/O read error for block 151322631
md: recovery thread got woken up ...
md2: no spare disk to reconstruct array! -- continuing in degraded mode
md: recovery thread finished ...
raid0_map bug: hash->zone0==NULL for block 1057751388
attempt to access beyond end of device
09:01: rw=0, want=1057751389, limit=26876352
dev 09:02 blksize=1024 blocknr=-1089732260 sector=2115502776 size=1024
count=1
raid1: only one disk left and IO error.
raid1: md2: rescheduling block 3205235036
raid0_map bug: hash->zone0==NULL for block 150847488
attempt to access beyond end of device
09:01: rw=0, want=150847489, limit=26876352
dev 09:02 blksize=1024 blocknr=150847488 sector=301694976 size=1024
count=1
raid1: only one disk left and IO error.
raid1: md2: rescheduling block 150847488
WARNING: This version of ksymoops is obsolete.
WARNING: The current version can be obtained from ftp://ftp.ocs.com.au/pub/ksymoops
Options used: -V (default)
-o /lib/modules/2.2.12-20smp/ (default)
-k /proc/ksyms (default)
-l /proc/modules (default)
-m /usr/src/linux/System.map (default)
-c 1 (default)
You did not tell me where to find symbol information. I will assume
that the log matches the kernel and modules that are running right now
and I'll use the default options above for symbol resolution.
If the current kernel and/or modules do not match the log, you can get
more accurate output by telling me the kernel version and where to find
map, modules, ksyms etc. ksymoops -h explains the options.
Unable to handle kernel paging request at virtual address fc9e4850
current->tss.cr3 = 0d00c000, %cr3 = 0d00c000
*pde = 3b609063
Oops: 0000
CPU: 1
EIP: 0010:[<fc824603>]
EFLAGS: 00010286
eax: 06ce6c2c ebx: fc9e4850 ecx: fc828040 edx: 00000002
esi: 6ce6c2c0 edi: d9cd8580 ebp: 00000004 esp: ccf0dd6c
ds: 0018 es: 0018 ss: 0018
Process rsync (pid: 4020, process nr: 85, stackpage=ccf0d000)
Stack: 00000000 fb5bea00 fc828040 00000010 fc826000 00000020 00000002 c018ca7d
fb5c8140 00000901 d81e5a52 d81e5a54 00000002 d81e5a44 fc821256 00000901
d81e5a52 d81e5a54 00000002 fb5bea1c fc82140b 00000000 d81e5a44 00000000
Call Trace: [<fc828040>] [<fc826000>] [<c018ca7d>] [<fc821256>] [<fc82140b>] [<c01c5479>] [<c018ca7d>]
[<c018cb12>] [<c0184bc0>] [<c012ca4f>] [<c01443f4>] [<c012cc21>] [<c01210fe>] [<c0121543>] [<c0121913>]
[<c0121850>] [<c012a022>] [<c0109410>]
Code: 8b 3b 85 ff 0f 84 9b 00 00 00 8b 07 03 47 08 39 c6 7c 0d 8b
>>EIP: fc824603 <raid1_personality+16c3/ad110>
Trace: fc828040 <raid1_personality+5100/ad110>
Trace: fc826000 <raid1_personality+30c0/ad110>
Trace: c018ca7d <md_map_Rsmpad14c7c4+3d/2b0>
Trace: fc821256 <map_and_make_request+26/50>
Trace: fc82140b <raid1_make_request+18b/310>
Trace: c01c5479 <proc_print_scsidevice_Rsmpb99064f0+339/7790>
Trace: c018ca7d <md_map_Rsmpad14c7c4+3d/2b0>
Trace: c018cb12 <md_map_Rsmpad14c7c4+d2/2b0>
Trace: c0121850 <update_vm_cache_Rsmp94f6e487+880/8e0>
Code: fc824603 <raid1_personality+16c3/ad110> 00000000 <_EIP>: <===
Code: fc824603 <raid1_personality+16c3/ad110> 0: 8b 3b movl (%ebx),%edi <===
Code: fc824605 <raid1_personality+16c5/ad110> 2: 85 ff testl %edi,%edi
Code: fc824607 <raid1_personality+16c7/ad110> 4: 0f 84 9b 00 00 00 je fc8246a8 <raid1_personality+1768/ad110>
Code: fc82460d <raid1_personality+16cd/ad110> a: 8b 07 movl (%edi),%eax
Code: fc82460f <raid1_personality+16cf/ad110> c: 03 47 08 addl 0x8(%edi),%eax
Code: fc824612 <raid1_personality+16d2/ad110> f: 39 c6 cmpl %eax,%esi
Code: fc824614 <raid1_personality+16d4/ad110> 11: 7c 0d jl fc824623 <raid1_personality+16e3/ad110>
Code: fc824616 <raid1_personality+16d6/ad110> 13: 8b 00 movl (%eax),%eax
6 warnings and 1 error issued. Results may not be reliable.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/