> I just got serveral Oopsen on my firewall last night. I plugged a monitor
> and keyboard in and took some photos of the screen and tried hitting
> sysrq. Unfortunately the batteries in my camera have gone flat so I can't
> look at the images. I did write down the EIP of one of the oopsen.
> It was c01837a2.
> System.map:
> c01835f8 T do_tty_hangup
> c0183870 T tty_hangup
>
> SysRq S, U and T all Oopsed. P and M worked and B locked the machine
> completely so that no further SysRqs worked.
>
> The kernel is the same one (in fact I haven't rebooted since) as mentioned
> in my last Oops report (2.2.11 Oops in ipfw_output_check.) - that is,
> 2.2.11 with the following patches applied:
> 1. the autofs unfix that was in 2.2.12pre kernels;
> 2. cipe-1.3.0;
> 3. a patch for avoiding nmap scans (been on l-k a couple of times); and
> 4. the tcp fixes from 2.2.11 release notes.
>
> Uptime was about 8 days.
I should have said that I'd just tried connecting to the Internet at the
time of the hang. pppd seemed to think it had come up and logged:
Sep 2 23:02:07 falcon pppd[25920]: local IP address 212.126.149.239
Sep 2 23:02:07 falcon pppd[25920]: remote IP address 212.126.144.20
and started ip-up but no packets were transmitted. The last thing that
was logged was:
Sep 2 23:02:24 falcon named[321]: XX /192.168.0.5/munchkin.cipe/A
(the nameserver on falcon is a secondary for .cipe).
When I plugged in the monitor, this was on the screen:
--------------------------------------------------------------------------------
e9>] [<c010eebe>] [<c01998b4>]
[<c0114fe3>] [<c0107eb0>] [<c0198784>] [<c01997bb>] [<c010cf04>] [<c01997
bb>] [<c0107ae9>] [<c010eebe>]
[<c01998b4>] [<c0114fe3>] [<c0107eb0>] [<c0198784>] [<c01997bb>] [<c010cf
04>] [<c01997bb>] [<c0107ae9>]
[<c010eebe>] [<c01998b4>] [<c0114fe3>] [<c0107eab>] [<c0107eb0>] [<c01987
84>] [<c01997bb>] [<c010cf04>]
[<c01997bb>] [<c0107ae9>] [<c010eebe>] [<c01998b4>] [<c0114fe3>] [<c0107e
b0>] [<c0198784>] [<c01997bb>]
[<c010cf04>] [<c01997bb>] [<c0107ae9>] [<c010eebe>] [<c01998b4>] [<c0114f
e3>] [<c0107eab>] [<c0107eb0>]
[<c0198784>] [<c01997bb>] [<c010cf04>] [<c01997bb>] [<c0107ae9>] [<c010ee
be>] [<c01998b4>] [<c0114fe3>]
[<c0107eb0>] [<c0198784>] [<c01997bb>] [<c010cf04>] [<c01997bb>] [<c0107a
e9>] [<c010eebe>] [<c01998b4>]
[<c0114fe3>] [<c0107eb0>] [<c0198784>] [<c01997bb>] [<c010cf04>] [<c01997
bb>] [<c0107ae9>] [<c010eebe>]
[<c01998b4>] [<c0114fe3>] [<c0107eb0>] [<c0198784>] [<c01997bb>] [<c010cf
04>] [<c01997bb>] [<c0107ae9>]
[<c010eebe>] [<c01998b4>]
Code: 8a 93 11 01 00 00 c1 e2 08 31 c0 8a 83 10 01 00 00 89 c3 09
Aiee, killing interrupt handler
Kernel panic: Attempted to kill the idle task!
In interrupt handler - not syncing
--------------------------------------------------------------------------------
ksymoops-0.7c says (with a little persuasion):
ksymoops 0.7c on i586 2.2.12. Options used
-v /usr/src/linux-running/vmlinux (specified)
-K (specified)
-L (specified)
-o /lib/modules/2.2.11/ (specified)
-m /usr/src/linux-running/System.map (specified)
No modules in ksyms, skipping objects
Reading Oops report from the terminal
Call Trace: [<c010eebe>] [<c01998b4>]
[<c0114fe3>] [<c0107eb0>] [<c0198784>] [<c01997bb>] [<c010cf04>] [<c01997bb>] [<c0107ae9>] [<c010eebe>]
[<c01998b4>] [<c0114fe3>] [<c0107eb0>] [<c0198784>] [<c01997bb>] [<c010cf04>] [<c01997bb>] [<c0107ae9>]
[<c010eebe>] [<c01998b4>] [<c0114fe3>] [<c0107eab>] [<c0107eb0>] [<c0198784>] [<c01997bb>] [<c010cf04>]
[<c01997bb>] [<c0107ae9>] [<c010eebe>] [<c01998b4>] [<c0114fe3>] [<c0107eb0>] [<c0198784>] [<c01997bb>]
[<c010cf04>] [<c01997bb>] [<c0107ae9>] [<c010eebe>] [<c01998b4>] [<c0114fe3>] [<c0107eab>] [<c0107eb0>]
[<c0198784>] [<c01997bb>] [<c010cf04>] [<c01997bb>] [<c0107ae9>] [<c010eebe>] [<c01998b4>] [<c0114fe3>]
[<c0107eb0>] [<c0198784>] [<c01997bb>] [<c010cf04>] [<c01997bb>] [<c0107ae9>] [<c010eebe>] [<c01998b4>]
[<c0114fe3>] [<c0107eb0>] [<c0198784>] [<c01997bb>] [<c010cf04>] [<c01997bb>] [<c0107ae9>] [<c010eebe>]
[<c01998b4>] [<c0114fe3>] [<c0107eb0>] [<c0198784>] [<c01997bb>] [<c010cf04>] [<c01997bb>] [<c0107ae9>]
[<c010eebe>] [<c01998b4>]
Code: 8a 93 11 01 00 00 c1 e2 08 31 c0 8a 83 10 01 00 00 89 c3 09
Using defaults from ksymoops -t elf32-i386 -a i386
Trace; c010eebe <schedule+26e/284>
Trace; c01998b4 <tvecs+70/2586>
Trace; c0114fe3 <do_exit+233/238>
Trace; c0107eb0 <die_if_no_fixup+0/40>
Trace; c0198784 <stext_lock+e60/1f20>
Trace; c01997bb <stext_lock+1e97/1f20>
Trace; c010cf04 <do_page_fault+2b8/308>
Trace; c01997bb <stext_lock+1e97/1f20>
Trace; c0107ae9 <error_code+2d/34>
Trace; c010eebe <schedule+26e/284>
Trace; c01998b4 <tvecs+70/2586>
Trace; c0114fe3 <do_exit+233/238>
Trace; c0107eb0 <die_if_no_fixup+0/40>
Trace; c0198784 <stext_lock+e60/1f20>
Trace; c01997bb <stext_lock+1e97/1f20>
Trace; c010cf04 <do_page_fault+2b8/308>
Trace; c01997bb <stext_lock+1e97/1f20>
Trace; c0107ae9 <error_code+2d/34>
Trace; c010eebe <schedule+26e/284>
Trace; c01998b4 <tvecs+70/2586>
Trace; c0114fe3 <do_exit+233/238>
Trace; c0107eab <die+33/38>
Trace; c0107eb0 <die_if_no_fixup+0/40>
Trace; c0198784 <stext_lock+e60/1f20>
Trace; c01997bb <stext_lock+1e97/1f20>
Trace; c010cf04 <do_page_fault+2b8/308>
Trace; c01997bb <stext_lock+1e97/1f20>
Trace; c0107ae9 <error_code+2d/34>
Trace; c010eebe <schedule+26e/284>
Trace; c01998b4 <tvecs+70/2586>
Trace; c0114fe3 <do_exit+233/238>
Trace; c0107eb0 <die_if_no_fixup+0/40>
Trace; c0198784 <stext_lock+e60/1f20>
Trace; c01997bb <stext_lock+1e97/1f20>
Trace; c010cf04 <do_page_fault+2b8/308>
Trace; c01997bb <stext_lock+1e97/1f20>
Trace; c0107ae9 <error_code+2d/34>
Trace; c010eebe <schedule+26e/284>
Trace; c01998b4 <tvecs+70/2586>
Trace; c0114fe3 <do_exit+233/238>
Trace; c0107eab <die+33/38>
Trace; c0107eb0 <die_if_no_fixup+0/40>
Trace; c0198784 <stext_lock+e60/1f20>
Trace; c01997bb <stext_lock+1e97/1f20>
Trace; c010cf04 <do_page_fault+2b8/308>
Trace; c01997bb <stext_lock+1e97/1f20>
Trace; c0107ae9 <error_code+2d/34>
Trace; c010eebe <schedule+26e/284>
Trace; c01998b4 <tvecs+70/2586>
Trace; c0114fe3 <do_exit+233/238>
Trace; c0107eb0 <die_if_no_fixup+0/40>
Trace; c0198784 <stext_lock+e60/1f20>
Trace; c01997bb <stext_lock+1e97/1f20>
Trace; c010cf04 <do_page_fault+2b8/308>
Trace; c01997bb <stext_lock+1e97/1f20>
Trace; c0107ae9 <error_code+2d/34>
Trace; c010eebe <schedule+26e/284>
Trace; c01998b4 <tvecs+70/2586>
Trace; c0114fe3 <do_exit+233/238>
Trace; c0107eb0 <die_if_no_fixup+0/40>
Trace; c0198784 <stext_lock+e60/1f20>
Trace; c01997bb <stext_lock+1e97/1f20>
Trace; c010cf04 <do_page_fault+2b8/308>
Trace; c01997bb <stext_lock+1e97/1f20>
Trace; c0107ae9 <error_code+2d/34>
Trace; c010eebe <schedule+26e/284>
Trace; c01998b4 <tvecs+70/2586>
Trace; c0114fe3 <do_exit+233/238>
Trace; c0107eb0 <die_if_no_fixup+0/40>
Trace; c0198784 <stext_lock+e60/1f20>
Trace; c01997bb <stext_lock+1e97/1f20>
Trace; c010cf04 <do_page_fault+2b8/308>
Trace; c01997bb <stext_lock+1e97/1f20>
Trace; c0107ae9 <error_code+2d/34>
Trace; c010eebe <schedule+26e/284>
Trace; c01998b4 <tvecs+70/2586>
Code; 00000000 Before first symbol
00000000 <_EIP>:
Code; 00000000 Before first symbol
0: 8a 93 11 01 00 movb 0x111(%ebx),%dl
Code; 00000005 Before first symbol
5: 00
Code; 00000006 Before first symbol
6: c1 e2 08 shll $0x8,%edx
Code; 00000009 Before first symbol
9: 31 c0 xorl %eax,%eax
Code; 0000000b Before first symbol
b: 8a 83 10 01 00 movb 0x110(%ebx),%al
Code; 00000010 Before first symbol
10: 00
Code; 00000011 Before first symbol
11: 89 c3 movl %eax,%ebx
Code; 00000013 Before first symbol
13: 09 00 orl %eax,(%eax)
Aiee, killing interrupt handler
Kernel panic: Attempted to kill the idle task!
I pressed SysRq-s and got:
--------------------------------------------------------------------------------
SysRq: Emergency Sync
Syncing device 03:01 ... <1>Unable to handle kernel NULL pointer dereference at
virtual address 00000328
current->tss.cr3 = 00000000, %cr3 = 00101000
*pde = 00000000
Oops: 0000
CPU: 0
EIP: 0010:[<c01837a2>]
EFLAGS: 00010202
eax: 00006540 ebx: 000002c0
esi: c4adc000 edi: c4adc100
ds: 0018 es: 0018 ss: 0018
Process (pid: 0, process nr: 19, stackpage=c4539000)
Stack: c46ab240 c4539f84 c4adc000 00000000 c010ee6a c4adc000 00000000 c4539f9c
c46ab240 00000246 00000000 c4538000 c0123655 00000032 c3d03480 00000001
00000000 c4538000 c46ab26c c0123829 c46ab240 00000301 00000301 00000121
Call Trace: [<c010ee6a>] [<c0123655>] [<c0123829>] [<c01238bc>] [<c018314a>]
Code: 39 43 68 75 2e 83 7b 6c 00 74 28 6a 01 53 6a 01 e8 29 a2 f8
Aiee, killing interrupt handler
Kernel panic: Attempted to kill the idle task!
In interrupt handler - not syncing
--------------------------------------------------------------------------------
ksymoops 0.7c on i586 2.2.12. Options used
-v /usr/src/linux-running/vmlinux (specified)
-K (specified)
-L (specified)
-o /lib/modules/2.2.11/ (specified)
-m /usr/src/linux-running/System.map (specified)
No modules in ksyms, skipping objects
Reading Oops report from the terminal
Syncing device 03:01 ... <1>Unable to handle kernel NULL pointer dereference at
current->tss.cr3 = 00000000, %cr3 = 00101000
*pde = 00000000
Oops: 0000
CPU: 0
EIP: 0010:[<c01837a2>]
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010202
eax: 00006540 ebx: 000002c0
esi: c4adc000 edi: c4adc100
ds: 0018 es: 0018 ss: 0018
Process (pid: 0, process nr: 19, stackpage=c4539000)
Stack: c46ab240 c4539f84 c4adc000 00000000 c010ee6a c4adc000 00000000 c4539f9c
c46ab240 00000246 00000000 c4538000 c0123655 00000032 c3d03480 00000001
00000000 c4538000 c46ab26c c0123829 c46ab240 00000301 00000301 00000121
Call Trace: [<c010ee6a>] [<c0123655>] [<c0123829>] [<c01238bc>] [<c018314a>]
Code: 39 43 68 75 2e 83 7b 6c 00 74 28 6a 01 53 6a 01 e8 29 a2 f8
>>EIP; c01837a2 <do_tty_hangup+1aa/278> <=====
Trace; c010ee6a <schedule+21a/284>
Trace; c0123655 <__wait_on_buffer+8d/c4>
Trace; c0123829 <sync_buffers+19d/1d0>
Trace; c01238bc <fsync_dev+2c/34>
Trace; c018314a <go_sync+106/11c>
Code; c01837a2 <do_tty_hangup+1aa/278>
00000000 <_EIP>:
Code; c01837a2 <do_tty_hangup+1aa/278> <=====
0: 39 43 68 cmpl %eax,0x68(%ebx) <=====
Code; c01837a5 <do_tty_hangup+1ad/278>
3: 75 2e jne 33 <_EIP+0x33> c01837d5 <do_tty_hangup+1dd/278>
Code; c01837a7 <do_tty_hangup+1af/278>
5: 83 7b 6c 00 cmpl $0x0,0x6c(%ebx)
Code; c01837ab <do_tty_hangup+1b3/278>
9: 74 28 je 33 <_EIP+0x33> c01837d5 <do_tty_hangup+1dd/278>
Code; c01837ad <do_tty_hangup+1b5/278>
b: 6a 01 pushl $0x1
Code; c01837af <do_tty_hangup+1b7/278>
d: 53 pushl %ebx
Code; c01837b0 <do_tty_hangup+1b8/278>
e: 6a 01 pushl $0x1
Code; c01837b2 <do_tty_hangup+1ba/278>
10: e8 29 a2 f8 00 call f8a23e <_EIP+0xf8a23e> c110d9e0 <END_OF_CODE+f1d748/????>
Aiee, killing interrupt handler
Kernel panic: Attempted to kill the idle task!
SysRq-m:
--------------------------------------------------------------------------------
Free pages: 1416kB
( Free: 354 (64 128 512)
0*4kB 77*8kB 34*16kB 0*32kB 2*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB - 1
416kB)
Swap cache: add 11814, delete 11519, find 115326/128973
Free swap: 259232kB
20480 pages of RAM
534 reserved pages
2986 pages shared
295 pages swap cached
6908 pages in file cache
7203 pages in page cache
32 pages in page table cache
Buffer memory: 26480kB
Buffer heads: 26520
Buffer blocks: 26480
Buffer hashed: 24035
CLEAN: 24105 buffers, 119 used (last=24069), 0 locked, 0 protected, 0 dirty
LOCKED: 44 buffers, 2 used (last=38), 0 locked, 0 protected, 0 dirty
DIRTY: 15 buffers, 4 used (last=9), 0 locked, 0 protected, 15 dirty
Networking buffers in use : 425
Total network buffer allocations : 4935230
Total failed network buffer allocs : 0
IP fragment buffer size : 0
--------------------------------------------------------------------------------
SysRq-p:
--------------------------------------------------------------------------------
SysRq: Show Regs
EIP: 0010:[<c011140c>] EFLAGS: 00000246
EAX: 00000000 EBX: 00000000 ECX: 00000004 EDX: c3c26000
ESI: c4538000 EDI: 00000328 EBP: c4538000 DS: 0018 ES: 0018
CR0: 8005003b CR3: 00000328 CR3: 00101000
--------------------------------------------------------------------------------
SysRq-t:
--------------------------------------------------------------------------------
current->tss.cr3 = 00000000, %cr3 = 00101000
*pde = 00000000
Oops: 0000
CPU: 0
EIP: 0010:[<c01102be>]
EFLAGS: 00010006
eax: 00000000 ebx: 00000000 ecx: 00000004 edx: c3c26000
esi: 00000074 edi: c453a000 ebp: c4539da0 esp: c4539d74
ds: 0018 es: 0018 ss: 0018
Process (pid: 0, process nr: 19, stackpage=c4539000)
Stack: c4e24000 30303030 30303030 30303030 30303030 00000000 30303030 30303030
30303030 30303030 c01cec00 c4539db4 c01104c9 fffd58db c453a000 c01efad0
0000000f c0182e27 c01a2fe0 00000014 00002710 0000000c c4539e74 c0181f32
Call Trace: [<c01104d9>] [<c0182e27>] [<c01a2fe0>] [<c0181f32>] [<c018db84>] [<c
018dbce>] [<c0108aa5>]
[<c01087ef>] [<c0108bb8>] [<c0108838>] [<c011140c>] [<c0114df2>] [<c0199f
1a>] [<c0107eab>] [<c0107eb0>]
[<c0198784>] [<c01997bb>] [<c010cf04>] [<c01997bb>] [<c0107ae9>] [<c01837
a2>] [<c010ee6a>] [<c0123655>]
[<c0123829>] [<c01238bc>] [<c018314a>]
Code: 8b 40 0c 50 68 4b 99 19 c0 e8 28 14 00 00 83 c4 08 8d 87 c0
Aiee, killing interrupt handler
Kernel panic: Attempted to kill the idle task!
In interrupt handler - not syncing
--------------------------------------------------------------------------------
ksymoops 0.7c on i586 2.2.12. Options used
-v /usr/src/linux-running/vmlinux (specified)
-K (specified)
-L (specified)
-o /lib/modules/2.2.11/ (specified)
-m /usr/src/linux-running/System.map (specified)
No modules in ksyms, skipping objects
Reading Oops report from the terminal
current->tss.cr3 = 00000000, %cr3 = 00101000
*pde = 00000000
Oops: 0000
CPU: 0
EIP: 0010:[<c01102be>]
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010006
eax: 00000000 ebx: 00000000 ecx: 00000004 edx: c3c26000
esi: 00000074 edi: c453a000 ebp: c4539da0 esp: c4539d74
ds: 0018 es: 0018 ss: 0018
Process (pid: 0, process nr: 19, stackpage=c4539000)
Stack: c4e24000 30303030 30303030 30303030 30303030 00000000 30303030 30303030
30303030 30303030 c01cec00 c4539db4 c01104c9 fffd58db c453a000 c01efad0
0000000f c0182e27 c01a2fe0 00000014 00002710 0000000c c4539e74 c0181f32
Call Trace: [<c01104d9>] [<c0182e27>] [<c01a2fe0>] [<c0181f32>] [<c018db84>] [<c018dbce>] [<c0108aa5>]
[<c01087ef>] [<c0108bb8>] [<c0108838>] [<c011140c>] [<c0114df2>] [<c0199f1a>] [<c0107eab>] [<c0107eb0>]
[<c0198784>] [<c01997bb>] [<c010cf04>] [<c01997bb>] [<c0107ae9>] [<c01837a2>] [<c010ee6a>] [<c0123655>]
[<c0123829>] [<c01238bc>] [<c018314a>]
Code: 8b 40 0c 50 68 4b 99 19 c0 e8 28 14 00 00 83 c4 08 8d 87 c0
>>EIP; c01102be <show_task+86/1c0> <=====
Trace; c01104d9 <show_state+51/58>
Trace; c0182e27 <handle_sysrq+233/2dc>
Trace; c01a2fe0 <NR_TYPES+f0/888>
Trace; c0181f32 <handle_scancode+18e/2e4>
Trace; c018db84 <handle_kbd_event+4c/84>
Trace; c018dbce <keyboard_interrupt+12/18>
Trace; c0108aa5 <handle_IRQ_event+31/68>
Trace; c01087ef <do_8259A_IRQ+77/a8>
Trace; c0108bb8 <do_IRQ+24/40>
Trace; c0108838 <common_interrupt+18/20>
Trace; c011140c <panic+e0/f0>
Trace; c0114df2 <do_exit+42/238>
Trace; c0199f1a <tvecs+6d6/2586>
Trace; c0107eab <die+33/38>
Trace; c0107eb0 <die_if_no_fixup+0/40>
Trace; c0198784 <stext_lock+e60/1f20>
Trace; c01997bb <stext_lock+1e97/1f20>
Trace; c010cf04 <do_page_fault+2b8/308>
Trace; c01997bb <stext_lock+1e97/1f20>
Trace; c0107ae9 <error_code+2d/34>
Trace; c01837a2 <do_tty_hangup+1aa/278>
Trace; c010ee6a <schedule+21a/284>
Trace; c0123655 <__wait_on_buffer+8d/c4>
Trace; c0123829 <sync_buffers+19d/1d0>
Trace; c01238bc <fsync_dev+2c/34>
Trace; c018314a <go_sync+106/11c>
Code; c01102be <show_task+86/1c0>
00000000 <_EIP>:
Code; c01102be <show_task+86/1c0> <=====
0: 8b 40 0c movl 0xc(%eax),%eax <=====
Code; c01102c1 <show_task+89/1c0>
3: 50 pushl %eax
Code; c01102c2 <show_task+8a/1c0>
4: 68 4b 99 19 c0 pushl $0xc019994b
Code; c01102c7 <show_task+8f/1c0>
9: e8 28 14 00 00 call 1436 <_EIP+0x1436> c01116f4 <printk+0/174>
Code; c01102cc <show_task+94/1c0>
e: 83 c4 08 addl $0x8,%esp
Code; c01102cf <show_task+97/1c0>
11: 8d 87 c0 00 00 leal 0xc0(%edi),%eax
Code; c01102d4 <show_task+9c/1c0>
16: 00
Aiee, killing interrupt handler
Kernel panic: Attempted to kill the idle task!
Anything else I can add?
-- Tim Quidquid latine dictum sit, altum viditur.
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/