[OOPS] linux 2.6, lots of ipsec traffic while rebooting box.

From: Ranjeet Shetye (ranjeet.shetye2@zultys.com)
Date: Fri Jul 25 2003 - 18:53:56 EST


Test system.

kernel is 2.6.0-test1-bk (Fri, July 25), talking to a 2.4.21 kernel
with IPSec backport. IPSec is tunnel mode, ESP and AH with AES and MD5
for phase 1 and phase 2. PFS is off. default lifetimes. no modules
loaded on 2.6 kernel.

Linux 2.6 was running netserver as part of netperf testing. While IPSec
traffic was running, I issued a reboot on linux 2.6 box. That caused
the OOPS.

Also, /usr/src/linux is a symbolic link to the linux-2.5 bk tree, so
the ksymoops is working on the correct files.

Ranjeet.

INIT: Sending processes the TERM signal
INIT: Sending processes the KILL signal
INIT: Pid 3120 [id c2] seems to hang
INIT: Pid 3121 [id c3] seems to hang
INIT: Pid 3122 [id c4] seems to hang
INIT: Pid 3123 [id c5] seems to hang
INIT: Pid 3124 [id c6] seems to hang
  * Stopping local... [ ok ]
  * Stopping vcron... [ ok ]
  * Stopping msyslog... [ ok ]
  * Syncing hardware clock to system clock [UTC]...Debug: sleeping
function
  * called from invalid context at include/linux/rwsem.h:43
Call Trace:
  [<c011ae31>] __might_sleep+0x5f/0x72
  [<c0118375>] do_page_fault+0x199/0x4b7
  [<c0124552>] update_wall_time+0xf/0x3a
  [<c0124998>] do_timer+0xdf/0xe4
  [<c010eedd>] do_gettimeofday+0x19/0x90
  [<c012011a>] sys_gettimeofday+0xb7/0xbb
  [<c01181dc>] do_page_fault+0x0/0x4b7
  [<c0109c35>] error_code+0x2d/0x38

   [ ok ]
  * Saving random seed... [ ok ]
  * Stopping cupsd... [ ok ]
  * Bringing eth0 down... [ ok ]
  * Unmounting network filesystems... [ ok ]
  * Stopping samba... [ ok ]
  * Stopping spamd... [ ok ]
  * Stopping sshd... [ ok ]
  * Bringing eth1 down... [ ok ]
  * Bringing lo down... [ ok ]
  * Deactivating swap... [ ok ]
  * Stopping devfsd... [ ok ]
  * Sending all processes the TERserio: kseriod exiting
M signal...Unable to handle kernel paging request at virtual address
5a5a5a5e
  printing eip:
c041db40
*pde = 00000000
Oops: 0002 [#2]
CPU: 0
EIP: 0060:[<c041db40>] Not tainted
EFLAGS: 00010202
EIP is at __xfrm_state_delete+0x29/0xab
eax: 5a5a5a5a ebx: c12e9f48 ecx: c05e6460 edx: c05e6460
esi: c5b76000 edi: 000000ff ebp: c5b77cac esp: c5b77ca0
ds: 007b es: 007b ss: 0068
Process racoon (pid: 3245, threadinfo=c5b76000 task=c5b51280)
Stack: c3b59844 c12e9f48 c5b76000 c5b77cc0 c041dbf3 c12e9f48 c12e9f48
c5b76000 c5b77ce8 c041dcf8 c12e9f48 c05e6460 c05e6460 c5b76000
000002d8 000000ff cf5f1a20 c619bccc c5b77d0c c042a2ae 000000ff
000000d0 fffffffd 00000000 Call Trace:
  [<c041dbf3>] xfrm_state_delete+0x31/0x52
  [<c041dcf8>] xfrm_state_flush+0xe4/0x112
  [<c042a2ae>] pfkey_flush+0x62/0xef
  [<c042b5e4>] pfkey_process+0xa6/0xaa
  [<c042c5a2>] pfkey_sendmsg+0x165/0x1a4
  [<c03cc73e>] sock_sendmsg+0x92/0xaf
  [<c015d230>] link_path_walk+0x5f1/0x8e3
  [<c0165a91>] dput+0x1bd/0x213
  [<c0423477>] unix_find_other+0x8a/0x15c
  [<c03cc4fa>] sockfd_lookup+0x1a/0x72
  [<c03cdbcc>] sys_sendto+0xe8/0x107
  [<c03cc316>] sock_destroy_inode+0x1d/0x21
  [<c03cc316>] sock_destroy_inode+0x1d/0x21
  [<c0167668>] destroy_inode+0x36/0x50
  [<c0168868>] iput+0x63/0x7c
  [<c03cdc21>] sys_send+0x36/0x3a
  [<c03ce55d>] sys_socketcall+0x156/0x27b
  [<c014ef95>] filp_close+0x4b/0x74
  [<c014f022>] sys_close+0x64/0x96
  [<c010920b>] syscall_call+0x7/0xb

Code: 89 50 04 89 02 c7 43 04 00 02 20 00 c7 03 00 01 10 00 ff 4b
<0>Kernel panic: Fatal exception in interrupt
In interrupt handler - not syncing

ksymoops 2.4.9 on i686 2.6.0-test1-bk. Options used
      -v /usr/src/linux/vmlinux (specified)
      -k /proc/ksyms (default)
      -l /proc/modules (default)
      -o /lib/modules/2.6.0-test1-bk/ (default)
      -m /usr/src/linux/System.map (specified)

Error (regular_file): read_ksyms stat /proc/ksyms failed
No modules in ksyms, skipping objects
No ksyms, skipping lsmod
Oops: 0002 [#2]
CPU: 0
EIP: 0060:[<c041db40>] Not tainted
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010202
eax: 5a5a5a5a ebx: c12e9f48 ecx: c05e6460 edx: c05e6460
esi: c5b76000 edi: 000000ff ebp: c5b77cac esp: c5b77ca0
ds: 007b es: 007b ss: 0068
Stack: c3b59844 c12e9f48 c5b76000 c5b77cc0 c041dbf3 c12e9f48 c12e9f48
c5b76000 c5b77ce8 c041dcf8 c12e9f48 c05e6460 c05e6460 c5b76000
000002d8 000000ff cf5f1a20 c619bccc c5b77d0c c042a2ae 000000ff
000000d0 fffffffd 00000000 Call Trace:
  [<c041dbf3>] xfrm_state_delete+0x31/0x52
  [<c041dcf8>] xfrm_state_flush+0xe4/0x112
  [<c042a2ae>] pfkey_flush+0x62/0xef
  [<c042b5e4>] pfkey_process+0xa6/0xaa
  [<c042c5a2>] pfkey_sendmsg+0x165/0x1a4
  [<c03cc73e>] sock_sendmsg+0x92/0xaf
  [<c015d230>] link_path_walk+0x5f1/0x8e3
  [<c0165a91>] dput+0x1bd/0x213
  [<c0423477>] unix_find_other+0x8a/0x15c
  [<c03cc4fa>] sockfd_lookup+0x1a/0x72
  [<c03cdbcc>] sys_sendto+0xe8/0x107
  [<c03cc316>] sock_destroy_inode+0x1d/0x21
  [<c03cc316>] sock_destroy_inode+0x1d/0x21
  [<c0167668>] destroy_inode+0x36/0x50
  [<c0168868>] iput+0x63/0x7c
  [<c03cdc21>] sys_send+0x36/0x3a
  [<c03ce55d>] sys_socketcall+0x156/0x27b
  [<c014ef95>] filp_close+0x4b/0x74
  [<c014f022>] sys_close+0x64/0x96
  [<c010920b>] syscall_call+0x7/0xb
Code: 89 50 04 89 02 c7 43 04 00 02 20 00 c7 03 00 01 10 00 ff 4b

>> EIP; c041db40 <__xfrm_state_delete+29/ab> <=====

>> eax; 5a5a5a5a <__crc_snd_ctl_unregister_ioctl+12fd3f/273d82>
>> ebx; c12e9f48 <__crc_memcpy_tokerneliovec+8bec0/4e7a4d>
>> ecx; c05e6460 <xfrm_state_bydst+16c0/2000>
>> edx; c05e6460 <xfrm_state_bydst+16c0/2000>
>> esi; c5b76000 <__crc_neigh_sysctl_register+24c702/46b175>
>> ebp; c5b77cac <__crc_neigh_sysctl_register+24e3ae/46b175>
>> esp; c5b77ca0 <__crc_neigh_sysctl_register+24e3a2/46b175>

Trace; c041dbf3 <xfrm_state_delete+31/52>
Trace; c041dcf8 <xfrm_state_flush+e4/112>
Trace; c042a2ae <pfkey_flush+62/ef>
Trace; c042b5e4 <pfkey_process+a6/aa>
Trace; c042c5a2 <pfkey_sendmsg+165/1a4>
Trace; c03cc73e <sock_sendmsg+92/af>
Trace; c015d230 <link_path_walk+5f1/8e3>
Trace; c0165a91 <dput+1bd/213>
Trace; c0423477 <unix_find_other+8a/15c>
Trace; c03cc4fa <sockfd_lookup+1a/72>
Trace; c03cdbcc <sys_sendto+e8/107>
Trace; c03cc316 <sock_destroy_inode+1d/21>
Trace; c03cc316 <sock_destroy_inode+1d/21>
Trace; c0167668 <destroy_inode+36/50>
Trace; c0168868 <iput+63/7c>
Trace; c03cdc21 <sys_send+36/3a>
Trace; c03ce55d <sys_socketcall+156/27b>
Trace; c014ef95 <filp_close+4b/74>
Trace; c014f022 <sys_close+64/96>
Trace; c010920b <syscall_call+7/b>

Code; c041db40 <__xfrm_state_delete+29/ab>
00000000 <_EIP>:
Code; c041db40 <__xfrm_state_delete+29/ab> <=====
    0: 89 50 04 mov %edx,0x4(%eax) <=====
Code; c041db43 <__xfrm_state_delete+2c/ab>
    3: 89 02 mov %eax,(%edx)
Code; c041db45 <__xfrm_state_delete+2e/ab>
    5: c7 43 04 00 02 20 00 movl $0x200200,0x4(%ebx)
Code; c041db4c <__xfrm_state_delete+35/ab>
    c: c7 03 00 01 10 00 movl $0x100100,(%ebx)
Code; c041db52 <__xfrm_state_delete+3b/ab>
   12: ff 4b 00 decl 0x0(%ebx)

1 error issued. Results may not be reliable.

-- 
Ranjeet Shetye
Senior Software Engineer
Zultys Technologies
Ranjeet dot Shetye2 at Zultys dot com
http://www.zultys.com/
--
The views, opinions, and judgements expressed in this message are solely
those of the author. The message contents have not been reviewed or
approved by Zultys.
-
To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html



This archive was generated by hypermail 2b29 : Thu Jul 31 2003 - 22:00:01 EST