Re: [2.6.14-rt1] slowdown / oops.

From: Ingo Molnar
Date: Wed Nov 02 2005 - 08:55:08 EST



* Ingo Molnar <mingo@xxxxxxx> wrote:

>
> * PaweÅ Sikora <pluto@xxxxxxxx> wrote:
>
> > 2).
> > During `scp bigfile to another machine` I get an oops:
> > http://149.156.124.14/~pluto/tmp/2.6.14-rt2-oops.jpg [796 kB]
>
> is routing to that other box covered by any of the iptables NAT rules?
> Does the crash happen if you turn off all firewalling via "iptables
> -F"?

ah, managed to reproduce a crash in the NAT code with your .config (see
below). This indeed seems to be some sort of use-after-free bug:
0x6b6b6b6b6b is SLAB_DEBUG's POISON_FREE - use-after-free poison byte.
This bug is either caused by and unique to -rt, or possibly present
upstream too.

Ingo

BUG: Unable to handle kernel paging request at virtual address 6b6b6b6b
printing eip:
c03a859f
*pde = 00000000
Oops: 0000 [#1]
PREEMPT
Modules linked in:
CPU: 0
EIP: 0060:[<c03a859f>] Not tainted VLI
EFLAGS: 00010246 (2.6.14-rt4)
EIP is at __ip_conntrack_find+0x5f/0x110
eax: 00000000 ebx: 6b6b6b6b ecx: c013ae7f edx: 00000001
esi: c23d7e28 edi: 000018e0 ebp: c23d7df4 esp: c23d7de4
ds: 007b es: 007b ss: 0068 preempt: 00000001
Process softirq-net-rx/ (pid: 5, threadinfo=c23d6000 task=c23d47b0 stack_left=7600 worst_left=-1)
Stack: 00000000 00000000 c23d7e28 c23d7ecc c23d7e04 c03a8674 f490a57c c03aae90
c23d7e48 c03a9204 c23d7e28 c0498020 f881d040 00000000 00000000 c0498020
c23d7ecc 0101000a c0591600 0701000a 0006c9c2 00000000 c04975e8 c23d7e8c
Call Trace:
[<c0103cc7>] show_stack+0x97/0xd0 (32)
[<c0103eb2>] show_registers+0x192/0x250 (68)
[<c01040ef>] die+0xdf/0x190 (56)
[<c03f1ff6>] do_page_fault+0x176/0x57c (72)
[<c0103933>] error_code+0x4f/0x54 (76)
[<c03a8674>] ip_conntrack_find_get+0x24/0x60 (16)
[<c03a9204>] ip_conntrack_in+0xc4/0x370 (68)
[<c03c46f9>] nf_iterate+0x59/0x90 (36)
[<c03c4782>] nf_hook_slow+0x52/0x100 (48)
[<c0373f62>] ip_rcv+0x182/0x4f0 (64)
[<c035f71d>] netif_receive_skb+0x15d/0x1e0 (52)
[<c02f2b57>] rtl8139_rx+0x1b7/0x340 (80)
[<c02f2ec8>] rtl8139_poll+0x58/0x110 (40)
[<c035f8f2>] net_rx_action+0x72/0x140 (24)
[<c011ed19>] ksoftirqd+0xb9/0x140 (40)
[<c012d6e4>] kthread+0x94/0xa0 (28)
[<c01010d9>] kernel_thread_helper+0x5/0xc (1036156956)
---------------------------
| preempt count: 00000001 ]
| 1-level deep critical section nesting:
----------------------------------------
.. [<c013ae7f>] .... add_preempt_count+0xf/0x20
.....[<c0104048>] .. ( <= die+0x38/0x190)

------------------------------
| showing all locks held by: | (softirq-net-rx//5 [c23d47b0, 98]):
------------------------------

#001: [f7e2c664] {&tp->rx_lock}
... acquired at: rtl8139_poll+0x39/0x110

#002: [c0497bc0] {ip_conntrack_lock}
... acquired at: ip_conntrack_find_get+0x1b/0x60

Code: 01 00 00 00 e8 f3 28 d9 ff ff 05 a0 2a 59 c0 b8 01 00 00 00 e8 83 29 d9 ff a1 08 42 3f c0 8b 40 08 a8 08 0f 85 a6 00 00 00 8b 1b <8b> 03 0f 18 00 90 89 f8 03 05 80 2a 59 c0 39 c3 0f 84 82 00 00
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/