Re: mod_timer: list_add corruption: WARNING: CPU: 1 PID: 0 at lib/list_debug.c:33__list_add+0xbe/0xd0()
From: Srivatsa S. Bhat
Date: Fri Jul 19 2013 - 07:51:36 EST
On 07/19/2013 04:55 PM, Thomas Gleixner wrote:
> On Tue, 16 Jul 2013, Srivatsa S. Bhat wrote:
>> ------------[ cut here ]------------
>> WARNING: CPU: 1 PID: 0 at lib/list_debug.c:33 __list_add+0xbe/0xd0()
>> list_add corruption. prev->next should be next (ffff8810396b5568), but was (null). (prev=ffff88102c1344c0).
>
> Can you please enable debugobjects?
>
Sure Thomas, please find the new traces below, with
debug objects enabled.
Regards,
Srivatsa S. Bhat
------------[ cut here ]------------
WARNING: CPU: 1 PID: 0 at lib/debugobjects.c:260 debug_print_object+0x8e/0xb0()
ODEBUG: init active (active state 0) object type: timer_list hint: br_multicast_group_expired+0x0/0x110 [bridge]
Modules linked in: ebtable_nat ebtables nfs fscache nf_conntrack_ipv4 nf_defrag_ipv4 ipt_REJECT xt_CHECKSUM iptable_mangle iptable_filter ip_tables nfsd lockd nfs_acl exportfs auth_rpcgss autofs4 sunrpc 8021q garp bridge stp llc ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 vfat fat vhost_net macvtap macvlan vhost tun kvm_intel kvm uinput iTCO_wdt iTCO_vendor_support cdc_ether usbnet mii microcode i2c_i801 i2c_core lpc_ich mfd_core shpchp mlx4_core ioatdma dca be2net wmi acpi_cpufreq mperf ext4 jbd2 mbcache dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 3.11.0-rc1-dbg-a #16
Hardware name: IBM -[8737R2A]-/00Y2738, BIOS -[B2E120RUS-1.20]- 11/30/2012
0000000000000104 ffff88103fc43908 ffffffff8161b3c9 ffffffff81a14151
ffff88103fc43958 ffff88103fc43948 ffffffff8104e6bc 0000000000000000
ffffffff81a2cfae ffff88101b21cfd8 ffffffff81c37ba0 ffffffff828f2ca8
Call Trace:
<IRQ> [<ffffffff8161b3c9>] dump_stack+0x59/0x80
[<ffffffff8104e6bc>] warn_slowpath_common+0x8c/0xc0
[<ffffffff8104e7a6>] warn_slowpath_fmt+0x46/0x50
[<ffffffff812b5aee>] debug_print_object+0x8e/0xb0
[<ffffffffa04247f0>] ? br_multicast_free_pg+0x20/0x20 [bridge]
[<ffffffff812b65e2>] ? __debug_object_init+0x42/0x3f0
[<ffffffff812b67bf>] __debug_object_init+0x21f/0x3f0
[<ffffffff812b69df>] debug_object_init+0x1f/0x30
[<ffffffff81060ea9>] init_timer_key+0x39/0x100
[<ffffffffa0425ec5>] br_ip4_multicast_query+0x155/0x380 [bridge]
[<ffffffffa0427eef>] br_multicast_ipv4_rcv+0x2cf/0x3d0 [bridge]
[<ffffffff8162140b>] ? _raw_spin_unlock+0x2b/0x50
[<ffffffffa0419a9b>] ? br_fdb_update+0x1db/0x2b0 [bridge]
[<ffffffffa04284b5>] br_multicast_rcv+0x45/0x60 [bridge]
[<ffffffffa041bdfe>] br_handle_frame_finish+0x16e/0x3c0 [bridge]
[<ffffffffa041bac8>] br_handle_frame+0x238/0x400 [bridge]
[<ffffffffa041b890>] ? br_del_bridge+0x80/0x80 [bridge]
[<ffffffff81539ca7>] __netif_receive_skb_core+0x237/0x960
[<ffffffff81539ade>] ? __netif_receive_skb_core+0x6e/0x960
[<ffffffff8153a3f7>] __netif_receive_skb+0x27/0x70
[<ffffffff8153c6fd>] netif_receive_skb+0x2d/0x210
[<ffffffff81527e65>] ? __netdev_alloc_skb+0xa5/0x110
[<ffffffffa0129a0f>] be_rx_compl_process+0xef/0x140 [be2net]
[<ffffffffa0129dc2>] be_process_rx+0xe2/0x1a0 [be2net]
[<ffffffffa0129fbd>] be_poll+0x13d/0x1d0 [be2net]
[<ffffffff8153dab8>] net_rx_action+0xd8/0x2a0
[<ffffffff81058e19>] __do_softirq+0x149/0x400
[<ffffffff8105922d>] irq_exit+0xed/0x100
[<ffffffff8162d206>] do_IRQ+0x66/0xe0
[<ffffffff8162182f>] common_interrupt+0x6f/0x6f
<EOI> [<ffffffff814f148b>] ? cpuidle_enter_state+0x5b/0xe0
[<ffffffff814f1487>] ? cpuidle_enter_state+0x57/0xe0
[<ffffffff81625cd0>] ? notifier_call_chain+0x120/0x120
[<ffffffff814f197f>] cpuidle_idle_call+0xcf/0x160
[<ffffffff8100cf0e>] arch_cpu_idle+0xe/0x30
[<ffffffff810b1291>] cpu_idle_loop+0x81/0x3a0
[<ffffffff810b1620>] cpu_startup_entry+0x70/0x80
[<ffffffff810334fc>] start_secondary+0xdc/0xe0
---[ end trace 68b46b9f74e14585 ]---
------------[ cut here ]------------
WARNING: CPU: 1 PID: 0 at lib/debugobjects.c:260 debug_print_object+0x8e/0xb0()
ODEBUG: init active (active state 0) object type: timer_list hint: br_multicast_group_expired+0x0/0x110 [bridge]
Modules linked in: ebtable_nat ebtables nfs fscache nf_conntrack_ipv4 nf_defrag_ipv4 ipt_REJECT xt_CHECKSUM iptable_mangle iptable_filter ip_tables nfsd lockd nfs_acl exportfs auth_rpcgss autofs4 sunrpc 8021q garp bridge stp llc ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 vfat fat vhost_net macvtap macvlan vhost tun kvm_intel kvm uinput iTCO_wdt iTCO_vendor_support cdc_ether usbnet mii microcode i2c_i801 i2c_core lpc_ich mfd_core shpchp mlx4_core ioatdma dca be2net wmi acpi_cpufreq mperf ext4 jbd2 mbcache dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 0 Comm: swapper/1 Tainted: G W 3.11.0-rc1-dbg-a #16
Hardware name: IBM -[8737R2A]-/00Y2738, BIOS -[B2E120RUS-1.20]- 11/30/2012
0000000000000104 ffff88103fc43908 ffffffff8161b3c9 ffffffff81a14151
ffff88103fc43958 ffff88103fc43948 ffffffff8104e6bc 0000000000000000
ffffffff81a2cfae ffff88101b21cfd8 ffffffff81c37ba0 ffffffff828f2ca8
Call Trace:
<IRQ> [<ffffffff8161b3c9>] dump_stack+0x59/0x80
[<ffffffff8104e6bc>] warn_slowpath_common+0x8c/0xc0
[<ffffffff8104e7a6>] warn_slowpath_fmt+0x46/0x50
[<ffffffff812b5aee>] debug_print_object+0x8e/0xb0
[<ffffffffa04247f0>] ? br_multicast_free_pg+0x20/0x20 [bridge]
[<ffffffff812b65e2>] ? __debug_object_init+0x42/0x3f0
[<ffffffff812b67bf>] __debug_object_init+0x21f/0x3f0
[<ffffffff812b69df>] debug_object_init+0x1f/0x30
[<ffffffff81060ea9>] init_timer_key+0x39/0x100
[<ffffffffa0425ec5>] br_ip4_multicast_query+0x155/0x380 [bridge]
[<ffffffffa0427eef>] br_multicast_ipv4_rcv+0x2cf/0x3d0 [bridge]
[<ffffffff8156d93e>] ? nf_hook_slow+0x16e/0x2a0
[<ffffffffa0419a07>] ? br_fdb_update+0x147/0x2b0 [bridge]
[<ffffffffa04284b5>] br_multicast_rcv+0x45/0x60 [bridge]
[<ffffffffa041bdfe>] br_handle_frame_finish+0x16e/0x3c0 [bridge]
[<ffffffffa041bac8>] br_handle_frame+0x238/0x400 [bridge]
[<ffffffffa041b890>] ? br_del_bridge+0x80/0x80 [bridge]
[<ffffffff81539ca7>] __netif_receive_skb_core+0x237/0x960
[<ffffffff81539ade>] ? __netif_receive_skb_core+0x6e/0x960
[<ffffffff810c2d1d>] ? trace_hardirqs_on+0xd/0x10
[<ffffffff8153a3f7>] __netif_receive_skb+0x27/0x70
[<ffffffff8153c6fd>] netif_receive_skb+0x2d/0x210
[<ffffffff81527e65>] ? __netdev_alloc_skb+0xa5/0x110
[<ffffffffa0129a0f>] be_rx_compl_process+0xef/0x140 [be2net]
[<ffffffffa0129dc2>] be_process_rx+0xe2/0x1a0 [be2net]
[<ffffffffa0129fbd>] be_poll+0x13d/0x1d0 [be2net]
[<ffffffff8153dab8>] net_rx_action+0xd8/0x2a0
[<ffffffff81058e19>] __do_softirq+0x149/0x400
[<ffffffff8105922d>] irq_exit+0xed/0x100
[<ffffffff8162d206>] do_IRQ+0x66/0xe0
[<ffffffff8162182f>] common_interrupt+0x6f/0x6f
<EOI> [<ffffffff814f148b>] ? cpuidle_enter_state+0x5b/0xe0
[<ffffffff814f1487>] ? cpuidle_enter_state+0x57/0xe0
[<ffffffff81625cd0>] ? notifier_call_chain+0x120/0x120
[<ffffffff814f197f>] cpuidle_idle_call+0xcf/0x160
[<ffffffff8100cf0e>] arch_cpu_idle+0xe/0x30
[<ffffffff810b1291>] cpu_idle_loop+0x81/0x3a0
[<ffffffff810b1620>] cpu_startup_entry+0x70/0x80
[<ffffffff810334fc>] start_secondary+0xdc/0xe0
---[ end trace 68b46b9f74e14586 ]---
------------[ cut here ]------------
WARNING: CPU: 1 PID: 0 at lib/debugobjects.c:260 debug_print_object+0x8e/0xb0()
ODEBUG: init active (active state 0) object type: timer_list hint: br_multicast_group_expired+0x0/0x110 [bridge]
Modules linked in: ebtable_nat ebtables nfs fscache nf_conntrack_ipv4 nf_defrag_ipv4 ipt_REJECT xt_CHECKSUM iptable_mangle iptable_filter ip_tables nfsd lockd nfs_acl exportfs auth_rpcgss autofs4 sunrpc 8021q garp bridge stp llc ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 vfat fat vhost_net macvtap macvlan vhost tun kvm_intel kvm uinput iTCO_wdt iTCO_vendor_support cdc_ether usbnet mii microcode i2c_i801 i2c_core lpc_ich mfd_core shpchp mlx4_core ioatdma dca be2net wmi acpi_cpufreq mperf ext4 jbd2 mbcache dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 0 Comm: swapper/1 Tainted: G W 3.11.0-rc1-dbg-a #16
Hardware name: IBM -[8737R2A]-/00Y2738, BIOS -[B2E120RUS-1.20]- 11/30/2012
0000000000000104 ffff88103fc43908 ffffffff8161b3c9 ffffffff81a14151
ffff88103fc43958 ffff88103fc43948 ffffffff8104e6bc 0000000000000000
ffffffff81a2cfae ffff88101b21cfd8 ffffffff81c37ba0 ffffffff828f2ca8
Call Trace:
<IRQ> [<ffffffff8161b3c9>] dump_stack+0x59/0x80
[<ffffffff8104e6bc>] warn_slowpath_common+0x8c/0xc0
[<ffffffff8104e7a6>] warn_slowpath_fmt+0x46/0x50
[<ffffffff812b5aee>] debug_print_object+0x8e/0xb0
[<ffffffffa04247f0>] ? br_multicast_free_pg+0x20/0x20 [bridge]
[<ffffffff812b65e2>] ? __debug_object_init+0x42/0x3f0
[<ffffffff812b67bf>] __debug_object_init+0x21f/0x3f0
[<ffffffff810c2d1d>] ? trace_hardirqs_on+0xd/0x10
[<ffffffff812b69df>] debug_object_init+0x1f/0x30
[<ffffffff81060ea9>] init_timer_key+0x39/0x100
[<ffffffffa0425ec5>] br_ip4_multicast_query+0x155/0x380 [bridge]
[<ffffffffa0427eef>] br_multicast_ipv4_rcv+0x2cf/0x3d0 [bridge]
[<ffffffff8156d93e>] ? nf_hook_slow+0x16e/0x2a0
[<ffffffffa04199a4>] ? br_fdb_update+0xe4/0x2b0 [bridge]
[<ffffffffa04284b5>] br_multicast_rcv+0x45/0x60 [bridge]
[<ffffffffa041bdfe>] br_handle_frame_finish+0x16e/0x3c0 [bridge]
[<ffffffffa041bac8>] br_handle_frame+0x238/0x400 [bridge]
[<ffffffffa041b890>] ? br_del_bridge+0x80/0x80 [bridge]
[<ffffffff81539ca7>] __netif_receive_skb_core+0x237/0x960
[<ffffffff81539ade>] ? __netif_receive_skb_core+0x6e/0x960
[<ffffffff810c2d1d>] ? trace_hardirqs_on+0xd/0x10
[<ffffffff8153a3f7>] __netif_receive_skb+0x27/0x70
[<ffffffff8153c6fd>] netif_receive_skb+0x2d/0x210
[<ffffffff81527e65>] ? __netdev_alloc_skb+0xa5/0x110
[<ffffffffa0129a0f>] be_rx_compl_process+0xef/0x140 [be2net]
[<ffffffffa0129dc2>] be_process_rx+0xe2/0x1a0 [be2net]
[<ffffffffa0129fbd>] be_poll+0x13d/0x1d0 [be2net]
[<ffffffff8153dab8>] net_rx_action+0xd8/0x2a0
[<ffffffff81058e19>] __do_softirq+0x149/0x400
[<ffffffff8105922d>] irq_exit+0xed/0x100
[<ffffffff8162d206>] do_IRQ+0x66/0xe0
[<ffffffff8162182f>] common_interrupt+0x6f/0x6f
<EOI> [<ffffffff814f148b>] ? cpuidle_enter_state+0x5b/0xe0
[<ffffffff814f1487>] ? cpuidle_enter_state+0x57/0xe0
[<ffffffff81625cd0>] ? notifier_call_chain+0x120/0x120
[<ffffffff814f197f>] cpuidle_idle_call+0xcf/0x160
[<ffffffff8100cf0e>] arch_cpu_idle+0xe/0x30
[<ffffffff810b1291>] cpu_idle_loop+0x81/0x3a0
[<ffffffff810b1620>] cpu_startup_entry+0x70/0x80
[<ffffffff810334fc>] start_secondary+0xdc/0xe0
---[ end trace 68b46b9f74e14587 ]---
------------[ cut here ]------------
WARNING: CPU: 1 PID: 0 at lib/debugobjects.c:260 debug_print_object+0x8e/0xb0()
ODEBUG: init active (active state 0) object type: timer_list hint: br_multicast_group_expired+0x0/0x110 [bridge]
Modules linked in: ebtable_nat ebtables nfs fscache nf_conntrack_ipv4 nf_defrag_ipv4 ipt_REJECT xt_CHECKSUM iptable_mangle iptable_filter ip_tables nfsd lockd nfs_acl exportfs auth_rpcgss autofs4 sunrpc 8021q garp bridge stp llc ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables ipv6 vfat fat vhost_net macvtap macvlan vhost tun kvm_intel kvm uinput iTCO_wdt iTCO_vendor_support cdc_ether usbnet mii microcode i2c_i801 i2c_core lpc_ich mfd_core shpchp mlx4_core ioatdma dca be2net wmi acpi_cpufreq mperf ext4 jbd2 mbcache dm_mirror dm_region_hash dm_log dm_mod
CPU: 1 PID: 0 Comm: swapper/1 Tainted: G W 3.11.0-rc1-dbg-a #16
Hardware name: IBM -[8737R2A]-/00Y2738, BIOS -[B2E120RUS-1.20]- 11/30/2012
0000000000000104 ffff88103fc43908 ffffffff8161b3c9 ffffffff81a14151
ffff88103fc43958 ffff88103fc43948 ffffffff8104e6bc 0000000000000000
ffffffff81a2cfae ffff88101b21cfd8 ffffffff81c37ba0 ffffffff828f2ca8
Call Trace:
<IRQ> [<ffffffff8161b3c9>] dump_stack+0x59/0x80
[<ffffffff8104e6bc>] warn_slowpath_common+0x8c/0xc0
[<ffffffff8104e7a6>] warn_slowpath_fmt+0x46/0x50
[<ffffffff812b5aee>] debug_print_object+0x8e/0xb0
[<ffffffffa04247f0>] ? br_multicast_free_pg+0x20/0x20 [bridge]
[<ffffffff812b65e2>] ? __debug_object_init+0x42/0x3f0
[<ffffffff812b67bf>] __debug_object_init+0x21f/0x3f0
[<ffffffff810c2d1d>] ? trace_hardirqs_on+0xd/0x10
[<ffffffff812b69df>] debug_object_init+0x1f/0x30
[<ffffffff81060ea9>] init_timer_key+0x39/0x100
[<ffffffffa0425ec5>] br_ip4_multicast_query+0x155/0x380 [bridge]
[<ffffffffa0427eef>] br_multicast_ipv4_rcv+0x2cf/0x3d0 [bridge]
[<ffffffff8156d93e>] ? nf_hook_slow+0x16e/0x2a0
[<ffffffffa04199a4>] ? br_fdb_update+0xe4/0x2b0 [bridge]
[<ffffffffa04284b5>] br_multicast_rcv+0x45/0x60 [bridge]
[<ffffffffa041bdfe>] br_handle_frame_finish+0x16e/0x3c0 [bridge]
[<ffffffffa041bac8>] br_handle_frame+0x238/0x400 [bridge]
[<ffffffffa041b890>] ? br_del_bridge+0x80/0x80 [bridge]
[<ffffffff81539ca7>] __netif_receive_skb_core+0x237/0x960
[<ffffffff81539ade>] ? __netif_receive_skb_core+0x6e/0x960
[<ffffffff8153a3f7>] __netif_receive_skb+0x27/0x70
[<ffffffff8153c6fd>] netif_receive_skb+0x2d/0x210
[<ffffffff81527e65>] ? __netdev_alloc_skb+0xa5/0x110
[<ffffffffa0129a0f>] be_rx_compl_process+0xef/0x140 [be2net]
[<ffffffffa0129dc2>] be_process_rx+0xe2/0x1a0 [be2net]
[<ffffffffa0129fbd>] be_poll+0x13d/0x1d0 [be2net]
[<ffffffff8153dab8>] net_rx_action+0xd8/0x2a0
[<ffffffff81058e19>] __do_softirq+0x149/0x400
[<ffffffff8105922d>] irq_exit+0xed/0x100
[<ffffffff8162d206>] do_IRQ+0x66/0xe0
[<ffffffff8162182f>] common_interrupt+0x6f/0x6f
<EOI> [<ffffffff814f148b>] ? cpuidle_enter_state+0x5b/0xe0
[<ffffffff814f1487>] ? cpuidle_enter_state+0x57/0xe0
[<ffffffff81625cd0>] ? notifier_call_chain+0x120/0x120
[<ffffffff814f197f>] cpuidle_idle_call+0xcf/0x160
[<ffffffff8100cf0e>] arch_cpu_idle+0xe/0x30
[<ffffffff810b1291>] cpu_idle_loop+0x81/0x3a0
[<ffffffff810b1620>] cpu_startup_entry+0x70/0x80
[<ffffffff810334fc>] start_secondary+0xdc/0xe0
---[ end trace 68b46b9f74e14588 ]---
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/