Re: kmemleak reports in kernel 3.9.5+

From: Ben Greear
Date: Tue Jun 11 2013 - 14:36:35 EST

On 06/10/2013 03:32 PM, Catalin Marinas wrote:
On 10 June 2013 19:22, Ben Greear <greearb@xxxxxxxxxxxxxxx> wrote:
We had a system go OOM while doing lots of wireless
stations. (System had 8GB of RAM, so I suspect a leak).

I enabled kmemleak in a 3.9.5 (plus some local patches) and
I see the entries below. Any idea if these are real or not?

unreferenced object 0xffff880212281c80 (size 128):
comm "systemd", pid 1, jiffies 4294682684 (age 1159.517s)
hex dump (first 32 bytes):
60 39 27 12 02 88 ff ff 00 02 20 00 00 00 ad de `9'....... .....
10 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<ffffffff815de7bf>] kmemleak_alloc+0x73/0x98
[<ffffffff8118b4d4>] slab_post_alloc_hook+0x28/0x2a
[<ffffffff8118d605>] __kmalloc+0xf9/0x122
[<ffffffff8154946d>] kzalloc.clone.0+0xe/0x10
[<ffffffff81549494>] fib_default_rule_add+0x25/0x7a
[<ffffffffa014f5a9>] ip6mr_net_init+0x7e/0x118 [ipv6]
[<ffffffff8152c992>] ops_init+0xd6/0xf7
[<ffffffff8152cb51>] register_pernet_operations+0xc2/0x16b
[<ffffffff8152cc87>] register_pernet_subsys+0x2e/0x47
[<ffffffffa016db69>] 0xffffffffa016db69
[<ffffffffa016d109>] 0xffffffffa016d109
[<ffffffff8100207f>] do_one_initcall+0x7f/0x13e
[<ffffffff810f3985>] do_init_module+0x44/0x18f
[<ffffffff810f5da7>] load_module+0x14d1/0x168e
[<ffffffff810f6114>] sys_init_module+0xfd/0x101
[<ffffffff815f6599>] system_call_fastpath+0x16/0x1b

No idea yet. You can try:

echo clear > /sys/kernel/debug/kmemleak

and see if there are more appearing after. All seem to have a common
allocation path via debug_object_activate -> ... ->
rcuhead_fixup_activate -> ... -> __debug_object_init.

I let the system run overnight, and there are a good many more
kmemleak reports this morning. I have not yet tried the clear
option.

I have SLUB debugging enabled, so that is probably explains
the debug stuff in the stacks.

Here are some more that may be of interest.

unreferenced object 0xffff88010b719700 (size 40):
comm "gua", pid 11301, jiffies 4352877266 (age 29217.682s)
hex dump (first 32 bytes):
00 e0 10 08 01 88 ff ff a0 68 1e 13 01 88 ff ff .........h......
01 00 00 00 00 00 00 00 70 3d 56 13 01 88 ff ff ........p=V.....
backtrace:
[<ffffffff815de7bf>] kmemleak_alloc+0x73/0x98
[<ffffffff8118b4d4>] slab_post_alloc_hook+0x28/0x2a
[<ffffffff8118d9a7>] kmem_cache_alloc+0xb2/0x123
[<ffffffff81316919>] __debug_object_init+0x43/0x35f
[<ffffffff81316c4c>] debug_object_init_on_stack+0x17/0x19
[<ffffffff810bb5fa>] hrtimer_init_on_stack+0x27/0x75
[<ffffffff815ed663>] schedule_hrtimeout_range_clock+0x67/0x113
[<ffffffff815ed722>] schedule_hrtimeout_range+0x13/0x15
[<ffffffff811aa1fa>] poll_schedule_timeout+0x48/0x64
[<ffffffff811aaff6>] do_select+0x519/0x550
[<ffffffff811df227>] compat_core_sys_select+0x1c7/0x25a
[<ffffffff811df526>] compat_sys_select+0x99/0xc1
[<ffffffff815f7c4c>] sysenter_dispatch+0x7/0x26
[<ffffffffffffffff>] 0xffffffffffffffff


unreferenced object 0xffff880111d33420 (size 40):
comm "ps", pid 25114, jiffies 4352875696 (age 29219.150s)
hex dump (first 32 bytes):
c0 3c 68 0b 01 88 ff ff 70 41 ec 11 01 88 ff ff .<h.....pA......
01 00 00 00 00 00 00 00 e0 6c 5a 12 01 88 ff ff .........lZ.....
backtrace:
[<ffffffff815de7bf>] kmemleak_alloc+0x73/0x98
[<ffffffff8118b4d4>] slab_post_alloc_hook+0x28/0x2a
[<ffffffff8118d9a7>] kmem_cache_alloc+0xb2/0x123
[<ffffffff81316919>] __debug_object_init+0x43/0x35f
[<ffffffff81316c62>] debug_object_init+0x14/0x16
[<ffffffff810b4e0a>] rcuhead_fixup_activate+0x2b/0xba
[<ffffffff81315f12>] debug_object_fixup+0x15/0x1d
[<ffffffff81316557>] debug_object_activate+0x126/0x139
[<ffffffff81118e4a>] __call_rcu.clone.1+0x58/0x22a
[<ffffffff81119065>] call_rcu+0x17/0x19
[<ffffffff8119be2b>] file_free+0x31/0x35
[<ffffffff8119c05a>] __fput+0x1bb/0x1db
[<ffffffff8119c0ca>] ____fput+0xe/0x10
[<ffffffff810b48d5>] task_work_run+0x85/0xb0
[<ffffffff810106c6>] do_notify_resume+0x6d/0x80
[<ffffffff815f68d2>] int_signal+0x12/0x17

unreferenced object 0xffff88010cc92e60 (size 40):
comm "updatedb", pid 24952, jiffies 4352871287 (age 29223.220s)
hex dump (first 32 bytes):
00 60 16 0e 01 88 ff ff d0 af 19 10 01 88 ff ff .`..............
01 00 00 00 00 00 00 00 a0 2d 39 0e 01 88 ff ff .........-9.....
backtrace:
[<ffffffff815de7bf>] kmemleak_alloc+0x73/0x98
[<ffffffff8118b4d4>] slab_post_alloc_hook+0x28/0x2a
[<ffffffff8118d9a7>] kmem_cache_alloc+0xb2/0x123
[<ffffffff81316919>] __debug_object_init+0x43/0x35f
[<ffffffff81316c62>] debug_object_init+0x14/0x16
[<ffffffff810b296c>] __init_work+0x27/0x29
[<ffffffff81241371>] ext4_alloc_inode+0x173/0x1cb
[<ffffffff811b0db4>] alloc_inode+0x1d/0x78
[<ffffffff811b0e7c>] iget_locked+0x6d/0x12f
[<ffffffff81229817>] ext4_iget+0x2c/0x932
[<ffffffff8122f485>] ext4_lookup+0xd2/0x12a
[<ffffffff811a3556>] lookup_real+0x2c/0x47
[<ffffffff811a3e0c>] __lookup_hash+0x33/0x3c
[<ffffffff811a5897>] walk_component+0x73/0x171
[<ffffffff811a6e2c>] path_lookupat+0xa7/0x304
[<ffffffff811a70b4>] filename_lookup+0x2b/0x82

(I sent a similar one to the ext4 mailing list already.)


unreferenced object 0xffff8801c6d2eb80 (size 40):
comm "softirq", pid 0, jiffies 4296641468 (age 85451.733s)
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 70 01 6e 7b 01 88 ff ff ........p.n{....
02 00 00 00 00 00 00 00 58 54 24 ca 01 88 ff ff ........XT$.....
backtrace:
[<ffffffff815de7bf>] kmemleak_alloc+0x73/0x98
[<ffffffff8118b4d4>] slab_post_alloc_hook+0x28/0x2a
[<ffffffff8118d9a7>] kmem_cache_alloc+0xb2/0x123
[<ffffffff81316919>] __debug_object_init+0x43/0x35f
[<ffffffff81316c62>] debug_object_init+0x14/0x16
[<ffffffff810a70df>] init_timer_key+0x2e/0xb3
[<ffffffffa0128e4d>] ipv6_add_addr+0x19f/0x3bc [ipv6]
[<ffffffffa012c885>] addrconf_add_linklocal+0x4c/0x7f [ipv6]
[<ffffffffa012d06a>] addrconf_notify+0x773/0x95e [ipv6]
[<ffffffff815f30d6>] notifier_call_chain+0x37/0x63
[<ffffffff810bc994>] raw_notifier_call_chain+0x14/0x16
[<ffffffff815333ae>] call_netdevice_notifiers+0x4a/0x4f
[<ffffffff8153404d>] netdev_state_change+0x27/0x3a
[<ffffffff815418ef>] linkwatch_do_dev+0x46/0x54
[<ffffffff81541b0e>] __linkwatch_run_queue+0x180/0x1ca
[<ffffffff81541b7d>] linkwatch_event+0x25/0x2c


unreferenced object 0xffff8801c8e41e78 (size 192):
comm "kworker/u:2", pid 157, jiffies 4295509873 (age 86582.869s)
hex dump (first 32 bytes):
41 0d 00 30 02 00 00 00 6b 6b 6b 6b 6b 6b 6b 6b A..0....kkkkkkkk
6b 6b 6b 6b 6b 6b 6b 6b 69 00 00 00 00 0c 2e 32 kkkkkkkki......2
backtrace:
[<ffffffff815de7bf>] kmemleak_alloc+0x73/0x98
[<ffffffff8118b4d4>] slab_post_alloc_hook+0x28/0x2a
[<ffffffff8118d605>] __kmalloc+0xf9/0x122
[<ffffffffa027cb27>] cfg80211_inform_bss_frame+0x114/0x1f8 [cfg80211]
[<ffffffffa03d6865>] ieee80211_bss_info_update+0x66/0x21f [mac80211]
[<ffffffffa040aec6>] ieee80211_rx_bss_info+0x12f/0x1ca [mac80211]
[<ffffffffa040b017>] ieee80211_rx_mgmt_probe_resp+0xb6/0x197 [mac80211]
[<ffffffffa040e8a3>] ieee80211_sta_rx_queued_mgmt+0xdd/0x60e [mac80211]
[<ffffffffa03df0ee>] ieee80211_iface_work+0x238/0x2cc [mac80211]
[<ffffffff810b0cd3>] process_one_work+0x292/0x42e
[<ffffffff810b36af>] worker_thread+0x14f/0x264
[<ffffffff810b7bea>] kthread+0xc7/0xcf
[<ffffffff815f64ec>] ret_from_fork+0x7c/0xb0
[<ffffffffffffffff>] 0xffffffffffffffff

(I plan to investigate the one above...I have some understanding of
the wifi code, and have some patches of my own applied to that code.)


Thanks,
Ben

--
Ben Greear <greearb@xxxxxxxxxxxxxxx>
Candela Technologies Inc http://www.candelatech.com

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/