Re: perf: perf_fuzzer crashing due to slab poison

From: Vince Weaver
Date: Mon Sep 19 2016 - 10:42:24 EST



I've updated both the haswell and skylake machine to stock 4.8-rc7
and both are still getting slab corruption in task_structs when running
the perf_fuzzer.

On skylake:
[35484.208607] Slab corruption (Not tainted): task_struct start=ffffa181f19b6140, len=3648
[35484.216764] 010: 6a 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b jkkkkkkkkkkkkkkk
[35484.224534] Single bit error detected. Probably bad RAM.
[35484.229932] Run memtest86+ or a similar memory test tool.

On haswell it seems random which bug gets hit first, the task_struct slab
corruption one or the cgroup_attach hang. This morning it hit the
latter.

[36074.181453] INFO: rcu_sched detected stalls on CPUs/tasks:
[36074.187387] 4-...: (32 GPs behind) idle=5c9/1/2 softirq=206028/206028 fqs=2626
[36074.195292] (detected by 3, t=5254 jiffies, g=254720, c=254719, q=131)
[36074.202396] Task dump for CPU 4:
[36074.205831] swapper/4 R running task 0 0 1 0x00000008
[36074.213447] 0000000000000246 ffff88011978fe70 0000000000000000 00000000000003b8
[36074.221529] 000020ca48fcb186 000020ca49179475 ffff880119790000 ffffe8ffffd0a400
[36074.229523] ffffffff81cd40e0 ffffffff81d32580 0000000000000004 ffff88011978fec0
[36074.237523] Call Trace:
[36074.240142] [<ffffffff815b0b97>] ? cpuidle_enter+0x17/0x20
[36074.246093] [<ffffffff810bba3a>] ? call_cpuidle+0x2a/0x50
[36074.251975] [<ffffffff810bbe11>] ? cpu_startup_entry+0x291/0x340
[36074.258553] [<ffffffff8104feb3>] ? start_secondary+0x133/0x140
[36102.071290] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [perf_fuzzer:2966]
[36102.152977] CPU: 3 PID: 2966 Comm: perf_fuzzer Not tainted 4.8.0-rc7 #195
[36102.160250] Hardware name: LENOVO 10AM000AUS/SHARKBAY, BIOS FBKT72AUS 01/26/2014
[36102.168163] task: ffff8800bbfc6380 task.stack: ffff880113708000
[36102.174558] RIP: 0010:[<ffffffff81100291>] [<ffffffff81100291>] smp_call_function_single+0xb1/0x110
[36102.184364] RSP: 0018:ffff88011370bdb8 EFLAGS: 00000202
[36102.190026] RAX: 0000000000000000 RBX: 0000000000000004 RCX: 0000000000000830
[36102.197657] RDX: 0000000000000001 RSI: 00000000000008fb RDI: 0000000000000830
[36102.205321] RBP: ffff88011370bdf8 R08: 00000000c48d3dfe R09: 1e3cdd0700000000
[36102.212995] R10: 0000000000000001 R11: 0000000000000000 R12: ffffffff8116ab00
[36102.220647] R13: ffff8800be96e800 R14: ffffffff81170220 R15: 0000000000000000
[36102.228254] FS: 00007fb736b6d700(0000) GS:ffff88011eac0000(0000) knlGS:0000000000000000
[36102.236921] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[36102.243106] CR2: 0000000000835620 CR3: 0000000112503000 CR4: 00000000001407e0
[36102.250733] DR0: 0000000000000000 DR1: 000000ec4dbff2cf DR2: 0000000000000000
[36102.258420] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000600
[36102.266020] Stack:
[36102.268202] ffff8800bbfc6380 0000000000000246 0000000000000000 ffffffff8116ab00
[36102.276774] ffff88011370be28 0000000000000003 ffff88011eb208e8 ffff8800be96e800
[36102.285405] ffff88011370be70 ffffffff811697ee 0000000000000000 ffff8800be96e800
[36102.294030] Call Trace:
[36102.297207] [<ffffffff8116ab00>] ? perf_cgroup_attach+0x50/0x50
[36102.304170] [<ffffffff811697ee>] event_function_call+0x13e/0x150
[36102.311235] [<ffffffff81170220>] ? __perf_cgroup_move+0x60/0x60
[36102.318241] [<ffffffff81169190>] ? update_group_times+0x40/0x40
[36102.325246] [<ffffffff81169860>] ? _perf_event_disable+0x60/0x60
[36102.332295] [<ffffffff811698ab>] _perf_event_enable+0x4b/0x70
[36102.339097] [<ffffffff811693ba>] perf_event_for_each_child+0x3a/0xa0
[36102.346479] [<ffffffff81172397>] perf_event_task_enable+0x57/0xa0
[36102.353676] [<ffffffff8108c3cc>] SyS_prctl+0x35c/0x460
[36102.359855] [<ffffffff81003016>] ? lockdep_sys_exit_thunk+0x16/0x30
[36102.367159] [<ffffffff81727c36>] entry_SYSCALL_64_fastpath+0x1e/0xad
[36102.374596] Code: df 4c 89 e2 e8 51 fe ff ff 48 83 c4 30 5b 41 5c 5d c3 48 8d 75 d0 48 89 d1 89 df 4c 89 e2 e8 37 fe ff ff 8b 55 e8 83 e2 01 74 0a <f3> 90 8b 55 e8 83 e2 01 75 f6 48 83 c4 30 5b 41 5c 5d c3 8b 05
[36130.069119] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [perf_fuzzer:2966]
....