Re: sched: softlockups in multi_cpu_stop

From: Sasha Levin
Date: Fri Mar 06 2015 - 06:29:00 EST

Next message: Linus Walleij: "Re: [PATCH 0/2] pinctrl: Support for IMG Pistachio"
Previous message: Luis Henriques: "[PATCH 3.16.y-ckt 009/183] ARM: pxa: add regulator_has_full_constraints to corgi board file"
In reply to: Rafael David Tinoco: "Re: sched: softlockups in multi_cpu_stop"
Next in thread: Ingo Molnar: "Re: sched: softlockups in multi_cpu_stop"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

I've bisected this to "locking/rwsem: Check for active lock before bailing on spinning". Relevant parties Cc'ed.

Thanks,
Sasha

On 03/02/2015 02:45 AM, Sasha Levin wrote:
> Hi all,
>
> I'm seeing the following lockup pretty often while fuzzing with trinity:
>
> [ 880.960250] NMI watchdog: BUG: soft lockup - CPU#1 stuck for 447s! [migration/1:14]
> [ 880.960700] Modules linked in:
> [ 880.960700] irq event stamp: 380954
> [ 880.960700] hardirqs last enabled at (380953): restore_args (arch/x86/kernel/entry_64.S:780)
> [ 880.960700] hardirqs last disabled at (380954): apic_timer_interrupt (arch/x86/kernel/entry_64.S:920)
> [ 880.960700] softirqs last enabled at (380952): __do_softirq (./arch/x86/include/asm/preempt.h:22 kernel/softirq.c:300)
> [ 880.960700] softirqs last disabled at (380947): irq_exit (kernel/softirq.c:350 kernel/softirq.c:391)
> [ 880.960700] CPU: 1 PID: 14 Comm: migration/1 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff880052cc8000 ti: ffff880052cd0000 task.ti: ffff880052cd0000
> [ 880.960700] RIP: multi_cpu_stop (kernel/stop_machine.c:192)
> [ 880.960700] RSP: 0000:ffff880052cd7b78 EFLAGS: 00000246
> [ 880.960700] RAX: 0000000000000000 RBX: 0000000042506841 RCX: 1ffff1000a5992d5
> [ 880.960700] RDX: 0000000000000000 RSI: ffffffff9e370ae0 RDI: ffffffffa31716b4
> [ 880.960700] RBP: ffff880052cd7bd8 R08: 0000000000000001 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff9315e211
> [ 880.960700] R13: ffff880052cd7ae8 R14: ffffffff93220213 R15: ffff880052cd7ad8
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff880053200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 0000000002d88fd8 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffffffffa15a9540 ffffffff93532e90 ffff8805d4f67728 0000000000000282
> [ 880.960700] ffff880000000000 00ff880052cc8000 ffff880052cd7bd8 dffffc0000000000
> [ 880.960700] ffff880053218440 ffff8805d4f67778 ffff8805d4f67718 ffffffff935335b0
> [ 880.960700] Call Trace:
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? queue_stop_cpus_work (kernel/stop_machine.c:170)
> [ 880.960700] cpu_stopper_thread (kernel/stop_machine.c:474)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? cpu_stop_should_run (include/linux/list.h:189 kernel/stop_machine.c:443)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? ikconfig_read_current (kernel/stop_machine.c:437)
> [ 880.960700] ? trace_hardirqs_on_caller (kernel/locking/lockdep.c:2554 kernel/locking/lockdep.c:2601)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] smpboot_thread_fn (kernel/smpboot.c:161)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? schedule (./arch/x86/include/asm/bitops.h:311 (discriminator 1) kernel/sched/core.c:2824 (discriminator 1))
> [ 880.960700] ? __kthread_parkme (kernel/kthread.c:164)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] kthread (kernel/kthread.c:207)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ret_from_fork (arch/x86/kernel/entry_64.S:565)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] Code: eb 27 0f 1f 00 41 83 fe 03 75 0a 80 7d cf 00 0f 85 50 01 00 00 f0 41 ff 4f 24 74 71 41 83 fe 04 0f 84 d7 00 00 00 45 89 f0 f3 90 <4c> 89 e8 48 c1 e8 03 42 0f b6 04 20 84 c0 74 08 3c 03 0f 8e f5
> All code
> ========
> 0: eb 27 jmp 0x29
> 2: 0f 1f 00 nopl (%rax)
> 5: 41 83 fe 03 cmp $0x3,%r14d
> 9: 75 0a jne 0x15
> b: 80 7d cf 00 cmpb $0x0,-0x31(%rbp)
> f: 0f 85 50 01 00 00 jne 0x165
> 15: f0 41 ff 4f 24 lock decl 0x24(%r15)
> 1a: 74 71 je 0x8d
> 1c: 41 83 fe 04 cmp $0x4,%r14d
> 20: 0f 84 d7 00 00 00 je 0xfd
> 26: 45 89 f0 mov %r14d,%r8d
> 29: f3 90 pause
> 2b:* 4c 89 e8 mov %r13,%rax <-- trapping instruction
> 2e: 48 c1 e8 03 shr $0x3,%rax
> 32: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> 37: 84 c0 test %al,%al
> 39: 74 08 je 0x43
> 3b: 3c 03 cmp $0x3,%al
> 3d: 0f .byte 0xf
> 3e: 8e f5 mov %ebp,%?
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: 4c 89 e8 mov %r13,%rax
> 3: 48 c1 e8 03 shr $0x3,%rax
> 7: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> c: 84 c0 test %al,%al
> e: 74 08 je 0x18
> 10: 3c 03 cmp $0x3,%al
> 12: 0f .byte 0xf
> 13: 8e f5 mov %ebp,%?
> ...
> [ 880.960700] sending NMI to other CPUs:
> [ 881.290044] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 1.336 msecs
> [ 881.292380] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 2.083 msecs
> [ 881.297603] INFO: NMI handler (arch_trigger_all_cpu_backtrace_handler) took too long to run: 7.423 msecs
> [ 880.960700] NMI backtrace for cpu 0
> [ 880.960700] CPU: 0 PID: 11 Comm: migration/0 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff88000d6e3000 ti: ffff88000d6f0000 task.ti: ffff88000d6f0000
> [ 880.960700] RIP: multi_cpu_stop (kernel/stop_machine.c:192)
> [ 880.960700] RSP: 0000:ffff88000d6f7b78 EFLAGS: 00000246
> [ 880.960700] RAX: 0000000000000000 RBX: ffffed0007c21ee5 RCX: 1ffff10001adc8d5
> [ 880.960700] RDX: 0000000000000000 RSI: ffffffff9e370ae0 RDI: ffffffffa31716b4
> [ 880.960700] RBP: ffff88000d6f7bd8 R08: 0000000000000001 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: dffffc0000000000
> [ 880.960700] R13: ffff88003e10f738 R14: 0000000000000001 R15: ffff88003e10f718
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff880029000000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 0000000002d31fb0 CR3: 000000002082c000 CR4: 00000000000007b0
> [ 880.960700] Stack:
> [ 880.960700] ffffffffa15a9540 ffffffff93532e90 ffff88003e10f728 0000000000000282
> [ 880.960700] ffff880000000000 00ff88000d6e3000 ffff88000d6f7bd8 dffffc0000000000
> [ 880.960700] ffff880029018440 ffff88003e10f778 ffff88003e10f718 ffffffff935335b0
> [ 880.960700] Call Trace:
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? queue_stop_cpus_work (kernel/stop_machine.c:170)
> [ 880.960700] cpu_stopper_thread (kernel/stop_machine.c:474)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? cpu_stop_should_run (include/linux/list.h:189 kernel/stop_machine.c:443)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? ikconfig_read_current (kernel/stop_machine.c:437)
> [ 880.960700] ? trace_hardirqs_on_caller (kernel/locking/lockdep.c:2554 kernel/locking/lockdep.c:2601)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] smpboot_thread_fn (kernel/smpboot.c:161)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? schedule (./arch/x86/include/asm/bitops.h:311 (discriminator 1) kernel/sched/core.c:2824 (discriminator 1))
> [ 880.960700] ? __kthread_parkme (kernel/kthread.c:164)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] kthread (kernel/kthread.c:207)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ret_from_fork (arch/x86/kernel/entry_64.S:565)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] Code: eb 27 0f 1f 00 41 83 fe 03 75 0a 80 7d cf 00 0f 85 50 01 00 00 f0 41 ff 4f 24 74 71 41 83 fe 04 0f 84 d7 00 00 00 45 89 f0 f3 90 <4c> 89 e8 48 c1 e8 03 42 0f b6 04 20 84 c0 74 08 3c 03 0f 8e f5
> All code
> ========
> 0: eb 27 jmp 0x29
> 2: 0f 1f 00 nopl (%rax)
> 5: 41 83 fe 03 cmp $0x3,%r14d
> 9: 75 0a jne 0x15
> b: 80 7d cf 00 cmpb $0x0,-0x31(%rbp)
> f: 0f 85 50 01 00 00 jne 0x165
> 15: f0 41 ff 4f 24 lock decl 0x24(%r15)
> 1a: 74 71 je 0x8d
> 1c: 41 83 fe 04 cmp $0x4,%r14d
> 20: 0f 84 d7 00 00 00 je 0xfd
> 26: 45 89 f0 mov %r14d,%r8d
> 29: f3 90 pause
> 2b:* 4c 89 e8 mov %r13,%rax <-- trapping instruction
> 2e: 48 c1 e8 03 shr $0x3,%rax
> 32: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> 37: 84 c0 test %al,%al
> 39: 74 08 je 0x43
> 3b: 3c 03 cmp $0x3,%al
> 3d: 0f .byte 0xf
> 3e: 8e f5 mov %ebp,%?
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: 4c 89 e8 mov %r13,%rax
> 3: 48 c1 e8 03 shr $0x3,%rax
> 7: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> c: 84 c0 test %al,%al
> e: 74 08 je 0x18
> 10: 3c 03 cmp $0x3,%al
> 12: 0f .byte 0xf
> 13: 8e f5 mov %ebp,%?
> ...
> [ 880.960700] NMI backtrace for cpu 2
> [ 880.960700] CPU: 2 PID: 0 Comm: swapper/2 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff8802ccca0000 ti: ffff88000dee8000 task.ti: ffff88000dee8000
> [ 880.960700] RIP: native_safe_halt (./arch/x86/include/asm/irqflags.h:50)
> [ 880.960700] RSP: 0018:ffff88000deefd38 EFLAGS: 00000282
> [ 880.960700] RAX: dffffc0000000000 RBX: dffffc0000000000 RCX: 0000000000000000
> [ 880.960700] RDX: 1ffffffff4115884 RSI: 0000000000000001 RDI: ffffffffa08ac420
> [ 880.960700] RBP: ffff88000deefd38 R08: 0000000000000000 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa31716b4
> [ 880.960700] R13: ffff88000deeffd8 R14: 0000000000000000 R15: ffff88000deeffd8
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff88007d200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 00007f6baacd10f8 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffff88000deefd68 ffffffff93160774 dffffc0000000000 ffffffffa31716b4
> [ 880.960700] ffff88000deeffd8 0000000000000000 ffff88000deefd78 ffffffff931629ef
> [ 880.960700] ffff88000deefe68 ffffffff934018e0 ffff88000deefde8 ffffffff9dbf7ec5
> [ 880.960700] Call Trace:
> [ 880.960700] default_idle (./arch/x86/include/asm/paravirt.h:111 arch/x86/kernel/process.c:314)
> [ 880.960700] arch_cpu_idle (arch/x86/kernel/process.c:306)
> [ 880.960700] cpu_startup_entry (./arch/x86/include/asm/thread_info.h:162 include/linux/sched.h:2880 kernel/sched/idle.c:189 kernel/sched/idle.c:250 kernel/sched/idle.c:298)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? arch_cpu_idle_prepare (??:?)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? clockevents_register_device (kernel/time/clockevents.c:481)
> [ 880.960700] start_secondary (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] ? set_cpu_sibling_map (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] Code: 00 00 00 00 00 55 48 89 e5 fa 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb f4 <5d> c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 f4 5d c3 66 0f 1f 84
> All code
> ========
> 0: 00 00 add %al,(%rax)
> 2: 00 00 add %al,(%rax)
> 4: 00 55 48 add %dl,0x48(%rbp)
> 7: 89 e5 mov %esp,%ebp
> 9: fa cli
> a: 5d pop %rbp
> b: c3 retq
> c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 13: 00 00
> 15: 55 push %rbp
> 16: 48 89 e5 mov %rsp,%rbp
> 19: fb sti
> 1a: 5d pop %rbp
> 1b: c3 retq
> 1c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 23: 00 00
> 25: 55 push %rbp
> 26: 48 89 e5 mov %rsp,%rbp
> 29: fb sti
> 2a: f4 hlt
> 2b:* 5d pop %rbp <-- trapping instruction
> 2c: c3 retq
> 2d: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 34: 00
> 35: 55 push %rbp
> 36: 48 89 e5 mov %rsp,%rbp
> 39: f4 hlt
> 3a: 5d pop %rbp
> 3b: c3 retq
> 3c: 66 data16
> 3d: 0f .byte 0xf
> 3e: 1f (bad)
> 3f: 84 00 test %al,(%rax)
>
> Code starting with the faulting instruction
> ===========================================
> 0: 5d pop %rbp
> 1: c3 retq
> 2: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 9: 00
> a: 55 push %rbp
> b: 48 89 e5 mov %rsp,%rbp
> e: f4 hlt
> f: 5d pop %rbp
> 10: c3 retq
> 11: 66 data16
> 12: 0f .byte 0xf
> 13: 1f (bad)
> 14: 84 00 test %al,(%rax)
> [ 880.960700] NMI backtrace for cpu 3
> [ 880.960700] CPU: 3 PID: 32 Comm: migration/3 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff8800a6cc0000 ti: ffff8800a6cc8000 task.ti: ffff8800a6cc8000
> [ 880.960700] RIP: multi_cpu_stop (kernel/stop_machine.c:192)
> [ 880.960700] RSP: 0000:ffff8800a6ccfb78 EFLAGS: 00000246
> [ 880.960700] RAX: 0000000000000000 RBX: ffffed0067e09ee5 RCX: 1ffff10014d982d5
> [ 880.960700] RDX: 0000000000000000 RSI: ffffffff9e370ae0 RDI: ffffffffa31716b4
> [ 880.960700] RBP: ffff8800a6ccfbd8 R08: 0000000000000001 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: dffffc0000000000
> [ 880.960700] R13: ffff88033f04f738 R14: 0000000000000001 R15: ffff88033f04f718
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff8800a7200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 0000000001490fe8 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffffffffa15a9540 ffffffff93532e90 ffff88033f04f728 0000000000000282
> [ 880.960700] ffff880000000000 00ff8800a6cc0000 ffff8800a6ccfbd8 dffffc0000000000
> [ 880.960700] ffff8800a7218440 ffff88033f04f778 ffff88033f04f718 ffffffff935335b0
> [ 880.960700] Call Trace:
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? queue_stop_cpus_work (kernel/stop_machine.c:170)
> [ 880.960700] cpu_stopper_thread (kernel/stop_machine.c:474)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? cpu_stop_should_run (include/linux/list.h:189 kernel/stop_machine.c:443)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? ikconfig_read_current (kernel/stop_machine.c:437)
> [ 880.960700] ? trace_hardirqs_on_caller (kernel/locking/lockdep.c:2554 kernel/locking/lockdep.c:2601)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] smpboot_thread_fn (kernel/smpboot.c:161)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? schedule (./arch/x86/include/asm/bitops.h:311 (discriminator 1) kernel/sched/core.c:2824 (discriminator 1))
> [ 880.960700] ? __kthread_parkme (kernel/kthread.c:164)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] kthread (kernel/kthread.c:207)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ret_from_fork (arch/x86/kernel/entry_64.S:565)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] Code: eb 27 0f 1f 00 41 83 fe 03 75 0a 80 7d cf 00 0f 85 50 01 00 00 f0 41 ff 4f 24 74 71 41 83 fe 04 0f 84 d7 00 00 00 45 89 f0 f3 90 <4c> 89 e8 48 c1 e8 03 42 0f b6 04 20 84 c0 74 08 3c 03 0f 8e f5
> All code
> ========
> 0: eb 27 jmp 0x29
> 2: 0f 1f 00 nopl (%rax)
> 5: 41 83 fe 03 cmp $0x3,%r14d
> 9: 75 0a jne 0x15
> b: 80 7d cf 00 cmpb $0x0,-0x31(%rbp)
> f: 0f 85 50 01 00 00 jne 0x165
> 15: f0 41 ff 4f 24 lock decl 0x24(%r15)
> 1a: 74 71 je 0x8d
> 1c: 41 83 fe 04 cmp $0x4,%r14d
> 20: 0f 84 d7 00 00 00 je 0xfd
> 26: 45 89 f0 mov %r14d,%r8d
> 29: f3 90 pause
> 2b:* 4c 89 e8 mov %r13,%rax <-- trapping instruction
> 2e: 48 c1 e8 03 shr $0x3,%rax
> 32: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> 37: 84 c0 test %al,%al
> 39: 74 08 je 0x43
> 3b: 3c 03 cmp $0x3,%al
> 3d: 0f .byte 0xf
> 3e: 8e f5 mov %ebp,%?
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: 4c 89 e8 mov %r13,%rax
> 3: 48 c1 e8 03 shr $0x3,%rax
> 7: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> c: 84 c0 test %al,%al
> e: 74 08 je 0x18
> 10: 3c 03 cmp $0x3,%al
> 12: 0f .byte 0xf
> 13: 8e f5 mov %ebp,%?
> ...
> [ 880.960700] NMI backtrace for cpu 4
> [ 880.960700] CPU: 4 PID: 41 Comm: migration/4 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff8800cacb3000 ti: ffff8800cacc0000 task.ti: ffff8800cacc0000
> [ 880.960700] RIP: multi_cpu_stop (kernel/stop_machine.c:192)
> [ 880.960700] RSP: 0000:ffff8800cacc7b78 EFLAGS: 00000246
> [ 880.960700] RAX: 0000000000000000 RBX: ffffed006e615ee5 RCX: 1ffff100195968d5
> [ 880.960700] RDX: 0000000000000000 RSI: ffffffff9e370ae0 RDI: ffffffffa31716b4
> [ 880.960700] RBP: ffff8800cacc7bd8 R08: 0000000000000001 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: dffffc0000000000
> [ 880.960700] R13: ffff8803730af738 R14: 0000000000000001 R15: ffff8803730af718
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff8800cf200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 000000000189efd8 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffffffffa15a9540 ffffffff93532e90 ffff8803730af728 0000000000000282
> [ 880.960700] ffff880000000000 00ff8800cacb3000 ffff8800cacc7bd8 dffffc0000000000
> [ 880.960700] ffff8800cf218440 ffff8803730af778 ffff8803730af718 ffffffff935335b0
> [ 880.960700] Call Trace:
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? queue_stop_cpus_work (kernel/stop_machine.c:170)
> [ 880.960700] cpu_stopper_thread (kernel/stop_machine.c:474)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? cpu_stop_should_run (include/linux/list.h:189 kernel/stop_machine.c:443)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? ikconfig_read_current (kernel/stop_machine.c:437)
> [ 880.960700] ? trace_hardirqs_on_caller (kernel/locking/lockdep.c:2554 kernel/locking/lockdep.c:2601)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] smpboot_thread_fn (kernel/smpboot.c:161)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? schedule (./arch/x86/include/asm/bitops.h:311 (discriminator 1) kernel/sched/core.c:2824 (discriminator 1))
> [ 880.960700] ? __kthread_parkme (kernel/kthread.c:164)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] kthread (kernel/kthread.c:207)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ret_from_fork (arch/x86/kernel/entry_64.S:565)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] Code: eb 27 0f 1f 00 41 83 fe 03 75 0a 80 7d cf 00 0f 85 50 01 00 00 f0 41 ff 4f 24 74 71 41 83 fe 04 0f 84 d7 00 00 00 45 89 f0 f3 90 <4c> 89 e8 48 c1 e8 03 42 0f b6 04 20 84 c0 74 08 3c 03 0f 8e f5
> All code
> ========
> 0: eb 27 jmp 0x29
> 2: 0f 1f 00 nopl (%rax)
> 5: 41 83 fe 03 cmp $0x3,%r14d
> 9: 75 0a jne 0x15
> b: 80 7d cf 00 cmpb $0x0,-0x31(%rbp)
> f: 0f 85 50 01 00 00 jne 0x165
> 15: f0 41 ff 4f 24 lock decl 0x24(%r15)
> 1a: 74 71 je 0x8d
> 1c: 41 83 fe 04 cmp $0x4,%r14d
> 20: 0f 84 d7 00 00 00 je 0xfd
> 26: 45 89 f0 mov %r14d,%r8d
> 29: f3 90 pause
> 2b:* 4c 89 e8 mov %r13,%rax <-- trapping instruction
> 2e: 48 c1 e8 03 shr $0x3,%rax
> 32: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> 37: 84 c0 test %al,%al
> 39: 74 08 je 0x43
> 3b: 3c 03 cmp $0x3,%al
> 3d: 0f .byte 0xf
> 3e: 8e f5 mov %ebp,%?
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: 4c 89 e8 mov %r13,%rax
> 3: 48 c1 e8 03 shr $0x3,%rax
> 7: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> c: 84 c0 test %al,%al
> e: 74 08 je 0x18
> 10: 3c 03 cmp $0x3,%al
> 12: 0f .byte 0xf
> 13: 8e f5 mov %ebp,%?
> ...
> [ 880.960700] NMI backtrace for cpu 5
> [ 880.960700] CPU: 5 PID: 50 Comm: migration/5 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff880128c6b000 ti: ffff880128c88000 task.ti: ffff880128c88000
> [ 880.960700] RIP: multi_cpu_stop (kernel/stop_machine.c:192)
> [ 880.960700] RSP: 0000:ffff880128c8fb78 EFLAGS: 00000246
> [ 880.960700] RAX: 0000000000000000 RBX: ffffed008c0bbee5 RCX: 1ffff1002518d8d5
> [ 880.960700] RDX: 0000000000000000 RSI: ffffffff9e370ae0 RDI: ffffffffa31716b4
> [ 880.960700] RBP: ffff880128c8fbd8 R08: 0000000000000001 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: dffffc0000000000
> [ 880.960700] R13: ffff8804605df738 R14: 0000000000000001 R15: ffff8804605df718
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff880129000000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 0000000002bf8ff0 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffffffffa15a9540 ffffffff93532e90 ffff8804605df728 0000000000000282
> [ 880.960700] ffff880100000000 00ff880128c6b000 ffff880128c8fbd8 dffffc0000000000
> [ 880.960700] ffff880129018440 ffff8804605df778 ffff8804605df718 ffffffff935335b0
> [ 880.960700] Call Trace:
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? queue_stop_cpus_work (kernel/stop_machine.c:170)
> [ 880.960700] cpu_stopper_thread (kernel/stop_machine.c:474)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? cpu_stop_should_run (include/linux/list.h:189 kernel/stop_machine.c:443)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? ikconfig_read_current (kernel/stop_machine.c:437)
> [ 880.960700] ? trace_hardirqs_on_caller (kernel/locking/lockdep.c:2554 kernel/locking/lockdep.c:2601)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] smpboot_thread_fn (kernel/smpboot.c:161)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? schedule (./arch/x86/include/asm/bitops.h:311 (discriminator 1) kernel/sched/core.c:2824 (discriminator 1))
> [ 880.960700] ? __kthread_parkme (kernel/kthread.c:164)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] kthread (kernel/kthread.c:207)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ret_from_fork (arch/x86/kernel/entry_64.S:565)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] Code: eb 27 0f 1f 00 41 83 fe 03 75 0a 80 7d cf 00 0f 85 50 01 00 00 f0 41 ff 4f 24 74 71 41 83 fe 04 0f 84 d7 00 00 00 45 89 f0 f3 90 <4c> 89 e8 48 c1 e8 03 42 0f b6 04 20 84 c0 74 08 3c 03 0f 8e f5
> All code
> ========
> 0: eb 27 jmp 0x29
> 2: 0f 1f 00 nopl (%rax)
> 5: 41 83 fe 03 cmp $0x3,%r14d
> 9: 75 0a jne 0x15
> b: 80 7d cf 00 cmpb $0x0,-0x31(%rbp)
> f: 0f 85 50 01 00 00 jne 0x165
> 15: f0 41 ff 4f 24 lock decl 0x24(%r15)
> 1a: 74 71 je 0x8d
> 1c: 41 83 fe 04 cmp $0x4,%r14d
> 20: 0f 84 d7 00 00 00 je 0xfd
> 26: 45 89 f0 mov %r14d,%r8d
> 29: f3 90 pause
> 2b:* 4c 89 e8 mov %r13,%rax <-- trapping instruction
> 2e: 48 c1 e8 03 shr $0x3,%rax
> 32: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> 37: 84 c0 test %al,%al
> 39: 74 08 je 0x43
> 3b: 3c 03 cmp $0x3,%al
> 3d: 0f .byte 0xf
> 3e: 8e f5 mov %ebp,%?
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: 4c 89 e8 mov %r13,%rax
> 3: 48 c1 e8 03 shr $0x3,%rax
> 7: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> c: 84 c0 test %al,%al
> e: 74 08 je 0x18
> 10: 3c 03 cmp $0x3,%al
> 12: 0f .byte 0xf
> 13: 8e f5 mov %ebp,%?
> ...
> [ 880.960700] NMI backtrace for cpu 6
> [ 880.960700] CPU: 6 PID: 59 Comm: migration/6 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff880152cb3000 ti: ffff880152cc0000 task.ti: ffff880152cc0000
> [ 880.960700] RIP: multi_cpu_stop (kernel/stop_machine.c:192)
> [ 880.960700] RSP: 0000:ffff880152cc7b78 EFLAGS: 00000246
> [ 880.960700] RAX: 0000000000000000 RBX: ffffed0063d8cee5 RCX: 1ffff1002a5968d5
> [ 880.960700] RDX: 0000000000000000 RSI: ffffffff9e370ae0 RDI: ffffffffa31716b4
> [ 880.960700] RBP: ffff880152cc7bd8 R08: 0000000000000001 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: dffffc0000000000
> [ 880.960700] R13: ffff88031ec67738 R14: 0000000000000001 R15: ffff88031ec67718
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff880153200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 0000000002b54fa0 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffffffffa15a9540 ffffffff93532e90 ffff88031ec67728 0000000000000282
> [ 880.960700] ffff880100000000 00ff880152cb3000 ffff880152cc7bd8 dffffc0000000000
> [ 880.960700] ffff880153218440 ffff88031ec67778 ffff88031ec67718 ffffffff935335b0
> [ 880.960700] Call Trace:
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? queue_stop_cpus_work (kernel/stop_machine.c:170)
> [ 880.960700] cpu_stopper_thread (kernel/stop_machine.c:474)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? cpu_stop_should_run (include/linux/list.h:189 kernel/stop_machine.c:443)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? ikconfig_read_current (kernel/stop_machine.c:437)
> [ 880.960700] ? trace_hardirqs_on_caller (kernel/locking/lockdep.c:2554 kernel/locking/lockdep.c:2601)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] smpboot_thread_fn (kernel/smpboot.c:161)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? schedule (./arch/x86/include/asm/bitops.h:311 (discriminator 1) kernel/sched/core.c:2824 (discriminator 1))
> [ 880.960700] ? __kthread_parkme (kernel/kthread.c:164)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] kthread (kernel/kthread.c:207)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ret_from_fork (arch/x86/kernel/entry_64.S:565)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] Code: eb 27 0f 1f 00 41 83 fe 03 75 0a 80 7d cf 00 0f 85 50 01 00 00 f0 41 ff 4f 24 74 71 41 83 fe 04 0f 84 d7 00 00 00 45 89 f0 f3 90 <4c> 89 e8 48 c1 e8 03 42 0f b6 04 20 84 c0 74 08 3c 03 0f 8e f5
> All code
> ========
> 0: eb 27 jmp 0x29
> 2: 0f 1f 00 nopl (%rax)
> 5: 41 83 fe 03 cmp $0x3,%r14d
> 9: 75 0a jne 0x15
> b: 80 7d cf 00 cmpb $0x0,-0x31(%rbp)
> f: 0f 85 50 01 00 00 jne 0x165
> 15: f0 41 ff 4f 24 lock decl 0x24(%r15)
> 1a: 74 71 je 0x8d
> 1c: 41 83 fe 04 cmp $0x4,%r14d
> 20: 0f 84 d7 00 00 00 je 0xfd
> 26: 45 89 f0 mov %r14d,%r8d
> 29: f3 90 pause
> 2b:* 4c 89 e8 mov %r13,%rax <-- trapping instruction
> 2e: 48 c1 e8 03 shr $0x3,%rax
> 32: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> 37: 84 c0 test %al,%al
> 39: 74 08 je 0x43
> 3b: 3c 03 cmp $0x3,%al
> 3d: 0f .byte 0xf
> 3e: 8e f5 mov %ebp,%?
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: 4c 89 e8 mov %r13,%rax
> 3: 48 c1 e8 03 shr $0x3,%rax
> 7: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> c: 84 c0 test %al,%al
> e: 74 08 je 0x18
> 10: 3c 03 cmp $0x3,%al
> 12: 0f .byte 0xf
> 13: 8e f5 mov %ebp,%?
> ...
> [ 880.960700] NMI backtrace for cpu 7
> [ 880.960700] CPU: 7 PID: 0 Comm: swapper/7 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff8808dd0b0000 ti: ffff88000e2d0000 task.ti: ffff88000e2d0000
> [ 880.960700] RIP: native_safe_halt (./arch/x86/include/asm/irqflags.h:50)
> [ 880.960700] RSP: 0000:ffff88000e2d7d38 EFLAGS: 00000282
> [ 880.960700] RAX: dffffc0000000000 RBX: dffffc0000000000 RCX: 0000000000000000
> [ 880.960700] RDX: 1ffffffff4115884 RSI: 0000000000000001 RDI: ffffffffa08ac420
> [ 880.960700] RBP: ffff88000e2d7d38 R08: 0000000000000000 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa31716b4
> [ 880.960700] R13: ffff88000e2d7fd8 R14: 0000000000000000 R15: ffff88000e2d7fd8
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff88017d200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 0000000000000000 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffff88000e2d7d68 ffffffff93160774 dffffc0000000000 ffffffffa31716b4
> [ 880.960700] ffff88000e2d7fd8 0000000000000000 ffff88000e2d7d78 ffffffff931629ef
> [ 880.960700] ffff88000e2d7e68 ffffffff934018e0 0000000000000000 0000000000000000
> [ 880.960700] Call Trace:
> [ 880.960700] default_idle (./arch/x86/include/asm/paravirt.h:111 arch/x86/kernel/process.c:314)
> [ 880.960700] arch_cpu_idle (arch/x86/kernel/process.c:306)
> [ 880.960700] cpu_startup_entry (./arch/x86/include/asm/thread_info.h:162 include/linux/sched.h:2880 kernel/sched/idle.c:189 kernel/sched/idle.c:250 kernel/sched/idle.c:298)
> [ 880.960700] ? arch_cpu_idle_prepare (??:?)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? clockevents_register_device (kernel/time/clockevents.c:481)
> [ 880.960700] start_secondary (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] ? set_cpu_sibling_map (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] Code: 00 00 00 00 00 55 48 89 e5 fa 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb f4 <5d> c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 f4 5d c3 66 0f 1f 84
> All code
> ========
> 0: 00 00 add %al,(%rax)
> 2: 00 00 add %al,(%rax)
> 4: 00 55 48 add %dl,0x48(%rbp)
> 7: 89 e5 mov %esp,%ebp
> 9: fa cli
> a: 5d pop %rbp
> b: c3 retq
> c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 13: 00 00
> 15: 55 push %rbp
> 16: 48 89 e5 mov %rsp,%rbp
> 19: fb sti
> 1a: 5d pop %rbp
> 1b: c3 retq
> 1c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 23: 00 00
> 25: 55 push %rbp
> 26: 48 89 e5 mov %rsp,%rbp
> 29: fb sti
> 2a: f4 hlt
> 2b:* 5d pop %rbp <-- trapping instruction
> 2c: c3 retq
> 2d: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 34: 00
> 35: 55 push %rbp
> 36: 48 89 e5 mov %rsp,%rbp
> 39: f4 hlt
> 3a: 5d pop %rbp
> 3b: c3 retq
> 3c: 66 data16
> 3d: 0f .byte 0xf
> 3e: 1f (bad)
> 3f: 84 00 test %al,(%rax)
>
> Code starting with the faulting instruction
> ===========================================
> 0: 5d pop %rbp
> 1: c3 retq
> 2: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 9: 00
> a: 55 push %rbp
> b: 48 89 e5 mov %rsp,%rbp
> e: f4 hlt
> f: 5d pop %rbp
> 10: c3 retq
> 11: 66 data16
> 12: 0f .byte 0xf
> 13: 1f (bad)
> 14: 84 00 test %al,(%rax)
> [ 880.960700] NMI backtrace for cpu 8
> [ 880.960700] CPU: 8 PID: 77 Comm: migration/8 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff8801a6cc0000 ti: ffff8801a6cc8000 task.ti: ffff8801a6cc8000
> [ 880.960700] RIP: multi_cpu_stop (kernel/stop_machine.c:192)
> [ 880.960700] RSP: 0000:ffff8801a6ccfb78 EFLAGS: 00000246
> [ 880.960700] RAX: 0000000000000000 RBX: ffffed003abaaee5 RCX: 1ffff10034d982d5
> [ 880.960700] RDX: 0000000000000000 RSI: ffffffff9e370ae0 RDI: ffffffffa31716b4
> [ 880.960700] RBP: ffff8801a6ccfbd8 R08: 0000000000000001 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: dffffc0000000000
> [ 880.960700] R13: ffff8801d5d57738 R14: 0000000000000001 R15: ffff8801d5d57718
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff8801a7200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 0000000000e69fc8 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffffffffa15a9540 ffffffff93532e90 ffff8801d5d57728 0000000000000282
> [ 880.960700] ffff880100000000 00ff8801a6cc0000 ffff8801a6ccfbd8 dffffc0000000000
> [ 880.960700] ffff8801a7218440 ffff8801d5d57778 ffff8801d5d57718 ffffffff935335b0
> [ 880.960700] Call Trace:
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? queue_stop_cpus_work (kernel/stop_machine.c:170)
> [ 880.960700] cpu_stopper_thread (kernel/stop_machine.c:474)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? cpu_stop_should_run (include/linux/list.h:189 kernel/stop_machine.c:443)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? ikconfig_read_current (kernel/stop_machine.c:437)
> [ 880.960700] ? trace_hardirqs_on_caller (kernel/locking/lockdep.c:2554 kernel/locking/lockdep.c:2601)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] smpboot_thread_fn (kernel/smpboot.c:161)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? schedule (./arch/x86/include/asm/bitops.h:311 (discriminator 1) kernel/sched/core.c:2824 (discriminator 1))
> [ 880.960700] ? __kthread_parkme (kernel/kthread.c:164)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] kthread (kernel/kthread.c:207)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ret_from_fork (arch/x86/kernel/entry_64.S:565)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] Code: eb 27 0f 1f 00 41 83 fe 03 75 0a 80 7d cf 00 0f 85 50 01 00 00 f0 41 ff 4f 24 74 71 41 83 fe 04 0f 84 d7 00 00 00 45 89 f0 f3 90 <4c> 89 e8 48 c1 e8 03 42 0f b6 04 20 84 c0 74 08 3c 03 0f 8e f5
> All code
> ========
> 0: eb 27 jmp 0x29
> 2: 0f 1f 00 nopl (%rax)
> 5: 41 83 fe 03 cmp $0x3,%r14d
> 9: 75 0a jne 0x15
> b: 80 7d cf 00 cmpb $0x0,-0x31(%rbp)
> f: 0f 85 50 01 00 00 jne 0x165
> 15: f0 41 ff 4f 24 lock decl 0x24(%r15)
> 1a: 74 71 je 0x8d
> 1c: 41 83 fe 04 cmp $0x4,%r14d
> 20: 0f 84 d7 00 00 00 je 0xfd
> 26: 45 89 f0 mov %r14d,%r8d
> 29: f3 90 pause
> 2b:* 4c 89 e8 mov %r13,%rax <-- trapping instruction
> 2e: 48 c1 e8 03 shr $0x3,%rax
> 32: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> 37: 84 c0 test %al,%al
> 39: 74 08 je 0x43
> 3b: 3c 03 cmp $0x3,%al
> 3d: 0f .byte 0xf
> 3e: 8e f5 mov %ebp,%?
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: 4c 89 e8 mov %r13,%rax
> 3: 48 c1 e8 03 shr $0x3,%rax
> 7: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> c: 84 c0 test %al,%al
> e: 74 08 je 0x18
> 10: 3c 03 cmp $0x3,%al
> 12: 0f .byte 0xf
> 13: 8e f5 mov %ebp,%?
> ...
> [ 880.960700] NMI backtrace for cpu 9
> [ 880.960700] CPU: 9 PID: 86 Comm: migration/9 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff8801d0c88000 ti: ffff8801d0c90000 task.ti: ffff8801d0c90000
> [ 880.960700] RIP: multi_cpu_stop (kernel/stop_machine.c:192)
> [ 880.960700] RSP: 0000:ffff8801d0c97b78 EFLAGS: 00000246
> [ 880.960700] RAX: 0000000000000000 RBX: ffffed007245bee5 RCX: 1ffff1003a1912d5
> [ 880.960700] RDX: 0000000000000000 RSI: ffffffff9e370ae0 RDI: ffffffffa31716b4
> [ 880.960700] RBP: ffff8801d0c97bd8 R08: 0000000000000001 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: dffffc0000000000
> [ 880.960700] R13: ffff8803922df738 R14: 0000000000000001 R15: ffff8803922df718
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff8801d1000000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 00000000029a8fb0 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffffffffa15a9540 ffffffff93532e90 ffff8803922df728 0000000000000282
> [ 880.960700] ffff880100000000 00ff8801d0c88000 ffff8801d0c97bd8 dffffc0000000000
> [ 880.960700] ffff8801d1018440 ffff8803922df778 ffff8803922df718 ffffffff935335b0
> [ 880.960700] Call Trace:
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? queue_stop_cpus_work (kernel/stop_machine.c:170)
> [ 880.960700] cpu_stopper_thread (kernel/stop_machine.c:474)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? cpu_stop_should_run (include/linux/list.h:189 kernel/stop_machine.c:443)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? ikconfig_read_current (kernel/stop_machine.c:437)
> [ 880.960700] ? trace_hardirqs_on_caller (kernel/locking/lockdep.c:2554 kernel/locking/lockdep.c:2601)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] smpboot_thread_fn (kernel/smpboot.c:161)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? schedule (./arch/x86/include/asm/bitops.h:311 (discriminator 1) kernel/sched/core.c:2824 (discriminator 1))
> [ 880.960700] ? __kthread_parkme (kernel/kthread.c:164)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] kthread (kernel/kthread.c:207)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ret_from_fork (arch/x86/kernel/entry_64.S:565)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] Code: eb 27 0f 1f 00 41 83 fe 03 75 0a 80 7d cf 00 0f 85 50 01 00 00 f0 41 ff 4f 24 74 71 41 83 fe 04 0f 84 d7 00 00 00 45 89 f0 f3 90 <4c> 89 e8 48 c1 e8 03 42 0f b6 04 20 84 c0 74 08 3c 03 0f 8e f5
> All code
> ========
> 0: eb 27 jmp 0x29
> 2: 0f 1f 00 nopl (%rax)
> 5: 41 83 fe 03 cmp $0x3,%r14d
> 9: 75 0a jne 0x15
> b: 80 7d cf 00 cmpb $0x0,-0x31(%rbp)
> f: 0f 85 50 01 00 00 jne 0x165
> 15: f0 41 ff 4f 24 lock decl 0x24(%r15)
> 1a: 74 71 je 0x8d
> 1c: 41 83 fe 04 cmp $0x4,%r14d
> 20: 0f 84 d7 00 00 00 je 0xfd
> 26: 45 89 f0 mov %r14d,%r8d
> 29: f3 90 pause
> 2b:* 4c 89 e8 mov %r13,%rax <-- trapping instruction
> 2e: 48 c1 e8 03 shr $0x3,%rax
> 32: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> 37: 84 c0 test %al,%al
> 39: 74 08 je 0x43
> 3b: 3c 03 cmp $0x3,%al
> 3d: 0f .byte 0xf
> 3e: 8e f5 mov %ebp,%?
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: 4c 89 e8 mov %r13,%rax
> 3: 48 c1 e8 03 shr $0x3,%rax
> 7: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> c: 84 c0 test %al,%al
> e: 74 08 je 0x18
> 10: 3c 03 cmp $0x3,%al
> 12: 0f .byte 0xf
> 13: 8e f5 mov %ebp,%?
> ...
> [ 880.960700] NMI backtrace for cpu 10
> [ 880.960700] CPU: 10 PID: 0 Comm: swapper/10 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff880128c68000 ti: ffff88000e6c0000 task.ti: ffff88000e6c0000
> [ 880.960700] RIP: native_safe_halt (./arch/x86/include/asm/irqflags.h:50)
> [ 880.960700] RSP: 0018:ffff88000e6c7d38 EFLAGS: 00000282
> [ 880.960700] RAX: dffffc0000000000 RBX: dffffc0000000000 RCX: 0000000000000000
> [ 880.960700] RDX: 1ffffffff4115884 RSI: 0000000000000001 RDI: ffffffffa08ac420
> [ 880.960700] RBP: ffff88000e6c7d38 R08: 0000000000000000 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa31716b4
> [ 880.960700] R13: ffff88000e6c7fd8 R14: 0000000000000000 R15: ffff88000e6c7fd8
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff8801fb200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 0000000000636668 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffff88000e6c7d68 ffffffff93160774 dffffc0000000000 ffffffffa31716b4
> [ 880.960700] ffff88000e6c7fd8 0000000000000000 ffff88000e6c7d78 ffffffff931629ef
> [ 880.960700] ffff88000e6c7e68 ffffffff934018e0 0000000000000000 0000000000000000
> [ 880.960700] Call Trace:
> [ 880.960700] default_idle (./arch/x86/include/asm/paravirt.h:111 arch/x86/kernel/process.c:314)
> [ 880.960700] arch_cpu_idle (arch/x86/kernel/process.c:306)
> [ 880.960700] cpu_startup_entry (./arch/x86/include/asm/thread_info.h:162 include/linux/sched.h:2880 kernel/sched/idle.c:189 kernel/sched/idle.c:250 kernel/sched/idle.c:298)
> [ 880.960700] ? arch_cpu_idle_prepare (??:?)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? clockevents_register_device (kernel/time/clockevents.c:481)
> [ 880.960700] start_secondary (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] ? set_cpu_sibling_map (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] Code: 00 00 00 00 00 55 48 89 e5 fa 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb f4 <5d> c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 f4 5d c3 66 0f 1f 84
> All code
> ========
> 0: 00 00 add %al,(%rax)
> 2: 00 00 add %al,(%rax)
> 4: 00 55 48 add %dl,0x48(%rbp)
> 7: 89 e5 mov %esp,%ebp
> 9: fa cli
> a: 5d pop %rbp
> b: c3 retq
> c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 13: 00 00
> 15: 55 push %rbp
> 16: 48 89 e5 mov %rsp,%rbp
> 19: fb sti
> 1a: 5d pop %rbp
> 1b: c3 retq
> 1c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 23: 00 00
> 25: 55 push %rbp
> 26: 48 89 e5 mov %rsp,%rbp
> 29: fb sti
> 2a: f4 hlt
> 2b:* 5d pop %rbp <-- trapping instruction
> 2c: c3 retq
> 2d: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 34: 00
> 35: 55 push %rbp
> 36: 48 89 e5 mov %rsp,%rbp
> 39: f4 hlt
> 3a: 5d pop %rbp
> 3b: c3 retq
> 3c: 66 data16
> 3d: 0f .byte 0xf
> 3e: 1f (bad)
> 3f: 84 00 test %al,(%rax)
>
> Code starting with the faulting instruction
> ===========================================
> 0: 5d pop %rbp
> 1: c3 retq
> 2: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 9: 00
> a: 55 push %rbp
> b: 48 89 e5 mov %rsp,%rbp
> e: f4 hlt
> f: 5d pop %rbp
> 10: c3 retq
> 11: 66 data16
> 12: 0f .byte 0xf
> 13: 1f (bad)
> 14: 84 00 test %al,(%rax)
> [ 880.960700] NMI backtrace for cpu 11
> [ 880.960700] CPU: 11 PID: 0 Comm: swapper/11 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff8801a6ca8000 ti: ffff88000e6c8000 task.ti: ffff88000e6c8000
> [ 880.960700] RIP: native_safe_halt (./arch/x86/include/asm/irqflags.h:50)
> [ 880.960700] RSP: 0018:ffff88000e6cfd38 EFLAGS: 00000282
> [ 880.960700] RAX: dffffc0000000000 RBX: dffffc0000000000 RCX: 0000000000000000
> [ 880.960700] RDX: 1ffffffff4115884 RSI: 0000000000000001 RDI: ffffffffa08ac420
> [ 880.960700] RBP: ffff88000e6cfd38 R08: 0000000000000000 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa31716b4
> [ 880.960700] R13: ffff88000e6cffd8 R14: 0000000000000000 R15: ffff88000e6cffd8
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff880225200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 00000000030db000 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffff88000e6cfd68 ffffffff93160774 dffffc0000000000 ffffffffa31716b4
> [ 880.960700] ffff88000e6cffd8 0000000000000000 ffff88000e6cfd78 ffffffff931629ef
> [ 880.960700] ffff88000e6cfe68 ffffffff934018e0 ffff88000e6cfde8 ffffffff9dbf7ec5
> [ 880.960700] Call Trace:
> [ 880.960700] default_idle (./arch/x86/include/asm/paravirt.h:111 arch/x86/kernel/process.c:314)
> [ 880.960700] arch_cpu_idle (arch/x86/kernel/process.c:306)
> [ 880.960700] cpu_startup_entry (./arch/x86/include/asm/thread_info.h:162 include/linux/sched.h:2880 kernel/sched/idle.c:189 kernel/sched/idle.c:250 kernel/sched/idle.c:298)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? arch_cpu_idle_prepare (??:?)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? clockevents_register_device (kernel/time/clockevents.c:481)
> [ 880.960700] start_secondary (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] ? set_cpu_sibling_map (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] Code: 00 00 00 00 00 55 48 89 e5 fa 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb f4 <5d> c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 f4 5d c3 66 0f 1f 84
> All code
> ========
> 0: 00 00 add %al,(%rax)
> 2: 00 00 add %al,(%rax)
> 4: 00 55 48 add %dl,0x48(%rbp)
> 7: 89 e5 mov %esp,%ebp
> 9: fa cli
> a: 5d pop %rbp
> b: c3 retq
> c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 13: 00 00
> 15: 55 push %rbp
> 16: 48 89 e5 mov %rsp,%rbp
> 19: fb sti
> 1a: 5d pop %rbp
> 1b: c3 retq
> 1c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 23: 00 00
> 25: 55 push %rbp
> 26: 48 89 e5 mov %rsp,%rbp
> 29: fb sti
> 2a: f4 hlt
> 2b:* 5d pop %rbp <-- trapping instruction
> 2c: c3 retq
> 2d: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 34: 00
> 35: 55 push %rbp
> 36: 48 89 e5 mov %rsp,%rbp
> 39: f4 hlt
> 3a: 5d pop %rbp
> 3b: c3 retq
> 3c: 66 data16
> 3d: 0f .byte 0xf
> 3e: 1f (bad)
> 3f: 84 00 test %al,(%rax)
>
> Code starting with the faulting instruction
> ===========================================
> 0: 5d pop %rbp
> 1: c3 retq
> 2: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 9: 00
> a: 55 push %rbp
> b: 48 89 e5 mov %rsp,%rbp
> e: f4 hlt
> f: 5d pop %rbp
> 10: c3 retq
> 11: 66 data16
> 12: 0f .byte 0xf
> 13: 1f (bad)
> 14: 84 00 test %al,(%rax)
> [ 880.960700] NMI backtrace for cpu 12
> [ 880.960700] CPU: 12 PID: 113 Comm: migration/12 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff88024ecd0000 ti: ffff88024ecd8000 task.ti: ffff88024ecd8000
> [ 880.960700] RIP: multi_cpu_stop (kernel/stop_machine.c:192)
> [ 880.960700] RSP: 0000:ffff88024ecdfb78 EFLAGS: 00000246
> [ 880.960700] RAX: 0000000000000000 RBX: ffffed0073a32ee5 RCX: 1ffff10049d9a2d5
> [ 880.960700] RDX: 0000000000000000 RSI: ffffffff9e370ae0 RDI: ffffffffa31716b4
> [ 880.960700] RBP: ffff88024ecdfbd8 R08: 0000000000000001 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: dffffc0000000000
> [ 880.960700] R13: ffff88039d197738 R14: 0000000000000001 R15: ffff88039d197718
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff88024f200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 0000000002648fe0 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffffffffa15a9540 ffffffff93532e90 ffff88039d197728 0000000000000282
> [ 880.960700] ffff880200000000 00ff88024ecd0000 ffff88024ecdfbd8 dffffc0000000000
> [ 880.960700] ffff88024f218440 ffff88039d197778 ffff88039d197718 ffffffff935335b0
> [ 880.960700] Call Trace:
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? queue_stop_cpus_work (kernel/stop_machine.c:170)
> [ 880.960700] cpu_stopper_thread (kernel/stop_machine.c:474)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? cpu_stop_should_run (include/linux/list.h:189 kernel/stop_machine.c:443)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? ikconfig_read_current (kernel/stop_machine.c:437)
> [ 880.960700] ? trace_hardirqs_on_caller (kernel/locking/lockdep.c:2554 kernel/locking/lockdep.c:2601)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] smpboot_thread_fn (kernel/smpboot.c:161)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? schedule (./arch/x86/include/asm/bitops.h:311 (discriminator 1) kernel/sched/core.c:2824 (discriminator 1))
> [ 880.960700] ? __kthread_parkme (kernel/kthread.c:164)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] kthread (kernel/kthread.c:207)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ret_from_fork (arch/x86/kernel/entry_64.S:565)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] Code: eb 27 0f 1f 00 41 83 fe 03 75 0a 80 7d cf 00 0f 85 50 01 00 00 f0 41 ff 4f 24 74 71 41 83 fe 04 0f 84 d7 00 00 00 45 89 f0 f3 90 <4c> 89 e8 48 c1 e8 03 42 0f b6 04 20 84 c0 74 08 3c 03 0f 8e f5
> All code
> ========
> 0: eb 27 jmp 0x29
> 2: 0f 1f 00 nopl (%rax)
> 5: 41 83 fe 03 cmp $0x3,%r14d
> 9: 75 0a jne 0x15
> b: 80 7d cf 00 cmpb $0x0,-0x31(%rbp)
> f: 0f 85 50 01 00 00 jne 0x165
> 15: f0 41 ff 4f 24 lock decl 0x24(%r15)
> 1a: 74 71 je 0x8d
> 1c: 41 83 fe 04 cmp $0x4,%r14d
> 20: 0f 84 d7 00 00 00 je 0xfd
> 26: 45 89 f0 mov %r14d,%r8d
> 29: f3 90 pause
> 2b:* 4c 89 e8 mov %r13,%rax <-- trapping instruction
> 2e: 48 c1 e8 03 shr $0x3,%rax
> 32: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> 37: 84 c0 test %al,%al
> 39: 74 08 je 0x43
> 3b: 3c 03 cmp $0x3,%al
> 3d: 0f .byte 0xf
> 3e: 8e f5 mov %ebp,%?
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: 4c 89 e8 mov %r13,%rax
> 3: 48 c1 e8 03 shr $0x3,%rax
> 7: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> c: 84 c0 test %al,%al
> e: 74 08 je 0x18
> 10: 3c 03 cmp $0x3,%al
> 12: 0f .byte 0xf
> 13: 8e f5 mov %ebp,%?
> ...
> [ 880.960700] NMI backtrace for cpu 13
> [ 880.960700] CPU: 13 PID: 0 Comm: swapper/13 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff8802f6cc0000 ti: ffff88000e6d8000 task.ti: ffff88000e6d8000
> [ 880.960700] RIP: native_safe_halt (./arch/x86/include/asm/irqflags.h:50)
> [ 880.960700] RSP: 0018:ffff88000e6dfd38 EFLAGS: 00000282
> [ 880.960700] RAX: dffffc0000000000 RBX: dffffc0000000000 RCX: 0000000000000000
> [ 880.960700] RDX: 1ffffffff4115884 RSI: 0000000000000001 RDI: ffffffffa08ac420
> [ 880.960700] RBP: ffff88000e6dfd38 R08: 0000000000000000 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa31716b4
> [ 880.960700] R13: ffff88000e6dffd8 R14: 0000000000000000 R15: ffff88000e6dffd8
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff880279000000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 000000000063a69c CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffff88000e6dfd68 ffffffff93160774 dffffc0000000000 ffffffffa31716b4
> [ 880.960700] ffff88000e6dffd8 0000000000000000 ffff88000e6dfd78 ffffffff931629ef
> [ 880.960700] ffff88000e6dfe68 ffffffff934018e0 ffff88000e6dfde8 ffffffff9dbf7ec5
> [ 880.960700] Call Trace:
> [ 880.960700] default_idle (./arch/x86/include/asm/paravirt.h:111 arch/x86/kernel/process.c:314)
> [ 880.960700] arch_cpu_idle (arch/x86/kernel/process.c:306)
> [ 880.960700] cpu_startup_entry (./arch/x86/include/asm/thread_info.h:162 include/linux/sched.h:2880 kernel/sched/idle.c:189 kernel/sched/idle.c:250 kernel/sched/idle.c:298)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? arch_cpu_idle_prepare (??:?)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? clockevents_register_device (kernel/time/clockevents.c:481)
> [ 880.960700] start_secondary (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] ? set_cpu_sibling_map (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] Code: 00 00 00 00 00 55 48 89 e5 fa 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb f4 <5d> c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 f4 5d c3 66 0f 1f 84
> All code
> ========
> 0: 00 00 add %al,(%rax)
> 2: 00 00 add %al,(%rax)
> 4: 00 55 48 add %dl,0x48(%rbp)
> 7: 89 e5 mov %esp,%ebp
> 9: fa cli
> a: 5d pop %rbp
> b: c3 retq
> c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 13: 00 00
> 15: 55 push %rbp
> 16: 48 89 e5 mov %rsp,%rbp
> 19: fb sti
> 1a: 5d pop %rbp
> 1b: c3 retq
> 1c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 23: 00 00
> 25: 55 push %rbp
> 26: 48 89 e5 mov %rsp,%rbp
> 29: fb sti
> 2a: f4 hlt
> 2b:* 5d pop %rbp <-- trapping instruction
> 2c: c3 retq
> 2d: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 34: 00
> 35: 55 push %rbp
> 36: 48 89 e5 mov %rsp,%rbp
> 39: f4 hlt
> 3a: 5d pop %rbp
> 3b: c3 retq
> 3c: 66 data16
> 3d: 0f .byte 0xf
> 3e: 1f (bad)
> 3f: 84 00 test %al,(%rax)
>
> Code starting with the faulting instruction
> ===========================================
> 0: 5d pop %rbp
> 1: c3 retq
> 2: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 9: 00
> a: 55 push %rbp
> b: 48 89 e5 mov %rsp,%rbp
> e: f4 hlt
> f: 5d pop %rbp
> 10: c3 retq
> 11: 66 data16
> 12: 0f .byte 0xf
> 13: 1f (bad)
> 14: 84 00 test %al,(%rax)
> [ 880.960700] NMI backtrace for cpu 14
> [ 880.960700] CPU: 14 PID: 131 Comm: migration/14 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff8802a2cdb000 ti: ffff8802a2ce8000 task.ti: ffff8802a2ce8000
> [ 880.960700] RIP: multi_cpu_stop (kernel/stop_machine.c:192)
> [ 880.960700] RSP: 0000:ffff8802a2cefb78 EFLAGS: 00000246
> [ 880.960700] RAX: 0000000000000000 RBX: ffffed0059614ee5 RCX: 1ffff1005459b8d5
> [ 880.960700] RDX: 0000000000000000 RSI: ffffffff9e370ae0 RDI: ffffffffa31716b4
> [ 880.960700] RBP: ffff8802a2cefbd8 R08: 0000000000000001 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: dffffc0000000000
> [ 880.960700] R13: ffff8802cb0a7738 R14: 0000000000000001 R15: ffff8802cb0a7718
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff8802a3200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 0000000002cc8ff8 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffffffffa15a9540 ffffffff93532e90 ffff8802cb0a7728 0000000000000282
> [ 880.960700] ffff880200000000 00ff8802a2cdb000 ffff8802a2cefbd8 dffffc0000000000
> [ 880.960700] ffff8802a3218440 ffff8802cb0a7778 ffff8802cb0a7718 ffffffff935335b0
> [ 880.960700] Call Trace:
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? queue_stop_cpus_work (kernel/stop_machine.c:170)
> [ 880.960700] cpu_stopper_thread (kernel/stop_machine.c:474)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? cpu_stop_should_run (include/linux/list.h:189 kernel/stop_machine.c:443)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? ikconfig_read_current (kernel/stop_machine.c:437)
> [ 880.960700] ? trace_hardirqs_on_caller (kernel/locking/lockdep.c:2554 kernel/locking/lockdep.c:2601)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] smpboot_thread_fn (kernel/smpboot.c:161)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? schedule (./arch/x86/include/asm/bitops.h:311 (discriminator 1) kernel/sched/core.c:2824 (discriminator 1))
> [ 880.960700] ? __kthread_parkme (kernel/kthread.c:164)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] kthread (kernel/kthread.c:207)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ret_from_fork (arch/x86/kernel/entry_64.S:565)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] Code: eb 27 0f 1f 00 41 83 fe 03 75 0a 80 7d cf 00 0f 85 50 01 00 00 f0 41 ff 4f 24 74 71 41 83 fe 04 0f 84 d7 00 00 00 45 89 f0 f3 90 <4c> 89 e8 48 c1 e8 03 42 0f b6 04 20 84 c0 74 08 3c 03 0f 8e f5
> All code
> ========
> 0: eb 27 jmp 0x29
> 2: 0f 1f 00 nopl (%rax)
> 5: 41 83 fe 03 cmp $0x3,%r14d
> 9: 75 0a jne 0x15
> b: 80 7d cf 00 cmpb $0x0,-0x31(%rbp)
> f: 0f 85 50 01 00 00 jne 0x165
> 15: f0 41 ff 4f 24 lock decl 0x24(%r15)
> 1a: 74 71 je 0x8d
> 1c: 41 83 fe 04 cmp $0x4,%r14d
> 20: 0f 84 d7 00 00 00 je 0xfd
> 26: 45 89 f0 mov %r14d,%r8d
> 29: f3 90 pause
> 2b:* 4c 89 e8 mov %r13,%rax <-- trapping instruction
> 2e: 48 c1 e8 03 shr $0x3,%rax
> 32: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> 37: 84 c0 test %al,%al
> 39: 74 08 je 0x43
> 3b: 3c 03 cmp $0x3,%al
> 3d: 0f .byte 0xf
> 3e: 8e f5 mov %ebp,%?
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: 4c 89 e8 mov %r13,%rax
> 3: 48 c1 e8 03 shr $0x3,%rax
> 7: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> c: 84 c0 test %al,%al
> e: 74 08 je 0x18
> 10: 3c 03 cmp $0x3,%al
> 12: 0f .byte 0xf
> 13: 8e f5 mov %ebp,%?
> ...
> [ 880.960700] NMI backtrace for cpu 15
> [ 880.960700] CPU: 15 PID: 140 Comm: migration/15 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff8802ccca3000 ti: ffff8802cccd0000 task.ti: ffff8802cccd0000
> [ 880.960700] RIP: multi_cpu_stop (kernel/stop_machine.c:192)
> [ 880.960700] RSP: 0000:ffff8802cccd7b78 EFLAGS: 00000246
> [ 880.960700] RAX: 0000000000000000 RBX: ffffed0023834ee5 RCX: 1ffff100599948d5
> [ 880.960700] RDX: 0000000000000000 RSI: ffffffff9e370ae0 RDI: ffffffffa31716b4
> [ 880.960700] RBP: ffff8802cccd7bd8 R08: 0000000000000001 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: dffffc0000000000
> [ 880.960700] R13: ffff88011c1a7738 R14: 0000000000000001 R15: ffff88011c1a7718
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff8802cd200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 0000000001ad0a08 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffffffffa15a9540 ffffffff93532e90 ffff88011c1a7728 0000000000000282
> [ 880.960700] ffff880200000000 00ff8802ccca3000 ffff8802cccd7bd8 dffffc0000000000
> [ 880.960700] ffff8802cd218440 ffff88011c1a7778 ffff88011c1a7718 ffffffff935335b0
> [ 880.960700] Call Trace:
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? queue_stop_cpus_work (kernel/stop_machine.c:170)
> [ 880.960700] cpu_stopper_thread (kernel/stop_machine.c:474)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? cpu_stop_should_run (include/linux/list.h:189 kernel/stop_machine.c:443)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? ikconfig_read_current (kernel/stop_machine.c:437)
> [ 880.960700] ? trace_hardirqs_on_caller (kernel/locking/lockdep.c:2554 kernel/locking/lockdep.c:2601)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] smpboot_thread_fn (kernel/smpboot.c:161)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? schedule (./arch/x86/include/asm/bitops.h:311 (discriminator 1) kernel/sched/core.c:2824 (discriminator 1))
> [ 880.960700] ? __kthread_parkme (kernel/kthread.c:164)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] kthread (kernel/kthread.c:207)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ret_from_fork (arch/x86/kernel/entry_64.S:565)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] Code: eb 27 0f 1f 00 41 83 fe 03 75 0a 80 7d cf 00 0f 85 50 01 00 00 f0 41 ff 4f 24 74 71 41 83 fe 04 0f 84 d7 00 00 00 45 89 f0 f3 90 <4c> 89 e8 48 c1 e8 03 42 0f b6 04 20 84 c0 74 08 3c 03 0f 8e f5
> All code
> ========
> 0: eb 27 jmp 0x29
> 2: 0f 1f 00 nopl (%rax)
> 5: 41 83 fe 03 cmp $0x3,%r14d
> 9: 75 0a jne 0x15
> b: 80 7d cf 00 cmpb $0x0,-0x31(%rbp)
> f: 0f 85 50 01 00 00 jne 0x165
> 15: f0 41 ff 4f 24 lock decl 0x24(%r15)
> 1a: 74 71 je 0x8d
> 1c: 41 83 fe 04 cmp $0x4,%r14d
> 20: 0f 84 d7 00 00 00 je 0xfd
> 26: 45 89 f0 mov %r14d,%r8d
> 29: f3 90 pause
> 2b:* 4c 89 e8 mov %r13,%rax <-- trapping instruction
> 2e: 48 c1 e8 03 shr $0x3,%rax
> 32: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> 37: 84 c0 test %al,%al
> 39: 74 08 je 0x43
> 3b: 3c 03 cmp $0x3,%al
> 3d: 0f .byte 0xf
> 3e: 8e f5 mov %ebp,%?
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: 4c 89 e8 mov %r13,%rax
> 3: 48 c1 e8 03 shr $0x3,%rax
> 7: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> c: 84 c0 test %al,%al
> e: 74 08 je 0x18
> 10: 3c 03 cmp $0x3,%al
> 12: 0f .byte 0xf
> 13: 8e f5 mov %ebp,%?
> ...
> [ 880.960700] NMI backtrace for cpu 16
> [ 880.960700] CPU: 16 PID: 149 Comm: migration/16 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff8802f6ce8000 ti: ffff8802f6cf0000 task.ti: ffff8802f6cf0000
> [ 880.960700] RIP: multi_cpu_stop (kernel/stop_machine.c:192)
> [ 880.960700] RSP: 0000:ffff8802f6cf7b78 EFLAGS: 00000246
> [ 880.960700] RAX: 0000000000000000 RBX: ffffed00149d5ee5 RCX: 1ffff1005ed9d2d5
> [ 880.960700] RDX: 0000000000000000 RSI: ffffffff9e370ae0 RDI: ffffffffa31716b4
> [ 880.960700] RBP: ffff8802f6cf7bd8 R08: 0000000000000001 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: dffffc0000000000
> [ 880.960700] R13: ffff8800a4eaf738 R14: 0000000000000001 R15: ffff8800a4eaf718
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff8802f7200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 0000000001f6afe8 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffffffffa15a9540 ffffffff93532e90 ffff8800a4eaf728 0000000000000282
> [ 880.960700] ffff880200000000 00ff8802f6ce8000 ffff8802f6cf7bd8 dffffc0000000000
> [ 880.960700] ffff8802f7218440 ffff8800a4eaf778 ffff8800a4eaf718 ffffffff935335b0
> [ 880.960700] Call Trace:
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? queue_stop_cpus_work (kernel/stop_machine.c:170)
> [ 880.960700] cpu_stopper_thread (kernel/stop_machine.c:474)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? cpu_stop_should_run (include/linux/list.h:189 kernel/stop_machine.c:443)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? ikconfig_read_current (kernel/stop_machine.c:437)
> [ 880.960700] ? trace_hardirqs_on_caller (kernel/locking/lockdep.c:2554 kernel/locking/lockdep.c:2601)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] smpboot_thread_fn (kernel/smpboot.c:161)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? schedule (./arch/x86/include/asm/bitops.h:311 (discriminator 1) kernel/sched/core.c:2824 (discriminator 1))
> [ 880.960700] ? __kthread_parkme (kernel/kthread.c:164)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] kthread (kernel/kthread.c:207)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ret_from_fork (arch/x86/kernel/entry_64.S:565)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] Code: eb 27 0f 1f 00 41 83 fe 03 75 0a 80 7d cf 00 0f 85 50 01 00 00 f0 41 ff 4f 24 74 71 41 83 fe 04 0f 84 d7 00 00 00 45 89 f0 f3 90 <4c> 89 e8 48 c1 e8 03 42 0f b6 04 20 84 c0 74 08 3c 03 0f 8e f5
> All code
> ========
> 0: eb 27 jmp 0x29
> 2: 0f 1f 00 nopl (%rax)
> 5: 41 83 fe 03 cmp $0x3,%r14d
> 9: 75 0a jne 0x15
> b: 80 7d cf 00 cmpb $0x0,-0x31(%rbp)
> f: 0f 85 50 01 00 00 jne 0x165
> 15: f0 41 ff 4f 24 lock decl 0x24(%r15)
> 1a: 74 71 je 0x8d
> 1c: 41 83 fe 04 cmp $0x4,%r14d
> 20: 0f 84 d7 00 00 00 je 0xfd
> 26: 45 89 f0 mov %r14d,%r8d
> 29: f3 90 pause
> 2b:* 4c 89 e8 mov %r13,%rax <-- trapping instruction
> 2e: 48 c1 e8 03 shr $0x3,%rax
> 32: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> 37: 84 c0 test %al,%al
> 39: 74 08 je 0x43
> 3b: 3c 03 cmp $0x3,%al
> 3d: 0f .byte 0xf
> 3e: 8e f5 mov %ebp,%?
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: 4c 89 e8 mov %r13,%rax
> 3: 48 c1 e8 03 shr $0x3,%rax
> 7: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> c: 84 c0 test %al,%al
> e: 74 08 je 0x18
> 10: 3c 03 cmp $0x3,%al
> 12: 0f .byte 0xf
> 13: 8e f5 mov %ebp,%?
> ...
> [ 880.960700] NMI backtrace for cpu 17
> [ 880.960700] CPU: 17 PID: 158 Comm: migration/17 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff880320c7b000 ti: ffff880320c88000 task.ti: ffff880320c88000
> [ 880.960700] RIP: multi_cpu_stop (kernel/stop_machine.c:192)
> [ 880.960700] RSP: 0000:ffff880320c8fb78 EFLAGS: 00000246
> [ 880.960700] RAX: 0000000000000000 RBX: ffffed000b451ee5 RCX: 1ffff1006418f8d5
> [ 880.960700] RDX: 0000000000000000 RSI: ffffffff9e370ae0 RDI: ffffffffa31716b4
> [ 880.960700] RBP: ffff880320c8fbd8 R08: 0000000000000001 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: dffffc0000000000
> [ 880.960700] R13: ffff88005a28f738 R14: 0000000000000001 R15: ffff88005a28f718
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff880321000000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 000000000161bff8 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffffffffa15a9540 ffffffff93532e90 ffff88005a28f728 0000000000000282
> [ 880.960700] ffff880300000000 00ff880320c7b000 ffff880320c8fbd8 dffffc0000000000
> [ 880.960700] ffff880321018440 ffff88005a28f778 ffff88005a28f718 ffffffff935335b0
> [ 880.960700] Call Trace:
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? queue_stop_cpus_work (kernel/stop_machine.c:170)
> [ 880.960700] cpu_stopper_thread (kernel/stop_machine.c:474)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? cpu_stop_should_run (include/linux/list.h:189 kernel/stop_machine.c:443)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? ikconfig_read_current (kernel/stop_machine.c:437)
> [ 880.960700] ? trace_hardirqs_on_caller (kernel/locking/lockdep.c:2554 kernel/locking/lockdep.c:2601)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] smpboot_thread_fn (kernel/smpboot.c:161)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? schedule (./arch/x86/include/asm/bitops.h:311 (discriminator 1) kernel/sched/core.c:2824 (discriminator 1))
> [ 880.960700] ? __kthread_parkme (kernel/kthread.c:164)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] kthread (kernel/kthread.c:207)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ret_from_fork (arch/x86/kernel/entry_64.S:565)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] Code: eb 27 0f 1f 00 41 83 fe 03 75 0a 80 7d cf 00 0f 85 50 01 00 00 f0 41 ff 4f 24 74 71 41 83 fe 04 0f 84 d7 00 00 00 45 89 f0 f3 90 <4c> 89 e8 48 c1 e8 03 42 0f b6 04 20 84 c0 74 08 3c 03 0f 8e f5
> All code
> ========
> 0: eb 27 jmp 0x29
> 2: 0f 1f 00 nopl (%rax)
> 5: 41 83 fe 03 cmp $0x3,%r14d
> 9: 75 0a jne 0x15
> b: 80 7d cf 00 cmpb $0x0,-0x31(%rbp)
> f: 0f 85 50 01 00 00 jne 0x165
> 15: f0 41 ff 4f 24 lock decl 0x24(%r15)
> 1a: 74 71 je 0x8d
> 1c: 41 83 fe 04 cmp $0x4,%r14d
> 20: 0f 84 d7 00 00 00 je 0xfd
> 26: 45 89 f0 mov %r14d,%r8d
> 29: f3 90 pause
> 2b:* 4c 89 e8 mov %r13,%rax <-- trapping instruction
> 2e: 48 c1 e8 03 shr $0x3,%rax
> 32: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> 37: 84 c0 test %al,%al
> 39: 74 08 je 0x43
> 3b: 3c 03 cmp $0x3,%al
> 3d: 0f .byte 0xf
> 3e: 8e f5 mov %ebp,%?
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: 4c 89 e8 mov %r13,%rax
> 3: 48 c1 e8 03 shr $0x3,%rax
> 7: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> c: 84 c0 test %al,%al
> e: 74 08 je 0x18
> 10: 3c 03 cmp $0x3,%al
> 12: 0f .byte 0xf
> 13: 8e f5 mov %ebp,%?
> ...
> [ 880.960700] NMI backtrace for cpu 18
> [ 880.960700] CPU: 18 PID: 0 Comm: swapper/18 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff8806ad0b0000 ti: ffff88000eac0000 task.ti: ffff88000eac0000
> [ 880.960700] RIP: native_safe_halt (./arch/x86/include/asm/irqflags.h:50)
> [ 880.960700] RSP: 0018:ffff88000eac7d38 EFLAGS: 00000282
> [ 880.960700] RAX: dffffc0000000000 RBX: dffffc0000000000 RCX: 0000000000000000
> [ 880.960700] RDX: 1ffffffff4115884 RSI: 0000000000000001 RDI: ffffffffa08ac420
> [ 880.960700] RBP: ffff88000eac7d38 R08: 0000000000000000 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa31716b4
> [ 880.960700] R13: ffff88000eac7fd8 R14: 0000000000000000 R15: ffff88000eac7fd8
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff88034b200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 0000000001692ac0 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffff88000eac7d68 ffffffff93160774 dffffc0000000000 ffffffffa31716b4
> [ 880.960700] ffff88000eac7fd8 0000000000000000 ffff88000eac7d78 ffffffff931629ef
> [ 880.960700] ffff88000eac7e68 ffffffff934018e0 ffff88000eac7de8 ffffffff9dbf7ec5
> [ 880.960700] Call Trace:
> [ 880.960700] default_idle (./arch/x86/include/asm/paravirt.h:111 arch/x86/kernel/process.c:314)
> [ 880.960700] arch_cpu_idle (arch/x86/kernel/process.c:306)
> [ 880.960700] cpu_startup_entry (./arch/x86/include/asm/thread_info.h:162 include/linux/sched.h:2880 kernel/sched/idle.c:189 kernel/sched/idle.c:250 kernel/sched/idle.c:298)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? arch_cpu_idle_prepare (??:?)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? clockevents_register_device (kernel/time/clockevents.c:481)
> [ 880.960700] start_secondary (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] ? set_cpu_sibling_map (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] Code: 00 00 00 00 00 55 48 89 e5 fa 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb f4 <5d> c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 f4 5d c3 66 0f 1f 84
> All code
> ========
> 0: 00 00 add %al,(%rax)
> 2: 00 00 add %al,(%rax)
> 4: 00 55 48 add %dl,0x48(%rbp)
> 7: 89 e5 mov %esp,%ebp
> 9: fa cli
> a: 5d pop %rbp
> b: c3 retq
> c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 13: 00 00
> 15: 55 push %rbp
> 16: 48 89 e5 mov %rsp,%rbp
> 19: fb sti
> 1a: 5d pop %rbp
> 1b: c3 retq
> 1c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 23: 00 00
> 25: 55 push %rbp
> 26: 48 89 e5 mov %rsp,%rbp
> 29: fb sti
> 2a: f4 hlt
> 2b:* 5d pop %rbp <-- trapping instruction
> 2c: c3 retq
> 2d: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 34: 00
> 35: 55 push %rbp
> 36: 48 89 e5 mov %rsp,%rbp
> 39: f4 hlt
> 3a: 5d pop %rbp
> 3b: c3 retq
> 3c: 66 data16
> 3d: 0f .byte 0xf
> 3e: 1f (bad)
> 3f: 84 00 test %al,(%rax)
>
> Code starting with the faulting instruction
> ===========================================
> 0: 5d pop %rbp
> 1: c3 retq
> 2: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 9: 00
> a: 55 push %rbp
> b: 48 89 e5 mov %rsp,%rbp
> e: f4 hlt
> f: 5d pop %rbp
> 10: c3 retq
> 11: 66 data16
> 12: 0f .byte 0xf
> 13: 1f (bad)
> 14: 84 00 test %al,(%rax)
> [ 880.960700] NMI backtrace for cpu 19
> [ 880.960700] CPU: 19 PID: 0 Comm: swapper/19 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff8807ed0c8000 ti: ffff88000eac8000 task.ti: ffff88000eac8000
> [ 880.960700] RIP: native_safe_halt (./arch/x86/include/asm/irqflags.h:50)
> [ 880.960700] RSP: 0018:ffff88000eacfd38 EFLAGS: 00000282
> [ 880.960700] RAX: dffffc0000000000 RBX: dffffc0000000000 RCX: 0000000000000000
> [ 880.960700] RDX: 1ffffffff4115884 RSI: 0000000000000001 RDI: ffffffffa08ac420
> [ 880.960700] RBP: ffff88000eacfd38 R08: 0000000000000000 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa31716b4
> [ 880.960700] R13: ffff88000eacffd8 R14: 0000000000000000 R15: ffff88000eacffd8
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff880375200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 00007ffe1ae26d00 CR3: 000000000660a000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffff88000eacfd68 ffffffff93160774 dffffc0000000000 ffffffffa31716b4
> [ 880.960700] ffff88000eacffd8 0000000000000000 ffff88000eacfd78 ffffffff931629ef
> [ 880.960700] ffff88000eacfe68 ffffffff934018e0 ffff88000eacfde8 ffffffff9dbf7ec5
> [ 880.960700] Call Trace:
> [ 880.960700] default_idle (./arch/x86/include/asm/paravirt.h:111 arch/x86/kernel/process.c:314)
> [ 880.960700] arch_cpu_idle (arch/x86/kernel/process.c:306)
> [ 880.960700] cpu_startup_entry (./arch/x86/include/asm/thread_info.h:162 include/linux/sched.h:2880 kernel/sched/idle.c:189 kernel/sched/idle.c:250 kernel/sched/idle.c:298)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? arch_cpu_idle_prepare (??:?)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? clockevents_register_device (kernel/time/clockevents.c:481)
> [ 880.960700] start_secondary (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] ? set_cpu_sibling_map (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] Code: 00 00 00 00 00 55 48 89 e5 fa 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb f4 <5d> c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 f4 5d c3 66 0f 1f 84
> All code
> ========
> 0: 00 00 add %al,(%rax)
> 2: 00 00 add %al,(%rax)
> 4: 00 55 48 add %dl,0x48(%rbp)
> 7: 89 e5 mov %esp,%ebp
> 9: fa cli
> a: 5d pop %rbp
> b: c3 retq
> c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 13: 00 00
> 15: 55 push %rbp
> 16: 48 89 e5 mov %rsp,%rbp
> 19: fb sti
> 1a: 5d pop %rbp
> 1b: c3 retq
> 1c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 23: 00 00
> 25: 55 push %rbp
> 26: 48 89 e5 mov %rsp,%rbp
> 29: fb sti
> 2a: f4 hlt
> 2b:* 5d pop %rbp <-- trapping instruction
> 2c: c3 retq
> 2d: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 34: 00
> 35: 55 push %rbp
> 36: 48 89 e5 mov %rsp,%rbp
> 39: f4 hlt
> 3a: 5d pop %rbp
> 3b: c3 retq
> 3c: 66 data16
> 3d: 0f .byte 0xf
> 3e: 1f (bad)
> 3f: 84 00 test %al,(%rax)
>
> Code starting with the faulting instruction
> ===========================================
> 0: 5d pop %rbp
> 1: c3 retq
> 2: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 9: 00
> a: 55 push %rbp
> b: 48 89 e5 mov %rsp,%rbp
> e: f4 hlt
> f: 5d pop %rbp
> 10: c3 retq
> 11: 66 data16
> 12: 0f .byte 0xf
> 13: 1f (bad)
> 14: 84 00 test %al,(%rax)
> [ 880.960700] NMI backtrace for cpu 20
> [ 880.960700] CPU: 20 PID: 185 Comm: migration/20 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff88039ece0000 ti: ffff88039ece8000 task.ti: ffff88039ece8000
> [ 880.960700] RIP: multi_cpu_stop (kernel/stop_machine.c:192)
> [ 880.960700] RSP: 0000:ffff88039ecefb78 EFLAGS: 00000246
> [ 880.960700] RAX: 0000000000000000 RBX: ffffed008ddccee5 RCX: 1ffff10073d9c2d5
> [ 880.960700] RDX: 0000000000000000 RSI: ffffffff9e370ae0 RDI: ffffffffa31716b4
> [ 880.960700] RBP: ffff88039ecefbd8 R08: 0000000000000001 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: dffffc0000000000
> [ 880.960700] R13: ffff88046ee67738 R14: 0000000000000001 R15: ffff88046ee67718
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff88039f200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 00000000029aeff8 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffffffffa15a9540 ffffffff93532e90 ffff88046ee67728 0000000000000282
> [ 880.960700] ffff880300000000 00ff88039ece0000 ffff88039ecefbd8 dffffc0000000000
> [ 880.960700] ffff88039f218440 ffff88046ee67778 ffff88046ee67718 ffffffff935335b0
> [ 880.960700] Call Trace:
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? queue_stop_cpus_work (kernel/stop_machine.c:170)
> [ 880.960700] cpu_stopper_thread (kernel/stop_machine.c:474)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? cpu_stop_should_run (include/linux/list.h:189 kernel/stop_machine.c:443)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? ikconfig_read_current (kernel/stop_machine.c:437)
> [ 880.960700] ? trace_hardirqs_on_caller (kernel/locking/lockdep.c:2554 kernel/locking/lockdep.c:2601)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] smpboot_thread_fn (kernel/smpboot.c:161)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? schedule (./arch/x86/include/asm/bitops.h:311 (discriminator 1) kernel/sched/core.c:2824 (discriminator 1))
> [ 880.960700] ? __kthread_parkme (kernel/kthread.c:164)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] kthread (kernel/kthread.c:207)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ret_from_fork (arch/x86/kernel/entry_64.S:565)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] Code: eb 27 0f 1f 00 41 83 fe 03 75 0a 80 7d cf 00 0f 85 50 01 00 00 f0 41 ff 4f 24 74 71 41 83 fe 04 0f 84 d7 00 00 00 45 89 f0 f3 90 <4c> 89 e8 48 c1 e8 03 42 0f b6 04 20 84 c0 74 08 3c 03 0f 8e f5
> All code
> ========
> 0: eb 27 jmp 0x29
> 2: 0f 1f 00 nopl (%rax)
> 5: 41 83 fe 03 cmp $0x3,%r14d
> 9: 75 0a jne 0x15
> b: 80 7d cf 00 cmpb $0x0,-0x31(%rbp)
> f: 0f 85 50 01 00 00 jne 0x165
> 15: f0 41 ff 4f 24 lock decl 0x24(%r15)
> 1a: 74 71 je 0x8d
> 1c: 41 83 fe 04 cmp $0x4,%r14d
> 20: 0f 84 d7 00 00 00 je 0xfd
> 26: 45 89 f0 mov %r14d,%r8d
> 29: f3 90 pause
> 2b:* 4c 89 e8 mov %r13,%rax <-- trapping instruction
> 2e: 48 c1 e8 03 shr $0x3,%rax
> 32: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> 37: 84 c0 test %al,%al
> 39: 74 08 je 0x43
> 3b: 3c 03 cmp $0x3,%al
> 3d: 0f .byte 0xf
> 3e: 8e f5 mov %ebp,%?
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: 4c 89 e8 mov %r13,%rax
> 3: 48 c1 e8 03 shr $0x3,%rax
> 7: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> c: 84 c0 test %al,%al
> e: 74 08 je 0x18
> 10: 3c 03 cmp $0x3,%al
> 12: 0f .byte 0xf
> 13: 8e f5 mov %ebp,%?
> ...
> [ 880.960700] NMI backtrace for cpu 21
> [ 880.960700] CPU: 21 PID: 194 Comm: migration/21 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff8803c8ca8000 ti: ffff8803c8cb0000 task.ti: ffff8803c8cb0000
> [ 880.960700] RIP: multi_cpu_stop (kernel/stop_machine.c:192)
> [ 880.960700] RSP: 0000:ffff8803c8cb7b78 EFLAGS: 00000246
> [ 880.960700] RAX: 0000000000000000 RBX: ffffed00149caee5 RCX: 1ffff100791952d5
> [ 880.960700] RDX: 0000000000000000 RSI: ffffffff9e370ae0 RDI: ffffffffa31716b4
> [ 880.960700] RBP: ffff8803c8cb7bd8 R08: 0000000000000001 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: dffffc0000000000
> [ 880.960700] R13: ffff8800a4e57738 R14: 0000000000000001 R15: ffff8800a4e57718
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff8803c9000000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 0000000002914fb0 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffffffffa15a9540 ffffffff93532e90 ffff8800a4e57728 0000000000000282
> [ 880.960700] ffff880300000000 00ff8803c8ca8000 ffff8803c8cb7bd8 dffffc0000000000
> [ 880.960700] ffff8803c9018440 ffff8800a4e57778 ffff8800a4e57718 ffffffff935335b0
> [ 880.960700] Call Trace:
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? queue_stop_cpus_work (kernel/stop_machine.c:170)
> [ 880.960700] cpu_stopper_thread (kernel/stop_machine.c:474)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? cpu_stop_should_run (include/linux/list.h:189 kernel/stop_machine.c:443)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? ikconfig_read_current (kernel/stop_machine.c:437)
> [ 880.960700] ? trace_hardirqs_on_caller (kernel/locking/lockdep.c:2554 kernel/locking/lockdep.c:2601)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] smpboot_thread_fn (kernel/smpboot.c:161)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? schedule (./arch/x86/include/asm/bitops.h:311 (discriminator 1) kernel/sched/core.c:2824 (discriminator 1))
> [ 880.960700] ? __kthread_parkme (kernel/kthread.c:164)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] kthread (kernel/kthread.c:207)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ret_from_fork (arch/x86/kernel/entry_64.S:565)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] Code: eb 27 0f 1f 00 41 83 fe 03 75 0a 80 7d cf 00 0f 85 50 01 00 00 f0 41 ff 4f 24 74 71 41 83 fe 04 0f 84 d7 00 00 00 45 89 f0 f3 90 <4c> 89 e8 48 c1 e8 03 42 0f b6 04 20 84 c0 74 08 3c 03 0f 8e f5
> All code
> ========
> 0: eb 27 jmp 0x29
> 2: 0f 1f 00 nopl (%rax)
> 5: 41 83 fe 03 cmp $0x3,%r14d
> 9: 75 0a jne 0x15
> b: 80 7d cf 00 cmpb $0x0,-0x31(%rbp)
> f: 0f 85 50 01 00 00 jne 0x165
> 15: f0 41 ff 4f 24 lock decl 0x24(%r15)
> 1a: 74 71 je 0x8d
> 1c: 41 83 fe 04 cmp $0x4,%r14d
> 20: 0f 84 d7 00 00 00 je 0xfd
> 26: 45 89 f0 mov %r14d,%r8d
> 29: f3 90 pause
> 2b:* 4c 89 e8 mov %r13,%rax <-- trapping instruction
> 2e: 48 c1 e8 03 shr $0x3,%rax
> 32: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> 37: 84 c0 test %al,%al
> 39: 74 08 je 0x43
> 3b: 3c 03 cmp $0x3,%al
> 3d: 0f .byte 0xf
> 3e: 8e f5 mov %ebp,%?
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: 4c 89 e8 mov %r13,%rax
> 3: 48 c1 e8 03 shr $0x3,%rax
> 7: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> c: 84 c0 test %al,%al
> e: 74 08 je 0x18
> 10: 3c 03 cmp $0x3,%al
> 12: 0f .byte 0xf
> 13: 8e f5 mov %ebp,%?
> ...
> [ 880.960700] NMI backtrace for cpu 22
> [ 880.960700] CPU: 22 PID: 203 Comm: migration/22 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff8803f2cdb000 ti: ffff8803f2ce8000 task.ti: ffff8803f2ce8000
> [ 880.960700] RIP: multi_cpu_stop (kernel/stop_machine.c:192)
> [ 880.960700] RSP: 0000:ffff8803f2cefb78 EFLAGS: 00000246
> [ 880.960700] RAX: 0000000000000000 RBX: ffffed008dde3ee5 RCX: 1ffff1007e59b8d5
> [ 880.960700] RDX: 0000000000000000 RSI: ffffffff9e370ae0 RDI: ffffffffa31716b4
> [ 880.960700] RBP: ffff8803f2cefbd8 R08: 0000000000000001 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: dffffc0000000000
> [ 880.960700] R13: ffff88046ef1f738 R14: 0000000000000001 R15: ffff88046ef1f718
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff8803f3200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 0000000002e38b80 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffffffffa15a9540 ffffffff93532e90 ffff88046ef1f728 0000000000000282
> [ 880.960700] ffff880300000000 00ff8803f2cdb000 ffff8803f2cefbd8 dffffc0000000000
> [ 880.960700] ffff8803f3218440 ffff88046ef1f778 ffff88046ef1f718 ffffffff935335b0
> [ 880.960700] Call Trace:
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? queue_stop_cpus_work (kernel/stop_machine.c:170)
> [ 880.960700] cpu_stopper_thread (kernel/stop_machine.c:474)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? cpu_stop_should_run (include/linux/list.h:189 kernel/stop_machine.c:443)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? ikconfig_read_current (kernel/stop_machine.c:437)
> [ 880.960700] ? trace_hardirqs_on_caller (kernel/locking/lockdep.c:2554 kernel/locking/lockdep.c:2601)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] smpboot_thread_fn (kernel/smpboot.c:161)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? schedule (./arch/x86/include/asm/bitops.h:311 (discriminator 1) kernel/sched/core.c:2824 (discriminator 1))
> [ 880.960700] ? __kthread_parkme (kernel/kthread.c:164)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] kthread (kernel/kthread.c:207)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ret_from_fork (arch/x86/kernel/entry_64.S:565)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] Code: eb 27 0f 1f 00 41 83 fe 03 75 0a 80 7d cf 00 0f 85 50 01 00 00 f0 41 ff 4f 24 74 71 41 83 fe 04 0f 84 d7 00 00 00 45 89 f0 f3 90 <4c> 89 e8 48 c1 e8 03 42 0f b6 04 20 84 c0 74 08 3c 03 0f 8e f5
> All code
> ========
> 0: eb 27 jmp 0x29
> 2: 0f 1f 00 nopl (%rax)
> 5: 41 83 fe 03 cmp $0x3,%r14d
> 9: 75 0a jne 0x15
> b: 80 7d cf 00 cmpb $0x0,-0x31(%rbp)
> f: 0f 85 50 01 00 00 jne 0x165
> 15: f0 41 ff 4f 24 lock decl 0x24(%r15)
> 1a: 74 71 je 0x8d
> 1c: 41 83 fe 04 cmp $0x4,%r14d
> 20: 0f 84 d7 00 00 00 je 0xfd
> 26: 45 89 f0 mov %r14d,%r8d
> 29: f3 90 pause
> 2b:* 4c 89 e8 mov %r13,%rax <-- trapping instruction
> 2e: 48 c1 e8 03 shr $0x3,%rax
> 32: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> 37: 84 c0 test %al,%al
> 39: 74 08 je 0x43
> 3b: 3c 03 cmp $0x3,%al
> 3d: 0f .byte 0xf
> 3e: 8e f5 mov %ebp,%?
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: 4c 89 e8 mov %r13,%rax
> 3: 48 c1 e8 03 shr $0x3,%rax
> 7: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> c: 84 c0 test %al,%al
> e: 74 08 je 0x18
> 10: 3c 03 cmp $0x3,%al
> 12: 0f .byte 0xf
> 13: 8e f5 mov %ebp,%?
> ...
> [ 880.960700] NMI backtrace for cpu 23
> [ 880.960700] CPU: 23 PID: 212 Comm: migration/23 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff88041cc7b000 ti: ffff88041cce0000 task.ti: ffff88041cce0000
> [ 880.960700] RIP: multi_cpu_stop (kernel/stop_machine.c:192)
> [ 880.960700] RSP: 0000:ffff88041cce7b78 EFLAGS: 00000246
> [ 880.960700] RAX: 0000000000000000 RBX: ffffed006926eee5 RCX: 1ffff1008398f8d5
> [ 880.960700] RDX: 0000000000000000 RSI: ffffffff9e370ae0 RDI: ffffffffa31716b4
> [ 880.960700] RBP: ffff88041cce7bd8 R08: 0000000000000001 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: dffffc0000000000
> [ 880.960700] R13: ffff880349377738 R14: 0000000000000001 R15: ffff880349377718
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff88041d200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 00007f9c055599d0 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffffffffa15a9540 ffffffff93532e90 ffff880349377728 0000000000000282
> [ 880.960700] ffff880400000000 00ff88041cc7b000 ffff88041cce7bd8 dffffc0000000000
> [ 880.960700] ffff88041d218440 ffff880349377778 ffff880349377718 ffffffff935335b0
> [ 880.960700] Call Trace:
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? queue_stop_cpus_work (kernel/stop_machine.c:170)
> [ 880.960700] cpu_stopper_thread (kernel/stop_machine.c:474)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? cpu_stop_should_run (include/linux/list.h:189 kernel/stop_machine.c:443)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? ikconfig_read_current (kernel/stop_machine.c:437)
> [ 880.960700] ? trace_hardirqs_on_caller (kernel/locking/lockdep.c:2554 kernel/locking/lockdep.c:2601)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] smpboot_thread_fn (kernel/smpboot.c:161)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? schedule (./arch/x86/include/asm/bitops.h:311 (discriminator 1) kernel/sched/core.c:2824 (discriminator 1))
> [ 880.960700] ? __kthread_parkme (kernel/kthread.c:164)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] kthread (kernel/kthread.c:207)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ret_from_fork (arch/x86/kernel/entry_64.S:565)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] Code: eb 27 0f 1f 00 41 83 fe 03 75 0a 80 7d cf 00 0f 85 50 01 00 00 f0 41 ff 4f 24 74 71 41 83 fe 04 0f 84 d7 00 00 00 45 89 f0 f3 90 <4c> 89 e8 48 c1 e8 03 42 0f b6 04 20 84 c0 74 08 3c 03 0f 8e f5
> All code
> ========
> 0: eb 27 jmp 0x29
> 2: 0f 1f 00 nopl (%rax)
> 5: 41 83 fe 03 cmp $0x3,%r14d
> 9: 75 0a jne 0x15
> b: 80 7d cf 00 cmpb $0x0,-0x31(%rbp)
> f: 0f 85 50 01 00 00 jne 0x165
> 15: f0 41 ff 4f 24 lock decl 0x24(%r15)
> 1a: 74 71 je 0x8d
> 1c: 41 83 fe 04 cmp $0x4,%r14d
> 20: 0f 84 d7 00 00 00 je 0xfd
> 26: 45 89 f0 mov %r14d,%r8d
> 29: f3 90 pause
> 2b:* 4c 89 e8 mov %r13,%rax <-- trapping instruction
> 2e: 48 c1 e8 03 shr $0x3,%rax
> 32: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> 37: 84 c0 test %al,%al
> 39: 74 08 je 0x43
> 3b: 3c 03 cmp $0x3,%al
> 3d: 0f .byte 0xf
> 3e: 8e f5 mov %ebp,%?
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: 4c 89 e8 mov %r13,%rax
> 3: 48 c1 e8 03 shr $0x3,%rax
> 7: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> c: 84 c0 test %al,%al
> e: 74 08 je 0x18
> 10: 3c 03 cmp $0x3,%al
> 12: 0f .byte 0xf
> 13: 8e f5 mov %ebp,%?
> ...
> [ 880.960700] NMI backtrace for cpu 24
> [ 880.960700] CPU: 24 PID: 0 Comm: swapper/24 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff8801d0c68000 ti: ffff88000eed0000 task.ti: ffff88000eed0000
> [ 880.960700] RIP: native_safe_halt (./arch/x86/include/asm/irqflags.h:50)
> [ 880.960700] RSP: 0018:ffff88000eed7d38 EFLAGS: 00000282
> [ 880.960700] RAX: dffffc0000000000 RBX: dffffc0000000000 RCX: 0000000000000000
> [ 880.960700] RDX: 1ffffffff4115884 RSI: 0000000000000001 RDI: ffffffffa08ac420
> [ 880.960700] RBP: ffff88000eed7d38 R08: 0000000000000000 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa31716b4
> [ 880.960700] R13: ffff88000eed7fd8 R14: 0000000000000000 R15: ffff88000eed7fd8
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff880447200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 00000000029e4220 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffff88000eed7d68 ffffffff93160774 dffffc0000000000 ffffffffa31716b4
> [ 880.960700] ffff88000eed7fd8 0000000000000000 ffff88000eed7d78 ffffffff931629ef
> [ 880.960700] ffff88000eed7e68 ffffffff934018e0 ffff88000eed7de8 ffffffff9dbf7ec5
> [ 880.960700] Call Trace:
> [ 880.960700] default_idle (./arch/x86/include/asm/paravirt.h:111 arch/x86/kernel/process.c:314)
> [ 880.960700] arch_cpu_idle (arch/x86/kernel/process.c:306)
> [ 880.960700] cpu_startup_entry (./arch/x86/include/asm/thread_info.h:162 include/linux/sched.h:2880 kernel/sched/idle.c:189 kernel/sched/idle.c:250 kernel/sched/idle.c:298)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? arch_cpu_idle_prepare (??:?)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? clockevents_register_device (kernel/time/clockevents.c:481)
> [ 880.960700] start_secondary (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] ? set_cpu_sibling_map (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] Code: 00 00 00 00 00 55 48 89 e5 fa 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb f4 <5d> c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 f4 5d c3 66 0f 1f 84
> All code
> ========
> 0: 00 00 add %al,(%rax)
> 2: 00 00 add %al,(%rax)
> 4: 00 55 48 add %dl,0x48(%rbp)
> 7: 89 e5 mov %esp,%ebp
> 9: fa cli
> a: 5d pop %rbp
> b: c3 retq
> c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 13: 00 00
> 15: 55 push %rbp
> 16: 48 89 e5 mov %rsp,%rbp
> 19: fb sti
> 1a: 5d pop %rbp
> 1b: c3 retq
> 1c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 23: 00 00
> 25: 55 push %rbp
> 26: 48 89 e5 mov %rsp,%rbp
> 29: fb sti
> 2a: f4 hlt
> 2b:* 5d pop %rbp <-- trapping instruction
> 2c: c3 retq
> 2d: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 34: 00
> 35: 55 push %rbp
> 36: 48 89 e5 mov %rsp,%rbp
> 39: f4 hlt
> 3a: 5d pop %rbp
> 3b: c3 retq
> 3c: 66 data16
> 3d: 0f .byte 0xf
> 3e: 1f (bad)
> 3f: 84 00 test %al,(%rax)
>
> Code starting with the faulting instruction
> ===========================================
> 0: 5d pop %rbp
> 1: c3 retq
> 2: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 9: 00
> a: 55 push %rbp
> b: 48 89 e5 mov %rsp,%rbp
> e: f4 hlt
> f: 5d pop %rbp
> 10: c3 retq
> 11: 66 data16
> 12: 0f .byte 0xf
> 13: 1f (bad)
> 14: 84 00 test %al,(%rax)
> [ 880.960700] NMI backtrace for cpu 25
> [ 880.960700] CPU: 25 PID: 12438 Comm: trinity-c68 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff8802cb318000 ti: ffff8802cb3a8000 task.ti: ffff8802cb3a8000
> [ 880.960700] RIP: rwsem_down_write_failed (./arch/x86/include/asm/processor.h:658 ./arch/x86/include/asm/processor.h:663 kernel/locking/rwsem-xadd.c:413 kernel/locking/rwsem-xadd.c:442)
> [ 880.960700] RSP: 0018:ffff8802cb3afc08 EFLAGS: 00000286
> [ 880.960700] RAX: ffffffff00000001 RBX: ffffffffa18e12a8 RCX: ffff8802cb318000
> [ 880.960700] RDX: fffffffe00000001 RSI: fffffffeffffffff RDI: 0000000100000000
> [ 880.960700] RBP: ffff8802cb3afd98 R08: 0000000000000000 R09: ffffffff00000001
> [ 880.960700] R10: 0000000000000001 R11: 0000000000000001 R12: ffff8803f10c3000
> [ 880.960700] R13: ffffffffa18e12a0 R14: ffff8802cb3afd68 R15: ffffffffa18e1250
> [ 880.960700] FS: 00007f9c05559700(0000) GS:ffff880471000000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 880.960700] CR2: 00007f9c0549b614 CR3: 00000002cb385000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffffffff9dbf3ced ffff8802cb318d00 0000000000000000 ffff8802cb318000
> [ 880.960700] 00000000000004ff ffffffffa3ebf6b0 1ffff10059675f89 1ffff10059675f8d
> [ 880.960700] 0000000041b58ab3 ffff8802cb3a8010 1ffff10059675002 ffff8802cb3a8000
> [ 880.960700] Call Trace:
> [ 880.960700] ? rwsem_down_write_failed (include/linux/rcupdate.h:912 kernel/locking/rwsem-xadd.c:306 kernel/locking/rwsem-xadd.c:381 kernel/locking/rwsem-xadd.c:442)
> [ 880.960700] ? rwsem_down_read_failed (kernel/locking/rwsem-xadd.c:433)
> [ 880.960700] ? trace_hardirqs_off (kernel/locking/lockdep.c:2647)
> [ 880.960700] ? __acct_update_integrals (kernel/tsacct.c:125)
> [ 880.960700] ? taskstats_exit (kernel/tsacct.c:125)
> [ 880.960700] call_rwsem_down_write_failed (arch/x86/lib/rwsem.S:104)
> [ 880.960700] ? down_write (kernel/locking/rwsem.h:4 kernel/locking/rwsem.c:52)
> [ 880.960700] ? ipcget (ipc/util.c:349 ipc/util.c:646)
> [ 880.960700] ipcget (ipc/util.c:349 ipc/util.c:646)
> [ 880.960700] ? trace_hardirqs_on (kernel/locking/lockdep.c:2609)
> [ 880.960700] ? syscall_trace_enter_phase1 (include/linux/context_tracking.h:27 arch/x86/kernel/ptrace.c:1486)
> [ 880.960700] SyS_semget (ipc/sem.c:591 ipc/sem.c:572)
> [ 880.960700] ? sem_exit_ns (ipc/sem.c:572)
> [ 880.960700] ? trace_hardirqs_on_thunk (arch/x86/lib/thunk_64.S:42)
> [ 880.960700] tracesys_phase2 (arch/x86/kernel/entry_64.S:347)
> [ 880.960700] Code: 8d 0c 0a 48 89 d0 f0 49 0f b1 0f 48 39 c2 0f 84 2b 04 00 00 48 89 c2 48 8d 04 3a 48 85 f0 74 df 4d 85 e4 0f 84 50 04 00 00 f3 90 <e9> 57 ff ff ff e8 02 62 87 f5 84 c0 0f 85 f5 fd ff ff 48 c7 c2
> All code
> ========
> 0: 8d 0c 0a lea (%rdx,%rcx,1),%ecx
> 3: 48 89 d0 mov %rdx,%rax
> 6: f0 49 0f b1 0f lock cmpxchg %rcx,(%r15)
> b: 48 39 c2 cmp %rax,%rdx
> e: 0f 84 2b 04 00 00 je 0x43f
> 14: 48 89 c2 mov %rax,%rdx
> 17: 48 8d 04 3a lea (%rdx,%rdi,1),%rax
> 1b: 48 85 f0 test %rsi,%rax
> 1e: 74 df je 0xffffffffffffffff
> 20: 4d 85 e4 test %r12,%r12
> 23: 0f 84 50 04 00 00 je 0x479
> 29: f3 90 pause
> 2b:* e9 57 ff ff ff jmpq 0xffffffffffffff87 <-- trapping instruction
> 30: e8 02 62 87 f5 callq 0xfffffffff5876237
> 35: 84 c0 test %al,%al
> 37: 0f 85 f5 fd ff ff jne 0xfffffffffffffe32
> 3d: 48 rex.W
> 3e: c7 .byte 0xc7
> 3f: c2 .byte 0xc2
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: e9 57 ff ff ff jmpq 0xffffffffffffff5c
> 5: e8 02 62 87 f5 callq 0xfffffffff587620c
> a: 84 c0 test %al,%al
> c: 0f 85 f5 fd ff ff jne 0xfffffffffffffe07
> 12: 48 rex.W
> 13: c7 .byte 0xc7
> 14: c2 .byte 0xc2
> ...
> [ 880.960700] NMI backtrace for cpu 26
> [ 880.960700] CPU: 26 PID: 0 Comm: swapper/26 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff8803c8c78000 ti: ffff88000eee0000 task.ti: ffff88000eee0000
> [ 880.960700] RIP: native_safe_halt (./arch/x86/include/asm/irqflags.h:50)
> [ 880.960700] RSP: 0018:ffff88000eee7d38 EFLAGS: 00000282
> [ 880.960700] RAX: dffffc0000000000 RBX: dffffc0000000000 RCX: 0000000000000000
> [ 880.960700] RDX: 1ffffffff4115884 RSI: 0000000000000001 RDI: ffffffffa08ac420
> [ 880.960700] RBP: ffff88000eee7d38 R08: 0000000000000000 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa31716b4
> [ 880.960700] R13: ffff88000eee7fd8 R14: 0000000000000000 R15: ffff88000eee7fd8
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff88049b200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 00007f0f2bc1d5a8 CR3: 000000000678f000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffff88000eee7d68 ffffffff93160774 dffffc0000000000 ffffffffa31716b4
> [ 880.960700] ffff88000eee7fd8 0000000000000000 ffff88000eee7d78 ffffffff931629ef
> [ 880.960700] ffff88000eee7e68 ffffffff934018e0 ffff88000eee7de8 ffffffff9dbf7ec5
> [ 880.960700] Call Trace:
> [ 880.960700] default_idle (./arch/x86/include/asm/paravirt.h:111 arch/x86/kernel/process.c:314)
> [ 880.960700] arch_cpu_idle (arch/x86/kernel/process.c:306)
> [ 880.960700] cpu_startup_entry (./arch/x86/include/asm/thread_info.h:162 include/linux/sched.h:2880 kernel/sched/idle.c:189 kernel/sched/idle.c:250 kernel/sched/idle.c:298)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? arch_cpu_idle_prepare (??:?)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? clockevents_register_device (kernel/time/clockevents.c:481)
> [ 880.960700] start_secondary (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] ? set_cpu_sibling_map (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] Code: 00 00 00 00 00 55 48 89 e5 fa 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb f4 <5d> c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 f4 5d c3 66 0f 1f 84
> All code
> ========
> 0: 00 00 add %al,(%rax)
> 2: 00 00 add %al,(%rax)
> 4: 00 55 48 add %dl,0x48(%rbp)
> 7: 89 e5 mov %esp,%ebp
> 9: fa cli
> a: 5d pop %rbp
> b: c3 retq
> c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 13: 00 00
> 15: 55 push %rbp
> 16: 48 89 e5 mov %rsp,%rbp
> 19: fb sti
> 1a: 5d pop %rbp
> 1b: c3 retq
> 1c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 23: 00 00
> 25: 55 push %rbp
> 26: 48 89 e5 mov %rsp,%rbp
> 29: fb sti
> 2a: f4 hlt
> 2b:* 5d pop %rbp <-- trapping instruction
> 2c: c3 retq
> 2d: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 34: 00
> 35: 55 push %rbp
> 36: 48 89 e5 mov %rsp,%rbp
> 39: f4 hlt
> 3a: 5d pop %rbp
> 3b: c3 retq
> 3c: 66 data16
> 3d: 0f .byte 0xf
> 3e: 1f (bad)
> 3f: 84 00 test %al,(%rax)
>
> Code starting with the faulting instruction
> ===========================================
> 0: 5d pop %rbp
> 1: c3 retq
> 2: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 9: 00
> a: 55 push %rbp
> b: 48 89 e5 mov %rsp,%rbp
> e: f4 hlt
> f: 5d pop %rbp
> 10: c3 retq
> 11: 66 data16
> 12: 0f .byte 0xf
> 13: 1f (bad)
> 14: 84 00 test %al,(%rax)
> [ 880.960700] NMI backtrace for cpu 27
> [ 880.960700] CPU: 27 PID: 248 Comm: migration/27 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff8804c4cf8000 ti: ffff8804c3b00000 task.ti: ffff8804c3b00000
> [ 880.960700] RIP: multi_cpu_stop (kernel/stop_machine.c:192)
> [ 880.960700] RSP: 0000:ffff8804c3b07b78 EFLAGS: 00000246
> [ 880.960700] RAX: 0000000000000000 RBX: ffffed0072423ee5 RCX: 1ffff1009899f2d5
> [ 880.960700] RDX: 0000000000000000 RSI: ffffffff9e370ae0 RDI: ffffffffa31716b4
> [ 880.960700] RBP: ffff8804c3b07bd8 R08: 0000000000000001 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: dffffc0000000000
> [ 880.960700] R13: ffff88039211f738 R14: 0000000000000001 R15: ffff88039211f718
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff8804c5200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 00000000029c4ff8 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffffffffa15a9540 ffffffff93532e90 ffff88039211f728 0000000000000282
> [ 880.960700] ffff880400000000 00ff8804c4cf8000 ffff8804c3b07bd8 dffffc0000000000
> [ 880.960700] ffff8804c5218440 ffff88039211f778 ffff88039211f718 ffffffff935335b0
> [ 880.960700] Call Trace:
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? queue_stop_cpus_work (kernel/stop_machine.c:170)
> [ 880.960700] cpu_stopper_thread (kernel/stop_machine.c:474)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? cpu_stop_should_run (include/linux/list.h:189 kernel/stop_machine.c:443)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? ikconfig_read_current (kernel/stop_machine.c:437)
> [ 880.960700] ? trace_hardirqs_on_caller (kernel/locking/lockdep.c:2554 kernel/locking/lockdep.c:2601)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] smpboot_thread_fn (kernel/smpboot.c:161)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? schedule (./arch/x86/include/asm/bitops.h:311 (discriminator 1) kernel/sched/core.c:2824 (discriminator 1))
> [ 880.960700] ? __kthread_parkme (kernel/kthread.c:164)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] kthread (kernel/kthread.c:207)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ret_from_fork (arch/x86/kernel/entry_64.S:565)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] Code: eb 27 0f 1f 00 41 83 fe 03 75 0a 80 7d cf 00 0f 85 50 01 00 00 f0 41 ff 4f 24 74 71 41 83 fe 04 0f 84 d7 00 00 00 45 89 f0 f3 90 <4c> 89 e8 48 c1 e8 03 42 0f b6 04 20 84 c0 74 08 3c 03 0f 8e f5
> All code
> ========
> 0: eb 27 jmp 0x29
> 2: 0f 1f 00 nopl (%rax)
> 5: 41 83 fe 03 cmp $0x3,%r14d
> 9: 75 0a jne 0x15
> b: 80 7d cf 00 cmpb $0x0,-0x31(%rbp)
> f: 0f 85 50 01 00 00 jne 0x165
> 15: f0 41 ff 4f 24 lock decl 0x24(%r15)
> 1a: 74 71 je 0x8d
> 1c: 41 83 fe 04 cmp $0x4,%r14d
> 20: 0f 84 d7 00 00 00 je 0xfd
> 26: 45 89 f0 mov %r14d,%r8d
> 29: f3 90 pause
> 2b:* 4c 89 e8 mov %r13,%rax <-- trapping instruction
> 2e: 48 c1 e8 03 shr $0x3,%rax
> 32: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> 37: 84 c0 test %al,%al
> 39: 74 08 je 0x43
> 3b: 3c 03 cmp $0x3,%al
> 3d: 0f .byte 0xf
> 3e: 8e f5 mov %ebp,%?
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: 4c 89 e8 mov %r13,%rax
> 3: 48 c1 e8 03 shr $0x3,%rax
> 7: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> c: 84 c0 test %al,%al
> e: 74 08 je 0x18
> 10: 3c 03 cmp $0x3,%al
> 12: 0f .byte 0xf
> 13: 8e f5 mov %ebp,%?
> ...
> [ 880.960700] NMI backtrace for cpu 28
> [ 880.960700] CPU: 28 PID: 0 Comm: swapper/28 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff88065d0c0000 ti: ffff88000eef0000 task.ti: ffff88000eef0000
> [ 880.960700] RIP: native_safe_halt (./arch/x86/include/asm/irqflags.h:50)
> [ 880.960700] RSP: 0018:ffff88000eef7d38 EFLAGS: 00000282
> [ 880.960700] RAX: dffffc0000000000 RBX: dffffc0000000000 RCX: 0000000000000000
> [ 880.960700] RDX: 1ffffffff4115884 RSI: 0000000000000001 RDI: ffffffffa08ac420
> [ 880.960700] RBP: ffff88000eef7d38 R08: 0000000000000000 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa31716b4
> [ 880.960700] R13: ffff88000eef7fd8 R14: 0000000000000000 R15: ffff88000eef7fd8
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff8804ef200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 0000000000b34fb8 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffff88000eef7d68 ffffffff93160774 dffffc0000000000 ffffffffa31716b4
> [ 880.960700] ffff88000eef7fd8 0000000000000000 ffff88000eef7d78 ffffffff931629ef
> [ 880.960700] ffff88000eef7e68 ffffffff934018e0 0000000000000000 0000000000000000
> [ 880.960700] Call Trace:
> [ 880.960700] default_idle (./arch/x86/include/asm/paravirt.h:111 arch/x86/kernel/process.c:314)
> [ 880.960700] arch_cpu_idle (arch/x86/kernel/process.c:306)
> [ 880.960700] cpu_startup_entry (./arch/x86/include/asm/thread_info.h:162 include/linux/sched.h:2880 kernel/sched/idle.c:189 kernel/sched/idle.c:250 kernel/sched/idle.c:298)
> [ 880.960700] ? arch_cpu_idle_prepare (??:?)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? clockevents_register_device (kernel/time/clockevents.c:481)
> [ 880.960700] start_secondary (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] ? set_cpu_sibling_map (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] Code: 00 00 00 00 00 55 48 89 e5 fa 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb f4 <5d> c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 f4 5d c3 66 0f 1f 84
> All code
> ========
> 0: 00 00 add %al,(%rax)
> 2: 00 00 add %al,(%rax)
> 4: 00 55 48 add %dl,0x48(%rbp)
> 7: 89 e5 mov %esp,%ebp
> 9: fa cli
> a: 5d pop %rbp
> b: c3 retq
> c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 13: 00 00
> 15: 55 push %rbp
> 16: 48 89 e5 mov %rsp,%rbp
> 19: fb sti
> 1a: 5d pop %rbp
> 1b: c3 retq
> 1c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 23: 00 00
> 25: 55 push %rbp
> 26: 48 89 e5 mov %rsp,%rbp
> 29: fb sti
> 2a: f4 hlt
> 2b:* 5d pop %rbp <-- trapping instruction
> 2c: c3 retq
> 2d: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 34: 00
> 35: 55 push %rbp
> 36: 48 89 e5 mov %rsp,%rbp
> 39: f4 hlt
> 3a: 5d pop %rbp
> 3b: c3 retq
> 3c: 66 data16
> 3d: 0f .byte 0xf
> 3e: 1f (bad)
> 3f: 84 00 test %al,(%rax)
>
> Code starting with the faulting instruction
> ===========================================
> 0: 5d pop %rbp
> 1: c3 retq
> 2: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 9: 00
> a: 55 push %rbp
> b: 48 89 e5 mov %rsp,%rbp
> e: f4 hlt
> f: 5d pop %rbp
> 10: c3 retq
> 11: 66 data16
> 12: 0f .byte 0xf
> 13: 1f (bad)
> 14: 84 00 test %al,(%rax)
> [ 880.960700] NMI backtrace for cpu 29
> [ 880.960700] CPU: 29 PID: 0 Comm: swapper/29 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff88079d0b0000 ti: ffff88000eef8000 task.ti: ffff88000eef8000
> [ 880.960700] RIP: native_safe_halt (./arch/x86/include/asm/irqflags.h:50)
> [ 880.960700] RSP: 0018:ffff88000eeffd38 EFLAGS: 00000282
> [ 880.960700] RAX: dffffc0000000000 RBX: dffffc0000000000 RCX: 0000000000000000
> [ 880.960700] RDX: 1ffffffff4115884 RSI: 0000000000000001 RDI: ffffffffa08ac420
> [ 880.960700] RBP: ffff88000eeffd38 R08: 0000000000000000 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa31716b4
> [ 880.960700] R13: ffff88000eefffd8 R14: 0000000000000000 R15: ffff88000eefffd8
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff880519000000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 00007f0f2b8e5210 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffff88000eeffd68 ffffffff93160774 dffffc0000000000 ffffffffa31716b4
> [ 880.960700] ffff88000eefffd8 0000000000000000 ffff88000eeffd78 ffffffff931629ef
> [ 880.960700] ffff88000eeffe68 ffffffff934018e0 ffff88000eeffde8 ffffffff9dbf7ec5
> [ 880.960700] Call Trace:
> [ 880.960700] default_idle (./arch/x86/include/asm/paravirt.h:111 arch/x86/kernel/process.c:314)
> [ 880.960700] arch_cpu_idle (arch/x86/kernel/process.c:306)
> [ 880.960700] cpu_startup_entry (./arch/x86/include/asm/thread_info.h:162 include/linux/sched.h:2880 kernel/sched/idle.c:189 kernel/sched/idle.c:250 kernel/sched/idle.c:298)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? arch_cpu_idle_prepare (??:?)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? clockevents_register_device (kernel/time/clockevents.c:481)
> [ 880.960700] start_secondary (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] ? set_cpu_sibling_map (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] Code: 00 00 00 00 00 55 48 89 e5 fa 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb f4 <5d> c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 f4 5d c3 66 0f 1f 84
> All code
> ========
> 0: 00 00 add %al,(%rax)
> 2: 00 00 add %al,(%rax)
> 4: 00 55 48 add %dl,0x48(%rbp)
> 7: 89 e5 mov %esp,%ebp
> 9: fa cli
> a: 5d pop %rbp
> b: c3 retq
> c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 13: 00 00
> 15: 55 push %rbp
> 16: 48 89 e5 mov %rsp,%rbp
> 19: fb sti
> 1a: 5d pop %rbp
> 1b: c3 retq
> 1c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 23: 00 00
> 25: 55 push %rbp
> 26: 48 89 e5 mov %rsp,%rbp
> 29: fb sti
> 2a: f4 hlt
> 2b:* 5d pop %rbp <-- trapping instruction
> 2c: c3 retq
> 2d: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 34: 00
> 35: 55 push %rbp
> 36: 48 89 e5 mov %rsp,%rbp
> 39: f4 hlt
> 3a: 5d pop %rbp
> 3b: c3 retq
> 3c: 66 data16
> 3d: 0f .byte 0xf
> 3e: 1f (bad)
> 3f: 84 00 test %al,(%rax)
>
> Code starting with the faulting instruction
> ===========================================
> 0: 5d pop %rbp
> 1: c3 retq
> 2: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 9: 00
> a: 55 push %rbp
> b: 48 89 e5 mov %rsp,%rbp
> e: f4 hlt
> f: 5d pop %rbp
> 10: c3 retq
> 11: 66 data16
> 12: 0f .byte 0xf
> 13: 1f (bad)
> 14: 84 00 test %al,(%rax)
> [ 880.960700] NMI backtrace for cpu 30
> [ 880.960700] CPU: 30 PID: 269 Comm: migration/30 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff880542cf8000 ti: ffff880541b00000 task.ti: ffff880541b00000
> [ 880.960700] RIP: multi_cpu_stop (kernel/stop_machine.c:192)
> [ 880.960700] RSP: 0000:ffff880541b07b78 EFLAGS: 00000246
> [ 880.960700] RAX: 0000000000000000 RBX: ffffed005ea54ee5 RCX: 1ffff100a859f2d5
> [ 880.960700] RDX: 0000000000000000 RSI: ffffffff9e370ae0 RDI: ffffffffa31716b4
> [ 880.960700] RBP: ffff880541b07bd8 R08: 0000000000000001 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: dffffc0000000000
> [ 880.960700] R13: ffff8802f52a7738 R14: 0000000000000001 R15: ffff8802f52a7718
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff880543200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 00007f42552112c0 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffffffffa15a9540 ffffffff93532e90 ffff8802f52a7728 0000000000000282
> [ 880.960700] ffff880500000000 00ff880542cf8000 ffff880541b07bd8 dffffc0000000000
> [ 880.960700] ffff880543218440 ffff8802f52a7778 ffff8802f52a7718 ffffffff935335b0
> [ 880.960700] Call Trace:
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? queue_stop_cpus_work (kernel/stop_machine.c:170)
> [ 880.960700] cpu_stopper_thread (kernel/stop_machine.c:474)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] ? cpu_stop_should_run (include/linux/list.h:189 kernel/stop_machine.c:443)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? ikconfig_read_current (kernel/stop_machine.c:437)
> [ 880.960700] ? trace_hardirqs_on_caller (kernel/locking/lockdep.c:2554 kernel/locking/lockdep.c:2601)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? cpu_stop_create (kernel/stop_machine.c:449)
> [ 880.960700] smpboot_thread_fn (kernel/smpboot.c:161)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? schedule (./arch/x86/include/asm/bitops.h:311 (discriminator 1) kernel/sched/core.c:2824 (discriminator 1))
> [ 880.960700] ? __kthread_parkme (kernel/kthread.c:164)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] ? sort_range (kernel/smpboot.c:105)
> [ 880.960700] kthread (kernel/kthread.c:207)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] ret_from_fork (arch/x86/kernel/entry_64.S:565)
> [ 880.960700] ? flush_kthread_work (kernel/kthread.c:176)
> [ 880.960700] Code: eb 27 0f 1f 00 41 83 fe 03 75 0a 80 7d cf 00 0f 85 50 01 00 00 f0 41 ff 4f 24 74 71 41 83 fe 04 0f 84 d7 00 00 00 45 89 f0 f3 90 <4c> 89 e8 48 c1 e8 03 42 0f b6 04 20 84 c0 74 08 3c 03 0f 8e f5
> All code
> ========
> 0: eb 27 jmp 0x29
> 2: 0f 1f 00 nopl (%rax)
> 5: 41 83 fe 03 cmp $0x3,%r14d
> 9: 75 0a jne 0x15
> b: 80 7d cf 00 cmpb $0x0,-0x31(%rbp)
> f: 0f 85 50 01 00 00 jne 0x165
> 15: f0 41 ff 4f 24 lock decl 0x24(%r15)
> 1a: 74 71 je 0x8d
> 1c: 41 83 fe 04 cmp $0x4,%r14d
> 20: 0f 84 d7 00 00 00 je 0xfd
> 26: 45 89 f0 mov %r14d,%r8d
> 29: f3 90 pause
> 2b:* 4c 89 e8 mov %r13,%rax <-- trapping instruction
> 2e: 48 c1 e8 03 shr $0x3,%rax
> 32: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> 37: 84 c0 test %al,%al
> 39: 74 08 je 0x43
> 3b: 3c 03 cmp $0x3,%al
> 3d: 0f .byte 0xf
> 3e: 8e f5 mov %ebp,%?
> ...
>
> Code starting with the faulting instruction
> ===========================================
> 0: 4c 89 e8 mov %r13,%rax
> 3: 48 c1 e8 03 shr $0x3,%rax
> 7: 42 0f b6 04 20 movzbl (%rax,%r12,1),%eax
> c: 84 c0 test %al,%al
> e: 74 08 je 0x18
> 10: 3c 03 cmp $0x3,%al
> 12: 0f .byte 0xf
> 13: 8e f5 mov %ebp,%?
> ...
> [ 880.960700] NMI backtrace for cpu 31
> [ 880.960700] CPU: 31 PID: 0 Comm: swapper/31 Not tainted 4.0.0-rc1-next-20150227-sasha-00039-gae4a688-dirty #1971
> [ 880.960700] task: ffff88000f2d0000 ti: ffff88000f2d8000 task.ti: ffff88000f2d8000
> [ 880.960700] RIP: native_safe_halt (./arch/x86/include/asm/irqflags.h:50)
> [ 880.960700] RSP: 0000:ffff88000f2dfd38 EFLAGS: 00000282
> [ 880.960700] RAX: dffffc0000000000 RBX: dffffc0000000000 RCX: 0000000000000000
> [ 880.960700] RDX: 1ffffffff4115884 RSI: 0000000000000001 RDI: ffffffffa08ac420
> [ 880.960700] RBP: ffff88000f2dfd38 R08: 0000000000000000 R09: 0000000000000000
> [ 880.960700] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa31716b4
> [ 880.960700] R13: ffff88000f2dffd8 R14: 0000000000000000 R15: ffff88000f2dffd8
> [ 880.960700] FS: 0000000000000000(0000) GS:ffff88056d200000(0000) knlGS:0000000000000000
> [ 880.960700] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 880.960700] CR2: 00007fff144e2980 CR3: 000000002082c000 CR4: 00000000000007a0
> [ 880.960700] Stack:
> [ 880.960700] ffff88000f2dfd68 ffffffff93160774 dffffc0000000000 ffffffffa31716b4
> [ 880.960700] ffff88000f2dffd8 0000000000000000 ffff88000f2dfd78 ffffffff931629ef
> [ 880.960700] ffff88000f2dfe68 ffffffff934018e0 ffff88000f2dfde8 ffffffff9dbf7ec5
> [ 880.960700] Call Trace:
> [ 880.960700] default_idle (./arch/x86/include/asm/paravirt.h:111 arch/x86/kernel/process.c:314)
> [ 880.960700] arch_cpu_idle (arch/x86/kernel/process.c:306)
> [ 880.960700] cpu_startup_entry (./arch/x86/include/asm/thread_info.h:162 include/linux/sched.h:2880 kernel/sched/idle.c:189 kernel/sched/idle.c:250 kernel/sched/idle.c:298)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/paravirt.h:809 include/linux/spinlock_api_smp.h:162 kernel/locking/spinlock.c:191)
> [ 880.960700] ? arch_cpu_idle_prepare (??:?)
> [ 880.960700] ? _raw_spin_unlock_irqrestore (./arch/x86/include/asm/preempt.h:95 include/linux/spinlock_api_smp.h:163 kernel/locking/spinlock.c:191)
> [ 880.960700] ? clockevents_register_device (kernel/time/clockevents.c:481)
> [ 880.960700] start_secondary (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] ? set_cpu_sibling_map (arch/x86/kernel/smpboot.c:219)
> [ 880.960700] Code: 00 00 00 00 00 55 48 89 e5 fa 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb f4 <5d> c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 f4 5d c3 66 0f 1f 84
> All code
> ========
> 0: 00 00 add %al,(%rax)
> 2: 00 00 add %al,(%rax)
> 4: 00 55 48 add %dl,0x48(%rbp)
> 7: 89 e5 mov %esp,%ebp
> 9: fa cli
> a: 5d pop %rbp
> b: c3 retq
> c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 13: 00 00
> 15: 55 push %rbp
> 16: 48 89 e5 mov %rsp,%rbp
> 19: fb sti
> 1a: 5d pop %rbp
> 1b: c3 retq
> 1c: 66 0f 1f 84 00 00 00 nopw 0x0(%rax,%rax,1)
> 23: 00 00
> 25: 55 push %rbp
> 26: 48 89 e5 mov %rsp,%rbp
> 29: fb sti
> 2a: f4 hlt
> 2b:* 5d pop %rbp <-- trapping instruction
> 2c: c3 retq
> 2d: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 34: 00
> 35: 55 push %rbp
> 36: 48 89 e5 mov %rsp,%rbp
> 39: f4 hlt
> 3a: 5d pop %rbp
> 3b: c3 retq
> 3c: 66 data16
> 3d: 0f .byte 0xf
> 3e: 1f (bad)
> 3f: 84 00 test %al,(%rax)
>
> Code starting with the faulting instruction
> ===========================================
> 0: 5d pop %rbp
> 1: c3 retq
> 2: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1)
> 9: 00
> a: 55 push %rbp
> b: 48 89 e5 mov %rsp,%rbp
> e: f4 hlt
> f: 5d pop %rbp
> 10: c3 retq
> 11: 66 data16
> 12: 0f .byte 0xf
> 13: 1f (bad)
> 14: 84 00 test %al,(%rax)
>
>
> Thanks,
> Sasha
>

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Next message: Linus Walleij: "Re: [PATCH 0/2] pinctrl: Support for IMG Pistachio"
Previous message: Luis Henriques: "[PATCH 3.16.y-ckt 009/183] ARM: pxa: add regulator_has_full_constraints to corgi board file"
In reply to: Rafael David Tinoco: "Re: sched: softlockups in multi_cpu_stop"
Next in thread: Ingo Molnar: "Re: sched: softlockups in multi_cpu_stop"
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]