Re: perf: stalls in perf_install_in_context/perf_remove_from_context

[Dmitry Vyukov]

On Tue, Dec 15, 2015 at 2:41 PM, Dmitry Vyukov <dvyukov@xxxxxxxxxx> wrote:
> On Tue, Dec 15, 2015 at 2:39 PM, Peter Zijlstra <peterz@xxxxxxxxxxxxx> wrote:
>> On Tue, Dec 15, 2015 at 02:01:32PM +0100, Dmitry Vyukov wrote:
>>
>>
>>> Call Trace:
>>>  <IRQ>  [<ffffffff810e5e38>] sched_show_task+0x148/0x250 kernel/sched/core.c:4934
>>
>>  <snip>
>>
>>>  [<ffffffff823ff46c>] apic_timer_interrupt+0x8c/0xa0 arch/x86/entry/entry_64.S:520
>>>
>>>  <EOI>  [<     inline     >] ? arch_local_irq_enable ./arch/x86/include/asm/paravirt.h:816
>>>  <EOI>  [<     inline     >] ? __raw_spin_unlock_irq include/linux/spinlock_api_smp.h:170
>>>  <EOI>  [<ffffffff823fde9e>] ? _raw_spin_unlock_irq+0x2e/0x40 kernel/locking/spinlock.c:199
>>>  [<     inline     >] ? __raw_spin_unlock_irq include/linux/spinlock_api_smp.h:170
>>>  [<ffffffff823fde97>] ? _raw_spin_unlock_irq+0x27/0x40 kernel/locking/spinlock.c:199
>>>  [<ffffffff8119c8bb>] perf_install_in_context+0x6b/0xf0 kernel/events/core.c:2173
>>>  [<ffffffff811a492c>] SYSC_perf_event_open+0x63c/0xe40 kernel/events/core.c:8565
>>>  [<ffffffff811a7739>] SyS_perf_event_open+0x9/0x10 kernel/events/core.c:8261
>>>  [<ffffffff823fe676>] entry_SYSCALL_64_fastpath+0x16/0x7a arch/x86/entry/entry_64.S:185
>>
>>
>> So this looks like its stuck in the retry loop in
>> perf_install_in_context(). We do the raw_spin_unlock_irq(), which allows
>> the above watchdog interrupt to happen, but make no progress.
>>
>> This means the task_function_call() thing keeps failing, which is
>> extremely unlikely.
>>
>> Let me have a play with your test-case to see if it reproduces.
>
> Yeah, both perf_install_in_context and perf_remove_from_context hang
> in a very similar loops.

Note that the test uses cpu%4 because I have 4 cpus. Here is how I
start qemu: -numa node,nodeid=0,cpus=0-1 -numa node,nodeid=1,cpus=2-3
-smp sockets=2,cores=2,threads=1
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/