[tip:x86/apic] x86/apic: Order irq_enter/exit() calls correctly vs. ack_APIC_irq()

From: tip-bot for Wanpeng Li
Date: Mon Sep 19 2016 - 18:43:28 EST

Commit-ID: b0f48706a176b71a6e54f399d7404bbeeaa7cfab
Gitweb: http://git.kernel.org/tip/b0f48706a176b71a6e54f399d7404bbeeaa7cfab
Author: Wanpeng Li <wanpeng.li@xxxxxxxxxxx>
AuthorDate: Sun, 18 Sep 2016 19:34:51 +0800
Committer: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
CommitDate: Tue, 20 Sep 2016 00:31:19 +0200

x86/apic: Order irq_enter/exit() calls correctly vs. ack_APIC_irq()

[ INFO: suspicious RCU usage. ]
4.8.0-rc6+ #5 Not tainted
./arch/x86/include/asm/msr-trace.h:47 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

RCU used illegally from idle CPU!
rcu_scheduler_active = 1, debug_locks = 0
RCU used illegally from extended quiescent state!
no locks held by swapper/2/0.

stack backtrace:
CPU: 2 PID: 0 Comm: swapper/2 Not tainted 4.8.0-rc6+ #5
Hardware name: Dell Inc. OptiPlex 7020/0F5C5X, BIOS A03 01/08/2015
0000000000000000 ffff8d1bd6003f10 ffffffff94446949 ffff8d1bd4a68000
0000000000000001 ffff8d1bd6003f40 ffffffff940e9247 ffff8d1bbdfcf3d0
000000000000080b 0000000000000000 0000000000000000 ffff8d1bd6003f70
Call Trace:
<IRQ> [<ffffffff94446949>] dump_stack+0x99/0xd0
[<ffffffff940e9247>] lockdep_rcu_suspicious+0xe7/0x120
[<ffffffff9448e0d5>] do_trace_write_msr+0x135/0x140
[<ffffffff9406e750>] native_write_msr+0x20/0x30
[<ffffffff9406503d>] native_apic_msr_eoi_write+0x1d/0x30
[<ffffffff9405b17e>] smp_trace_call_function_interrupt+0x1e/0x270
[<ffffffff948cb1d6>] trace_call_function_interrupt+0x96/0xa0
<EOI> [<ffffffff947200f4>] ? cpuidle_enter_state+0xe4/0x360
[<ffffffff947200df>] ? cpuidle_enter_state+0xcf/0x360
[<ffffffff947203a7>] cpuidle_enter+0x17/0x20
[<ffffffff940df008>] cpu_startup_entry+0x338/0x4d0
[<ffffffff9405bfc4>] start_secondary+0x154/0x180

This can be reproduced readily by running ftrace test case of kselftest.

Move the irq_enter() call before ack_APIC_irq(), because irq_enter() tells
the RCU susbstems to end the extended quiescent state, so that the
following trace call in ack_APIC_irq() works correctly. The same applies to
exiting_ack_irq() which calls ack_APIC_irq() after irq_exit().

[ tglx: Massaged changelog ]

Signed-off-by: Wanpeng Li <wanpeng.li@xxxxxxxxxxx>
Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
Cc: Wanpeng Li <wanpeng.li@xxxxxxxxxxx>
Link: http://lkml.kernel.org/r/1474198491-3738-1-git-send-email-wanpeng.li@xxxxxxxxxxx
Signed-off-by: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
arch/x86/include/asm/apic.h | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index 1243577..f5aaf6c 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -650,8 +650,8 @@ static inline void entering_ack_irq(void)

static inline void ipi_entering_ack_irq(void)
- ack_APIC_irq();
+ ack_APIC_irq();

static inline void exiting_irq(void)
@@ -661,9 +661,8 @@ static inline void exiting_irq(void)

static inline void exiting_ack_irq(void)
- irq_exit();
- /* Ack only at the end to avoid potential reentry */
+ irq_exit();

extern void ioapic_zap_locks(void);