[PATCH 3.12 231/235] x86/apic: Order irq_enter/exit() calls correctly vs. ack_APIC_irq()

From: Jiri Slaby
Date: Fri Jan 27 2017 - 06:18:01 EST

From: Wanpeng Li <wanpeng.li@xxxxxxxxxxx>

3.12-stable review patch. If anyone has any objections, please let me know.


commit b0f48706a176b71a6e54f399d7404bbeeaa7cfab upstream.

[ INFO: suspicious RCU usage. ]
4.8.0-rc6+ #5 Not tainted
./arch/x86/include/asm/msr-trace.h:47 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

RCU used illegally from idle CPU!
rcu_scheduler_active = 1, debug_locks = 0
RCU used illegally from extended quiescent state!
no locks held by swapper/2/0.

stack backtrace:
CPU: 2 PID: 0 Comm: swapper/2 Not tainted 4.8.0-rc6+ #5
Hardware name: Dell Inc. OptiPlex 7020/0F5C5X, BIOS A03 01/08/2015
0000000000000000 ffff8d1bd6003f10 ffffffff94446949 ffff8d1bd4a68000
0000000000000001 ffff8d1bd6003f40 ffffffff940e9247 ffff8d1bbdfcf3d0
000000000000080b 0000000000000000 0000000000000000 ffff8d1bd6003f70
Call Trace:
<IRQ> [<ffffffff94446949>] dump_stack+0x99/0xd0
[<ffffffff940e9247>] lockdep_rcu_suspicious+0xe7/0x120
[<ffffffff9448e0d5>] do_trace_write_msr+0x135/0x140
[<ffffffff9406e750>] native_write_msr+0x20/0x30
[<ffffffff9406503d>] native_apic_msr_eoi_write+0x1d/0x30
[<ffffffff9405b17e>] smp_trace_call_function_interrupt+0x1e/0x270
[<ffffffff948cb1d6>] trace_call_function_interrupt+0x96/0xa0
<EOI> [<ffffffff947200f4>] ? cpuidle_enter_state+0xe4/0x360
[<ffffffff947200df>] ? cpuidle_enter_state+0xcf/0x360
[<ffffffff947203a7>] cpuidle_enter+0x17/0x20
[<ffffffff940df008>] cpu_startup_entry+0x338/0x4d0
[<ffffffff9405bfc4>] start_secondary+0x154/0x180

This can be reproduced readily by running ftrace test case of kselftest.

Move the irq_enter() call before ack_APIC_irq(), because irq_enter() tells
the RCU susbstems to end the extended quiescent state, so that the
following trace call in ack_APIC_irq() works correctly. The same applies to
exiting_ack_irq() which calls ack_APIC_irq() after irq_exit().

[ tglx: Massaged changelog ]

Signed-off-by: Wanpeng Li <wanpeng.li@xxxxxxxxxxx>
Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
Cc: Wanpeng Li <wanpeng.li@xxxxxxxxxxx>
Link: http://lkml.kernel.org/r/1474198491-3738-1-git-send-email-wanpeng.li@xxxxxxxxxxx
Signed-off-by: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
Acked-by: Joerg Roedel <jroedel@xxxxxxx>
Signed-off-by: Jiri Slaby <jslaby@xxxxxxx>
arch/x86/include/asm/apic.h | 3 +--
arch/x86/kernel/smp.c | 2 +-
2 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index 29559831c94f..43849c3d6275 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -710,9 +710,8 @@ static inline void exiting_irq(void)

static inline void exiting_ack_irq(void)
- irq_exit();
- /* Ack only at the end to avoid potential reentry */
+ irq_exit();

extern void ioapic_zap_locks(void);
diff --git a/arch/x86/kernel/smp.c b/arch/x86/kernel/smp.c
index 7c3a5a61f2e4..e5d895fa1fe0 100644
--- a/arch/x86/kernel/smp.c
+++ b/arch/x86/kernel/smp.c
@@ -267,8 +267,8 @@ __visible void smp_reschedule_interrupt(struct pt_regs *regs)

static inline void smp_entering_irq(void)
- ack_APIC_irq();
+ ack_APIC_irq();

__visible void smp_trace_reschedule_interrupt(struct pt_regs *regs)