Re: [Patch V2] x86, mce: Ensure offline CPU's don't participate in mce rendezvous process.

[Borislav Petkov]

On Fri, Dec 04, 2015 at 07:29:36PM -0500, Ashok Raj wrote:
> Linux has logical cpu offline capability. That can be triggered by:
> 
> # echo 0 > /sys/devices/system/cpu/cpuX/online
> 
> In Intel Architecture, MCE's are broadcasted to all CPUs in the system.
> 
> This includes the CPUs marked offline by Linux. Unless the CPU's were removed
> via an ACPI notification, in which case the cpu's are removed from the
> cpu_present_map.
> 
> This patch ensures offline CPU's don't participate in MCE rendezvous, but
> simply perform clearing some status bits to ensure a second MCE wont cause
> automatic shutdown.
> 
> Without the patch, mce_start will increment mce_callin, but mce_start would
> only wait for all online_cpus. So offline cpu's should avoid participating
> in the rendezvous process.
> 
> Reviewed-by: Tony Luck <tony.luck@xxxxxxxxx>
> Cc: stable@xxxxxxxxxxxxxxx
> Signed-off-by: Ashok Raj <ashok.raj@xxxxxxxxx>
> ---
>  arch/x86/kernel/cpu/mcheck/mce.c | 13 +++++++++++++
>  1 file changed, 13 insertions(+)

Tested on a box here, massaged commit message and queued for 4.4,
thanks.

---
From: Ashok Raj <ashok.raj@xxxxxxxxx>
Date: Thu, 3 Dec 2015 19:16:10 -0500
Subject: [PATCH] x86/mce: Ensure offline CPUs don't participate in rendezvous
 process

Intel's MCA implementation broadcasts MCEs to all CPUs on the node.
This poses a problem for offlined CPUs which cannot participate in the
rendezvous process:

  Kernel panic - not syncing: Timeout: Not all CPUs entered broadcast exception handler
  Kernel Offset: disabled
  Rebooting in 100 seconds..

More specifically, Linux does a soft offline of a CPU when writing a 0
to /sys/devices/system/cpu/cpuX/online, which doesn't prevent the #MC
exception from being broadcasted to that CPU.

Ensure that offline CPUs don't participate in the MCE rendezvous and
clear the RIP valid status bit so that a second MCE won't cause a
shutdown.

Without the patch, mce_start() will increment mce_callin and wait for
all CPUs. Offlined CPUs should avoid participating in the rendezvous
process altogether.

Reviewed-by: Tony Luck <tony.luck@xxxxxxxxx>
Signed-off-by: Ashok Raj <ashok.raj@xxxxxxxxx>
Cc: "H. Peter Anvin" <hpa@xxxxxxxxx>
Cc: Ingo Molnar <mingo@xxxxxxxxxx>
Cc: linux-edac <linux-edac@xxxxxxxxxxxxxxx>
Cc: stable@xxxxxxxxxxxxxxx
Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
Cc: x86-ml <x86@xxxxxxxxxx>
Link: http://lkml.kernel.org/r/1449188170-3909-1-git-send-email-ashok.raj@xxxxxxxxx
[ Massage commit message. ]
Signed-off-by: Borislav Petkov <bp@xxxxxxx>
---
 arch/x86/kernel/cpu/mcheck/mce.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/arch/x86/kernel/cpu/mcheck/mce.c b/arch/x86/kernel/cpu/mcheck/mce.c
index 3865e95cc5ec..a006f4cd792b 100644
--- a/arch/x86/kernel/cpu/mcheck/mce.c
+++ b/arch/x86/kernel/cpu/mcheck/mce.c
@@ -1002,6 +1002,17 @@ void do_machine_check(struct pt_regs *regs, long error_code)
 	int flags = MF_ACTION_REQUIRED;
 	int lmce = 0;
 
+	/* If this CPU is offline, just bail out. */
+	if (cpu_is_offline(smp_processor_id())) {
+		u64 mcgstatus;
+
+		mcgstatus = mce_rdmsrl(MSR_IA32_MCG_STATUS);
+		if (mcgstatus & MCG_STATUS_RIPV) {
+			mce_wrmsrl(MSR_IA32_MCG_STATUS, 0);
+			return;
+		}
+	}
+
 	ist_enter(regs);
 
 	this_cpu_inc(mce_exception_count);
-- 
2.3.5

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/