Re: lapic-related boot crash in 4.15-rc1

[Dou Liyang]

Hi Thomas, Meelis

At 01/11/2018 02:34 AM, Thomas Gleixner wrote:
> On Wed, 10 Jan 2018, Meelis Roos wrote:
> > On 3 of my test computers, boot hangs with 4.15 git kernels. So far I
> > have traced it down to 4.14.0 being good and 4.15-rc1 being bad (bisect
> > is slow because the computers are somwehat remote). Also because of
> > trying to find when it started, I have not tries newer than rc5
> > kernels.
> >
> > The hang happens after Performance Events feature table has been printed
> > but before APIC initialization messages normally appear ("Enabling APIC
> > mode:  Flat.  Using 0 I/O APICs" or something like that).
> >
> > It happens on at least 3 distinct computers:
> > * AMD Athlon with AMD 750 chipset
> > * P3 1 GHz with i815 chipset
> > * P4 1.7 GHz with SiS chipset
> >
> > They all have have "lapic" parameter because they were telling me that
> > lapic has been disabled by BIOS and I re-enabled it with the parameter
> > to get apic timer interrupts. This worked up until 4.14.
> >
> > None of the computers have IO-APICs.
>
> Oh. Then it possibly falls into Dou's realm.
>
> Dou?

Ah, This problem looks like what Ville reported[1].

Revert the commit b371ae0d4a19 ("x86/apic: Remove init_bsp_APIC()") may
make it work again.

But, I don't know the true reason.

In Linux v4.15-rc1+, apic->setup_apic_routing() has been called earlier,
So, I guess kernel may hang in some early initcall fuctions. we can add
"initcall_debug" option in the kernel command line for tracing initcalls
as they are executed.

[1] https://marc.info/?l=linux-kernel&m=151188084018443&w=2

Thanks,
	dou.
--------------------------->8-----------------------------------

From: Ville SyrjÃlà <ville.syrjala@xxxxxxxxxxxxxxx>

This reverts commit b371ae0d4a194b178817b0edfb6a7395c7aec37a.

Causes my P3 UP machine to hang at boot with "lapic".

Cc: Dou Liyang <douly.fnst@xxxxxxxxxxxxxx>
Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
Cc: yinghai@xxxxxxxxxx
Cc: bhe@xxxxxxxxxx
Cc: Ingo Molnar <mingo@xxxxxxxxxx>
Cc: "H. Peter Anvin" <hpa@xxxxxxxxx>
Signed-off-by: Ville SyrjÃlà <ville.syrjala@xxxxxxxxxxxxxxx>
---
 arch/x86/include/asm/apic.h |  1 +
 arch/x86/kernel/apic/apic.c | 49 
+++++++++++++++++++++++++++++++++++++++++++++
 arch/x86/kernel/irqinit.c   |  3 +++
 3 files changed, 53 insertions(+)

diff --git a/arch/x86/include/asm/apic.h b/arch/x86/include/asm/apic.h
index a9e57f08bfa6..98722773391d 100644
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -136,6 +136,7 @@ extern void disconnect_bsp_APIC(int virt_wire_setup);
 extern void disable_local_APIC(void);
 extern void lapic_shutdown(void);
 extern void sync_Arb_IDs(void);
+extern void init_bsp_APIC(void);
 extern void apic_intr_mode_init(void);
 extern void setup_local_APIC(void);
 extern void init_apic_mappings(void);
diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c
index 6e272f3ea984..cec9aaea7f9d 100644
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -1286,6 +1286,55 @@ static int __init apic_intr_mode_select(void)
 	return APIC_SYMMETRIC_IO;
 }

+/*
+ * An initial setup of the virtual wire mode.
+ */
+void __init init_bsp_APIC(void)
+{
+	unsigned int value;
+
+	/*
+	 * Don't do the setup now if we have a SMP BIOS as the
+	 * through-I/O-APIC virtual wire mode might be active.
+	 */
+	if (smp_found_config || !boot_cpu_has(X86_FEATURE_APIC))
+		return;
+
+	/*
+	 * Do not trust the local APIC being empty at bootup.
+	 */
+	clear_local_APIC();
+
+	/*
+	 * Enable APIC.
+	 */
+	value = apic_read(APIC_SPIV);
+	value &= ~APIC_VECTOR_MASK;
+	value |= APIC_SPIV_APIC_ENABLED;
+
+#ifdef CONFIG_X86_32
+	/* This bit is reserved on P4/Xeon and should be cleared */
+	if ((boot_cpu_data.x86_vendor == X86_VENDOR_INTEL) &&
+	    (boot_cpu_data.x86 == 15))
+		value &= ~APIC_SPIV_FOCUS_DISABLED;
+	else
+#endif
+		value |= APIC_SPIV_FOCUS_DISABLED;
+	value |= SPURIOUS_APIC_VECTOR;
+	apic_write(APIC_SPIV, value);
+
+	/*
+	 * Set up the virtual wire mode.
+	 */
+	apic_write(APIC_LVT0, APIC_DM_EXTINT);
+	value = APIC_DM_NMI;
+	if (!lapic_is_integrated())		/* 82489DX */
+		value |= APIC_LVT_LEVEL_TRIGGER;
+	if (apic_extnmi == APIC_EXTNMI_NONE)
+		value |= APIC_LVT_MASKED;
+	apic_write(APIC_LVT1, value);
+}
+
 /* Init the interrupt delivery mode for the BSP */
 void __init apic_intr_mode_init(void)
 {
diff --git a/arch/x86/kernel/irqinit.c b/arch/x86/kernel/irqinit.c
index 8da3e909e967..a539410c4ea9 100644
--- a/arch/x86/kernel/irqinit.c
+++ b/arch/x86/kernel/irqinit.c
@@ -61,6 +61,9 @@ void __init init_ISA_irqs(void)
 	struct irq_chip *chip = legacy_pic->chip;
 	int i;

+#if defined(CONFIG_X86_64) || defined(CONFIG_X86_LOCAL_APIC)
+	init_bsp_APIC();
+#endif
 	legacy_pic->init(0);

 	for (i = 0; i < nr_legacy_irqs(); i++)
--
> Thanks,
>
> 	tglx