Re: [PATCH] x86/fred: Fix early boot failures on SEV-ES/SNP guests

[Nikunj A. Dadhania]

On 2/6/2026 6:08 PM, Nikunj A. Dadhania wrote:
> 
> 
> On 2/5/2026 11:09 PM, Tom Lendacky wrote:
>> On 2/5/26 11:20, Dave Hansen wrote:
>>> On 2/5/26 08:10, Dave Hansen wrote:
>>>> Shouldn't we flip the FRED CR4 bit _last_, once all the MSRs are set up?
>>>> Why is it backwards in the first place? Why can't it be fixed?
>>>
>>> Ahhh, it was done by CR4 pinning. It's the first thing in C code for
>>> booting secondaries:
>>>
>>> static void notrace __noendbr start_secondary(void *unused)
>>> {
>>>         cr4_init();
>>>
>>> Since FRED is set in 'cr4_pinned_mask', cr4_init() sets the FRED bit far
>>> before the FRED MSRs are ready. Anyone else doing native_write_cr4()
>>> will do the same thing. That's obviously not what was intended from the
>>> pinning code or the FRED init code.
>>>
>>> Shouldn't we fix this properly rather than moving printk()'s around?
>>
>> I believe that is what this part of the thread decided on:
>>
>> https://lore.kernel.org/kvm/02df7890-83c2-4047-8c88-46fbc6e0a892@xxxxxxxxx/T/#m3e44c2c53aca3bcd872de4ce1e50a14500e62e4e
>>
>> Thanks,
>> Tom
>>
>>>
>>> One idea is just to turn off all the CR-pinning logic while bringing
>>> CPUs up. That way, nothing before:
>>>
>>> 	set_cpu_online(smp_processor_id(), true);
>>>
>>> can get tripped up by CR pinning. I've attached a completely untested
>>> patch to do that.
> 
> Yes, this works as well. And Xin Li's patch also resolves the issue by
> moving the cr4_init() later after initializing FRED MSRs.

Hi Dave,

This is what I have in my patch queue, Can I include your authorship and SOB?

Only change I made in your patch is s/cr4_pinning/cr_pinning.

From: Dave Hansen <dave.hansen@xxxxxxxxxxxxxxx>
Subject: [PATCH] x86/cpu: Disable CR pinning during CPU bringup

CR pinning can prematurely enable features during secondary CPU bringup
before their supporting infrastructure is initialized. Specifically, when
FRED is enabled, cr4_init() sets CR4.FRED via the pinned mask early in
start_secondary(), long before cpu_init_fred_exceptions() configures the
required FRED MSRs. This creates a window where exceptions cannot be
properly handled.

For SEV-ES/SNP and TDX guests, any console output during this window
triggers #VC or #VE exceptions that result in triple faults because the
exception handlers rely on FRED MSRs that aren't yet configured.

Disable CR pinning for offline CPUs by checking cpu_online() in the pinning
enforcement path. This allows features like FRED to be enabled in CR4 only
after their supporting MSRs and handlers are fully configured.

Fixes: 14619d912b65 ("x86/fred: FRED entry/exit and dispatch code")
Reported-by: Nikunj A Dadhania <nikunj@xxxxxxx>
Not-yet-Signed-off-by: Dave Hansen <dave.hansen@xxxxxxxxxxxxxxx>
[ Function renamed and added commit message]
Signed-off-by: Nikunj A Dadhania <nikunj@xxxxxxx>
Cc: stable@xxxxxxxxxxxxxxx # 6.9+
---
 arch/x86/kernel/cpu/common.c | 21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index 1c3261cae40c..934ca3f139d3 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -434,6 +434,21 @@ static const unsigned long cr4_pinned_mask = X86_CR4_SMEP | X86_CR4_SMAP | X86_C
 static DEFINE_STATIC_KEY_FALSE_RO(cr_pinning);
 static unsigned long cr4_pinned_bits __ro_after_init;
 
+static bool cr_pinning_enabled(void)
+{
+	if (!static_branch_likely(&cr_pinning))
+		return false;
+
+	/*
+	 * Do not enforce pinning during CPU bringup. It might
+	 * turn on features that are not set up yet, like FRED.
+	 */
+	if (!cpu_online(smp_processor_id()))
+		return false;
+
+	return true;
+}
+
 void native_write_cr0(unsigned long val)
 {
 	unsigned long bits_missing = 0;
@@ -441,7 +456,7 @@ void native_write_cr0(unsigned long val)
 set_register:
 	asm volatile("mov %0,%%cr0": "+r" (val) : : "memory");
 
-	if (static_branch_likely(&cr_pinning)) {
+	if (cr_pinning_enabled()) {
 		if (unlikely((val & X86_CR0_WP) != X86_CR0_WP)) {
 			bits_missing = X86_CR0_WP;
 			val |= bits_missing;
@@ -460,7 +475,7 @@ void __no_profile native_write_cr4(unsigned long val)
 set_register:
 	asm volatile("mov %0,%%cr4": "+r" (val) : : "memory");
 
-	if (static_branch_likely(&cr_pinning)) {
+	if (cr_pinning_enabled()) {
 		if (unlikely((val & cr4_pinned_mask) != cr4_pinned_bits)) {
 			bits_changed = (val & cr4_pinned_mask) ^ cr4_pinned_bits;
 			val = (val & ~cr4_pinned_mask) | cr4_pinned_bits;
@@ -502,7 +517,7 @@ void cr4_init(void)
 
 	if (boot_cpu_has(X86_FEATURE_PCID))
 		cr4 |= X86_CR4_PCIDE;
-	if (static_branch_likely(&cr_pinning))
+	if (cr_pinning_enabled())
 		cr4 = (cr4 & ~cr4_pinned_mask) | cr4_pinned_bits;
 
 	__write_cr4(cr4);
-- 
2.48.1