Re: [PATCH v2 1/2] x86/cpu: Disable CR pinning during CPU bringup

[Tom Lendacky]

On 3/9/26 13:03, Dave Hansen wrote:
> On 3/9/26 09:15, Borislav Petkov wrote:
>> On Mon, Mar 09, 2026 at 08:38:10AM -0700, Dave Hansen wrote:
>>> On 3/9/26 06:46, Borislav Petkov wrote:
>>>> My SNP guest stops booting with this right:
>>> Could you dump out CR4 at wakeup_cpu_via_vmgexit() before and after this
>>> patch? Right here:
>>>
>>>         /* CR4 should maintain the MCE value */
>>>         cr4 = native_read_cr4() & X86_CR4_MCE;
>>>
>>> It's got to be some delta there.
>> Looks the same to me:
>>
>> before:      31  SEV: wakeup_cpu_via_vmgexit: CR4: 0x3506f0
>>
>> That's 31 CPUs - no BSP with the CR4 value above.
>>
>> after: [    3.354326] SEV: wakeup_cpu_via_vmgexit: CR4: 0x3506f0
>>
>> That stops after CPU1, i.e., the first AP. But the CR4 value is the same.
> 
> The only pinned bits in there are: SMAP, SMEP and FSGSBASE.
> 
> SMAP and SMEP are unlikely to be biting us here.
> 
> FSGSBASE is _possible_ but I don't see any of the {RD,WR}{F,G}SBASE
> instructions in early boot where it would bite us.
> 
> Can you boot this thing without FSGSBASE support?
> 
> The other option would be to boot a working system, normally and see
> what is getting flipped by pinning at cr4_init(). The attached patch
> does that. It also uses trace_printk() so it hopefully won't trip over
> #VC's during early boot with the console.
> 
> For me, it's flipping on 0x310800, which is:
> 
> 	#define X86_CR4_OSXMMEXCPT      (1ul << 10)
> 	#define X86_CR4_FSGSBASE        (1ul << 16)
> 	#define X86_CR4_SMEP            (1ul << 20)
> 	#define X86_CR4_SMAP            (1ul << 21)
> 
> *Maybe* the paranoid entry code is getting called from the #VC handler
> in early boot? It has ALTERNATIVEs on X86_FEATURE_FSGSBASE and might be
> using the FSGSBASE instructions in there.

Could be... before the patch the AP CR4 value is:

[    0.020362] *** DEBUG: cr4_init - cr4=0x3100f0

after the patch it is:

[    0.020284] *** DEBUG: cr4_init - cr4=0xf0

The SNP guest is dying in __x2apic_enable() when trying to read
MSR_IA32_APICBASE, which will trigger a #VC.

If I set CR4[16] in cr4_init() then the SNP guest boots fine.

Thanks,
Tom