Re: Kernel Shadow Stacks Versus XFEATURE_MASK_GUEST_SUPERVISOR

[Paolo Bonzini]

On 10/23/26 21:03, Dave Hansen wrote:
> So, I'm working on kernel shadow stacks. I have some nice optimization
> code like fred_update_rsp0() for SSP0. It avoids WRMSR's on exits to
> userspace if FRED_SSP0 hasn't changed, just like FRED_RSP0.
>
> Well, turns out that FRED_SSP0 isn't like FRED_RSP0 in one very
> important way: it is an alias for PL0_SSP, which is managed via
> XFEATURE_CET_KERNEL which is in:
>
> 	/* Supervisor features which are enabled only in guest FPUs */
> 	#define XFEATURE_MASK_GUEST_SUPERVISOR  XFEATURE_MASK_CET_KERNEL
>
> So, I was running along just fine in my nice shadow stack kernel going
> in and out of the kernel just fine. But the first time I did an XRSTOR,
> the XFEATURE_CET_KERNEL feature got set to its init state. My SSP0 cache
> didn't notice and I tried to run userspace with SSP0=0. Much hilarity
> ensued on the next kernel entry.
>
> The most adorable part of all this is that I have KVM compiled
> completely out on my config. The "only in guest FPUs" code is biting me
> on a kernel that can't even run guests.

If you want to quickly work around for your !CONFIG_KVM kernel, I would do

#ifdef CONFIG_KVM
/* Supervisor features which are enabled only in guest FPUs */
#define XFEATURE_MASK_GUEST_SUPERVISOR  XFEATURE_MASK_CET_KERNEL
#else
#define XFEATURE_MASK_GUEST_SUPERVISOR  0
#endif

and a small adjustment like

        BUILD_BUG_ON((XFEATURE_MASK_USER_SUPPORTED |
                      XFEATURE_MASK_SUPERVISOR_SUPPORTED) &
                     ~XFEATURES_INIT_FPSTATE_HANDLED);

That will get rid of the bit in XSS.

> Any thoughts on what we should do?
>
> Actually context switching XFEATURE_CET_KERNEL seems kinda silly in a
> lot of ways. PL0_SSP can always be derived _and_ it isn't actually even
> needed at context switch time, only when running userspace.

True but, just to understand, what is the performance like if you 
actually use XSAVES/XRSTORS to update SSP0 on context switches instead 
of WRMSR+cache?  Hopefully the microcode does a bunch of optimizations 
as well.

> I could just aggressively blow away my SSP0 cache any time an XRSTOR
> touches XFEATURE_CET_KERNEL. That would be easy. But it also seems silly
> that XFEATURE_CET_KERNEL is being blown away at _all_ during normal
> context switches. Heck, it goes further than that because:
>
> fpregs_restore_userregs() does:
>
> 	restore_fpregs_from_fpstate(fpu->fpstate,
> 				    XFEATURE_MASK_FPSTATE);
>
> Where XFEATURE_MASK_FPSTATE contains XFEATURE_MASK_SUPERVISOR_SUPPORTED.

The actual mask used by restore_fpregs_from_fpstate is tighter though, 
it's "fpu_kernel_cfg.max_features & XFEATURE_MASK_FPSTATE".

Can you replace fpu_kernel_cfg.max_features with some fpstate->xyz 
(sorry too late to think it through :)) that has CET_KERNEL for the 
guest FPU and not for the regular one?

> So a "restore_userregs" function is blowing away supervisor state.
>
> IOW, I think we're using XFEATURE_MASK_FPSTATE a bit too broadly.
>
> I'd like to get to a place where only the KVM code even has
> XFEATURE_CET_KERNEL in its RFBM.
That I agree, but I think it can be handled in 
{save,restore}_fpregs_to_fpstate().

Paolo