Kernel Shadow Stacks Versus XFEATURE_MASK_GUEST_SUPERVISOR

[Dave Hansen]

So, I'm working on kernel shadow stacks. I have some nice optimization
code like fred_update_rsp0() for SSP0. It avoids WRMSR's on exits to
userspace if FRED_SSP0 hasn't changed, just like FRED_RSP0.

Well, turns out that FRED_SSP0 isn't like FRED_RSP0 in one very
important way: it is an alias for PL0_SSP, which is managed via
XFEATURE_CET_KERNEL which is in:

	/* Supervisor features which are enabled only in guest FPUs */
	#define XFEATURE_MASK_GUEST_SUPERVISOR  XFEATURE_MASK_CET_KERNEL

So, I was running along just fine in my nice shadow stack kernel going
in and out of the kernel just fine. But the first time I did an XRSTOR,
the XFEATURE_CET_KERNEL feature got set to its init state. My SSP0 cache
didn't notice and I tried to run userspace with SSP0=0. Much hilarity
ensued on the next kernel entry.

The most adorable part of all this is that I have KVM compiled
completely out on my config. The "only in guest FPUs" code is biting me
on a kernel that can't even run guests.

I think this all started because of 8b05b3c98816 ("x86/fpu/xstate: Add
CET supervisor xfeature support as a guest-only feature"), which seems
like mostly my doing. So I have mostly myself to blame.

Any thoughts on what we should do?

Actually context switching XFEATURE_CET_KERNEL seems kinda silly in a
lot of ways. PL0_SSP can always be derived _and_ it isn't actually even
needed at context switch time, only when running userspace.

I could just aggressively blow away my SSP0 cache any time an XRSTOR
touches XFEATURE_CET_KERNEL. That would be easy. But it also seems silly
that XFEATURE_CET_KERNEL is being blown away at _all_ during normal
context switches. Heck, it goes further than that because:

fpregs_restore_userregs() does:

	restore_fpregs_from_fpstate(fpu->fpstate,
				    XFEATURE_MASK_FPSTATE);

Where XFEATURE_MASK_FPSTATE contains XFEATURE_MASK_SUPERVISOR_SUPPORTED.

So a "restore_userregs" function is blowing away supervisor state.

IOW, I think we're using XFEATURE_MASK_FPSTATE a bit too broadly.

I'd like to get to a place where only the KVM code even has
XFEATURE_CET_KERNEL in its RFBM.

Anybody have any bright ideas?