Re: [PATCH v1 1/1] x86/fred: Clear the WFE bit in missing-ENDBRANCH #CP

From: Xin Li
Date: Thu Sep 12 2024 - 04:54:23 EST


On 9/11/2024 5:22 PM, Dave Hansen wrote:
On 9/11/24 16:19, Xin Li (Intel) wrote:
+/*
+ * The WFE (WAIT_FOR_ENDBRANCH) bit in the augmented CS of FRED stack frame is
+ * set to 1 in missing-ENDBRANCH #CP exceptions.

I think there's a bit of relatively irrelevant info in there. For
instance, I don't think it's super important to mention that FRED is
involved and where the WFE bit is in memory.

FRED's involvement is kinda a no-brainer from the whole X86_FEATURE_FRED
thing, and if you're reading exception handler code and don't know that
'regs' is on the stack, this probably isn't the place to explain that.

I often find myself in a dilemma, should I mention some technical
background which sometimes could also be distracting :(

Based on your feedback, maybe the following is better?

static void ibt_clear_fred_wfe(struct pt_regs *regs)
{
if (regs->fred_cs.wfe)
regs->fred_cs.wfe = 0;
}

And we know only FRED hardware will set the WFE bit.


+ * If the WFE bit is left as 1, the CPU will generate another missing-ENDBRANCH
+ * #CP because the indirect branch tracker will be set in the WAIT_FOR_ENDBRANCH
+ * state upon completion of the following ERETS instruction and the CPU will
+ * restart from the IP that just caused a previous missing-ENDBRANCH #CP.
+ *
+ * Clear the WFE bit to avoid dead looping due to the above reason.
+ */
+static void ibt_clear_fred_wfe(struct pt_regs *regs)
+{
+ if (cpu_feature_enabled(X86_FEATURE_FRED))
+ regs->fred_cs.wfe = 0;
+}

Can I suggest a slightly different comment?

/*
* WFE==1 (WAIT_FOR_ENDBRANCH) means that the CPU expects the next ERETS
* to jump to an ENDBR instruction. If the ENDBR is missing, the CPU
* raises a #CP.
*
* Clear WFE to avoid that #CP.
*
* Use this function in a #CP handler to effectively give the next
* ERETS a free pass to ignore IBT for a single instruction.
*/

I think original comment really needs a "How do I use this?" sentence or
two.

A comment at the call site also wouldn't hurt:

if (unlikely(regs->ip == (unsigned long)&ibt_selftest_noendbr)){
regs->ax = 0;
+ /* Disable IBT enforcement for one exception return: */
+ ibt_clear_fred_wfe(regs);
return;
}

I'm finding it kinda hard to concisely differentiate between the
"disable IBT at one ERETS" and "disable IBT forever", but I hope this
sounds good to folks.


My understanding is that a missing-ENDBRANCH #CP is triggered in two
steps:

1) Upon completion of an indirect call/jmp, or an event return
instruction, the CPU indirect branch tracker is put in the
WAIT_FOR_ENDBRANCH state.

2) As the CPU is in WAIT_FOR_ENDBRANCH state, if the instruction to
be executed is ENDBR, the CPU indirect branch tracker exits
WAIT_FOR_ENDBRANCH state, otherwise a #CP is generated.

So this is more of "preserve WAIT_FOR_ENDBRANCH state" or not.

IDT is unable to preserve WAIT_FOR_ENDBRANCH state when returning from
event handling, which as Andrew mentioned is a security hole.