Re: [PATCH v8 12/20] KVM: x86: Support REX2-prefixed opcode decode
From: Jim Mattson
Date: Fri Nov 06 2026 - 09:15:35 EST
On Thu, Nov 5, 2026 at 8:48 PM Jim Mattson <jmattson@xxxxxxxxxx> wrote:
>
> On Wed, Sep 30, 2026 at 3:32 PM Chang S. Bae <chang.seok.bae@xxxxxxxxx> wrote:
> >
> > Extend the instruction decoder to recognize and handle the REX2 prefix,
> > including validation of prefix sequences and correct opcode table
> > selection.
> >
> > REX2 is a terminal prefix: once 0xD5 is encountered, the following byte
> > is the opcode. When REX.M=0, most prefix bytes are invalid after REX2,
> > including REX, VEX, EVEX, and another REX2. Also, REX2-prefixed
> > instructions are only valid in 64-bit mode.
> >
> > All of the invalid prefix combinations after REX2 coincide with opcodes
> > that are architecturally invalid in 64-bit mode. Thus, marking such
> > opcodes with No64 in opcode_table[] naturally disallows those illegal
> > prefix sequences.
>
> This is only true for 0x40..0x4F, 0xC4, 0xC5, and 0xD5.
"This" being a No64 entry in opcode_table[]. However, N (NotImpl)
certainly subsumes No64.
> Per Section 3.1.2.1 ("REX2 Prefix") of the Intel APX Architecture
> Specification (rev. 9.0):
>
> Furthermore, since the byte following a REX2 prefix is always
> interpreted as the main opcode byte, any legacy prefix byte (namely,
> 0x66, 0x67, 0xF0, 0xF2, 0xF3, and segment overrides), a REX prefix
> byte (0x4*), a VEX prefix byte (0xC4 or 0xC5), an EVEX prefix byte
> (0x62), or another REX2 prefix byte (0xD5) following a REX2 prefix
> with REX2.M0 = 0 must #UD, because none of those bytes is the opcode
> of a valid instruction in legacy map 0 in 64-bit mode.
>
> and:
>
> The 0x0F escape byte is neither needed nor allowed. (That is, the
> REX2 prefix followed by 0x0F triggers #UD.)
>
> In opcode_table[], 0x0F, 0x26, 0x2E, 0x36, 0x3E, 0x62, 0x64, 0x65, 0x66,
> 0x67, 0xF0, 0xF2, and 0xF3 are all N (NotImpl), not No64. When any of these
> bytes follows REX2 with M0=0, KVM will return EMULATION_FAILED to userspace
> instead of raising #UD.
As in 11/20, this is utter nonsense. On EMULATION_FAILED,
x86_emulate_instruction() will raise #UD, as it always has for N
(NotImpl), with or without REX2.
> > Suggested-by: Paolo Bonzini <pbonzini@xxxxxxxxxx>
> > Tested-by: Zhao Liu <zhao1.liu@xxxxxxxxx>
> > Signed-off-by: Chang S. Bae <chang.seok.bae@xxxxxxxxx>
> > Link: https://lore.kernel.org/CABgObfYYGTvkYpeyqLSr9JgKMDA_STSff2hXBNchLZuKFU+MMA@xxxxxxxxxxxxxx
> > ---
>
> [ ... ]
>
> > @@ -4958,6 +4969,17 @@ int x86_decode_insn(struct x86_emulate_ctxt *ctxt, void *insn, int insn_len, int
> > ctxt->rex_prefix = REX_PREFIX;
> > ctxt->rex_bits = ctxt->b & 0xf;
> > continue;
> > + case 0xd5: /* REX2 */
> > + if (mode != X86EMUL_MODE_PROT64)
> > + goto done_prefixes;
> > + if (rex2_invalid(ctxt)) {
> > + opcode = ud;
> > + goto done_modrm;
> > + }
>
> "ud" is defined as I(SrcNone, emulate_ud), and SrcNone is 0. Jumping to
> done_modrm with opcode.flags == 0 sets ctxt->d = 0, which then hits:
>
> /* Unrecognised? */
> if (ctxt->d == 0)
> return EMULATION_FAILED;
>
> before ctxt->execute is ever set. As a result, rex2_invalid() always
> returns EMULATION_FAILED instead of injecting #UD.
Once again, on EMULATION_FAILED, x86_emulate_instruction() will raise #UD.
That leaves only this question on this patch...
> Also, because any legacy prefix after a REX prefix resets ctxt->rex_prefix
> to REX_NONE at the bottom of the prefix loop, a sequence such as 0x48 0x66
> 0xD5 will not be caught by rex2_invalid(). Table 3.1 ("Legacy Prefix
> Applicability with REX2") lists 0x4* (REX) under "Prefix before REX2" as
> "Illegal", whereas the accompanying text says, "A REX prefix (0x4*)
> immediately preceding the REX2 prefix is not allowed and triggers #UD."
> Does hardware raise #UD on 0x48 0x66 0xD5, or does the intervening legacy
> prefix nullify the REX prefix as in legacy encodings?
The text clearly permits 0x48 0x66 0xD5, since the REX prefix doesn't
*immediately* precede the REX2 prefix. The table is unclear.