Re: [PATCH v8 11/20] KVM: x86: Handle EGPR index and REX2-incompatible opcodes
From: Jim Mattson
Date: Thu Nov 05 2026 - 23:35:46 EST
On Wed, Sep 30, 2026 at 4:38 PM Chang S. Bae <chang.seok.bae@xxxxxxxxx> wrote:
>
> Prepare the emulator for REX2 handling by introducing the NoRex2 opcode
> flag and supporting extended register indices. For the latter, factor out
> common logic for calculating register IDs.
>
> REX2 does not support three-byte opcodes. Instead, the REX2.M bit selects
> between one- and two-byte opcode tables, which were previously
> distinguished by the 0x0F escape byte.
>
> Some legacy instructions in those tables never reference extended
> registers. When prefixed with REX, such instructions are treated as if
> the prefix were absent. In contrast, a REX2 prefix causes a #UD, which
> should be handled explicitly.
>
> Suggested-by: Paolo Bonzini <pbonzini@xxxxxxxxxx>
> Tested-by: Zhao Liu <zhao1.liu@xxxxxxxxx>
> Signed-off-by: Chang S. Bae <chang.seok.bae@xxxxxxxxx>
> Link: https://lore.kernel.org/1ebf3a23-5671-41c1-8daa-c83f2f105936@xxxxxxxxxx
> ---
[ ... ]
> @@ -1119,9 +1130,9 @@ static int decode_modrm(struct x86_emulate_ctxt *ctxt,
> int rc = X86EMUL_CONTINUE;
> ulong modrm_ea = 0;
>
> - ctxt->modrm_reg = (ctxt->rex_bits & REX_R ? 8 : 0);
> - index_reg = (ctxt->rex_bits & REX_X ? 8 : 0);
> - base_reg = (ctxt->rex_bits & REX_B ? 8 : 0);
> + ctxt->modrm_reg = rex_get_rxb(ctxt->rex_bits, REX_R);
> + index_reg = rex_get_rxb(ctxt->rex_bits, REX_X);
> + base_reg = rex_get_rxb(ctxt->rex_bits, REX_B);
Per Section 3.1.2.1 ("REX2 Prefix") of the Intel APX Architecture
Specification (rev. 9.0):
Note that the R, X and B register identifiers can also address
non-GPR register types, such as vector registers, segment registers,
control registers and debug registers. When any of them does, the
highest-order bits REX2.R4, REX2.X4 or REX2.B4 are generally
ignored, except when the register being addressed is a control or
debug register. Furthermore, if ModRM.Reg addresses a segment
register, both REX2.R4 and REX2.R3 are ignored. For example, using
the REX2 prefix, the instruction "ADDPD xmm1, xmm2/m128" can use all
32 GPRs as the base and/or index registers in its memory operand,
but it still cannot access XMM16 to XMM31 because REX2.R4 and
REX2.B4 are ignored when the R and B register identifiers address
vector registers.
In __decode_register_operand(), unlike the Mmx case (which masks
"reg &= 7;"), the Avx | Sse cases do not mask reg to 15 before setting
op->addr.xmm and calling kvm_read_sse_reg(). If a guest executes a
REX2-prefixed SSE instruction in twobyte_table (e.g., MOVAPS with
REX2.M0=1 and REX2.R4=1 or REX2.B4=1) that traps into the emulator,
reg will be in the range 16..31, hitting the default BUG() in
_kvm_read_sse_reg() / _kvm_write_sse_reg() and crashing the host.
Similarly, em_mov_rm_sreg() and em_mov_sreg_rm() use ctxt->modrm_reg
directly without masking off REX.R / REX2.R3 and REX2.R4, so setting
R3=1 or R4=1 on opcode 0x8C or 0x8E will erroneously inject #UD.
[ ... ]
> @@ -4297,15 +4308,15 @@ static const struct opcode opcode_table[256] = {
> II(ImplicitOps | Stack, em_popf, popf),
> I(ImplicitOps, em_sahf), I(ImplicitOps, em_lahf),
> /* 0xA0 - 0xA7 */
> - I2bv(DstAcc | SrcMem | Mov | MemAbs, em_mov),
> - I2bv(DstMem | SrcAcc | Mov | MemAbs | PageTable, em_mov),
> - I2bv(SrcSI | DstDI | Mov | String | TwoMemOp, em_mov),
> - I2bv(SrcSI | DstDI | String | NoWrite | TwoMemOp, em_cmp_r),
> + I2bv(DstAcc | SrcMem | Mov | MemAbs | NoRex2, em_mov),
> + I2bv(DstMem | SrcAcc | Mov | MemAbs | PageTable | NoRex2, em_mov),
> + I2bv(SrcSI | DstDI | Mov | String | TwoMemOp | NoRex2, em_mov),
> + I2bv(SrcSI | DstDI | String | NoWrite | TwoMemOp | NoRex2, em_cmp_r),
Per Section 3.1.2.1 and Section 3.1.3.3 ("64-Bit Absolute Direct Jump
(JMPABS)") of the Intel APX specification, opcode 0xA1 in legacy map 0 is
an exception to row 0xA* being reserved under REX2; REX2 (with M0=0 and
W=0) followed by 0xA1 encodes JMPABS. Marking 0xA1 with NoRex2 causes KVM
to raise a bogus #UD on a valid JMPABS instruction.
[ ... ]
> @@ -4385,12 +4396,12 @@ static const struct opcode twobyte_table[256] = {
> N, GP(ModRM | DstMem | SrcReg | Mov | Sse | Avx, &pfx_0f_2b),
> N, N, N, N,
> /* 0x30 - 0x3F */
> - II(ImplicitOps | Priv, em_wrmsr, wrmsr),
> - IIP(ImplicitOps, em_rdtsc, rdtsc, check_rdtsc),
> - II(ImplicitOps | Priv, em_rdmsr, rdmsr),
> - IIP(ImplicitOps, em_rdpmc, rdpmc, check_rdpmc),
> - I(ImplicitOps | EmulateOnUD | IsBranch | ShadowStack, em_sysenter),
> - I(ImplicitOps | Priv | EmulateOnUD | IsBranch | ShadowStack, em_sysexit),
> + II(ImplicitOps | Priv | NoRex2, em_wrmsr, wrmsr),
> + IIP(ImplicitOps | NoRex2, em_rdtsc, rdtsc, check_rdtsc),
> + II(ImplicitOps | Priv | NoRex2, em_rdmsr, rdmsr),
> + IIP(ImplicitOps | NoRex2, em_rdpmc, rdpmc, check_rdpmc),
> + I(ImplicitOps | EmulateOnUD | IsBranch | ShadowStack | NoRex2, em_sysenter),
> + I(ImplicitOps | Priv | EmulateOnUD | IsBranch | ShadowStack | NoRex2, em_sysexit),
> N, N,
> N, N, N, N, N, N, N, N,
Per Section 3.1.2.1 of the Intel APX specification, all opcodes in row 0x3*
of legacy map 1 trigger #UD when prefixed with REX2. Leaving 0x36..0x3F as
N (NotImpl) causes KVM to return EMULATION_FAILED to userspace instead of
raising #UD when they are prefixed with REX2.
[ ... ]
> @@ -5001,6 +5012,13 @@ int x86_decode_insn(struct x86_emulate_ctxt *ctxt, void *insn, int insn_len, int
> opcode = opcode_table[ctxt->b];
> }
>
> + /*
> + * Instructions marked with NoRex2 ignore a legacy REX prefix, but #UD
> + * should be raised when prefixed with REX2.
> + */
> + if (opcode.flags & NoRex2 && ctxt->rex_prefix == REX2_PREFIX)
> + opcode.flags = Undefined;
All other opcode attribute checks in x86_decode_insn() are performed on
ctxt->d after the "while (ctxt->d & GroupMask)" loop. Checking NoRex2
before group expansion means NoRex2 will be silently ignored if it is ever
placed on a group table entry. Should this check be moved right after the
GroupMask loop?