[RFC PATCH v2 09/31] KVM: x86: hyperv: Divert intercepted descriptor-table loads
From: David Woodhouse
Date: Fri Nov 13 2026 - 15:18:36 EST
From: David Woodhouse <dwmw@xxxxxxxxxxxx>
The secure kernel arms descriptor-table write intercepts for VTL0 with
HvRegisterCrInterceptControl, and PatchGuard then runs its protected
patching window under an intercepted LIDT. The write must not be
applied: the window would run with a genuinely null IDT and the first
event triple-faults the guest. KVM has no notion of the intercept, so
it loads the IDT.
Add a per-vCPU KVM_CAP_HYPERV_REG_INTERCEPT through which the VMM sets
the control word. While a descriptor-table write bit is armed, the
vendor code traps descriptor-table loads, and the emulator diverts an
armed LGDT or LIDT executed in plane 0 into a
KVM_EXIT_HYPERV_REG_INTERCEPT exit instead of applying it. The exit
carries the HV_X64_REGISTER_* id, the base and limit the guest tried to
load, and the RIP and length of the instruction. RIP is rewound to the
instruction: the higher VTL owns completion, and Windows'
patch-protection probe matches its reflected #GP(0) against the probe
RIP. The diverted write is consumed before x86_emulate_instruction()
returns, so it never outlives the emulation.
api.rst documents the exit and its reg_intercept payload.
VMX traps with SECONDARY_EXEC_DESC; SVM arms the four VMCB
descriptor-table load intercepts as a block to match the all-or-nothing
VMX control, and routes the exits into kvm_emulate_instruction(). A
load the divert check declines is applied by the emulator as before.
Store intercepts stay untrapped, as the control has no bits for them.
The (dis)arming gets its own kvm_x86_ops.update_hv_reg_intercepts(),
touching only the L1 controls, rather than riding on hooks with
lifecycles of their own:
- vmx_set_cr4() owns SECONDARY_EXEC_DESC for UMIP emulation, so a
VTL0 set-then-clear of CR4.UMIP would disarm the very lidt/lgdt trap
this exists for. The CR4 path preserves the control while the
intercepts are armed.
- On VMX the op defers while vmcs02 is loaded, and nested_vmx_vmexit()
refreshes vmcs01 after switching back, rather than writing vmcs02
state that the next nested VM-exit would throw away.
- On SVM it arms vmcb01, and nested_vmcb02_recalc_intercepts() strips
the LOAD_* bits from vmcb02 unless vmcb12 wants them, matching the
VMX DESC strip. hv_divert_dt_write() bails in guest mode, so an L2
descriptor exit that still reaches the emulator (UMIP emulation,
for example) applies the load normally.
- init_vmcb() re-applies the SVM intercepts, since reg_intercept_control
survives a vCPU reset: a guest SHUTDOWN clears the VMCB and resets
the vCPU, and must not leave them disarmed behind the higher VTL's
back.
Arming is refused with -EINVAL where the diverted load cannot be
emulated, before anything is armed; disarming always succeeds:
- TDX does not allow descriptor-table loads to be intercepted.
- On SEV-ES, desc_interception() hits svm_check_emulate_instruction()
returning X86EMUL_RETRY_INSTR, which resumes at the same RIP with
the intercept still armed, so one descriptor-table load would spin a
vCPU (and a host CPU) forever.
- On plain SEV, the emulator has no instruction bytes and would inject
#UD on an architecturally valid lgdt.
For the same reason KVM_CAP_VM_MOVE_ENC_CONTEXT_FROM refuses to turn a
VM into an SEV guest while any of its vCPUs has the intercepts armed;
the VMM can disarm first. A failure to re-apply the intercepts after
that is a broken invariant, so svm_vcpu_after_set_cpuid() and
init_vmcb() warn and disarm rather than leave reg_intercept_control
claiming intercepts the VMCB does not have.
The cap stays refused on plane fds, like the other unaudited vCPU caps:
the divert only ever applies to plane 0.
Signed-off-by: David Woodhouse <dwmw@xxxxxxxxxxxx>
---
Documentation/virt/kvm/api.rst | 19 +++++
arch/x86/include/asm/kvm-x86-ops.h | 1 +
arch/x86/include/asm/kvm_host.h | 7 ++
arch/x86/kvm/hyperv.h | 40 ++++++++++
arch/x86/kvm/svm/nested.c | 16 ++++
arch/x86/kvm/svm/sev.c | 24 ++++++
arch/x86/kvm/svm/svm.c | 90 ++++++++++++++++++++++
arch/x86/kvm/vmx/main.c | 13 ++++
arch/x86/kvm/vmx/nested.c | 1 +
arch/x86/kvm/vmx/vmx.c | 65 +++++++++++++++-
arch/x86/kvm/vmx/x86_ops.h | 1 +
arch/x86/kvm/x86.c | 120 ++++++++++++++++++++++++++++-
include/uapi/linux/kvm.h | 11 +++
13 files changed, 401 insertions(+), 7 deletions(-)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index b76c46cc5000..83daee450284 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -7448,6 +7448,7 @@ EOI was received.
#define KVM_EXIT_HYPERV_HCALL 2
#define KVM_EXIT_HYPERV_SYNDBG 3
#define KVM_EXIT_HYPERV_VTL_INTERRUPT 4
+ #define KVM_EXIT_HYPERV_REG_INTERCEPT 5
__u32 type;
__u32 plane;
union {
@@ -7465,6 +7466,15 @@ EOI was received.
#define KVM_HYPERV_HCALL_UD (1ULL << 0)
__u64 flags;
} hcall;
+ struct {
+ __u32 reg;
+ __u8 insn_len;
+ __u8 pad[3];
+ __u64 base;
+ __u32 limit;
+ __u32 pad2;
+ __u64 rip;
+ } reg_intercept;
struct {
__u32 msr;
__u32 pad2;
@@ -7534,6 +7544,15 @@ single-shot: if the VMM declines the entry, the event waits in the
target plane for a cooperative entry rather than re-triggering the
exit.
+ - KVM_EXIT_HYPERV_REG_INTERCEPT -- a descriptor-table write that a
+
+higher plane intercepted (KVM_CAP_HYPERV_REG_INTERCEPT) has been
+diverted to userspace instead of being applied. 'reg_intercept'
+carries the HV_X64_REGISTER_* id of the written register, the
+descriptor-table base and limit the guest tried to load, and the RIP
+and length of the writing instruction. The vCPU's RIP has been
+rewound to that instruction; the higher plane owns its completion.
+
::
/* KVM_EXIT_ARM_NISV / KVM_EXIT_ARM_LDST64B */
diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h
index c50d7ba07ff4..1b56c290c412 100644
--- a/arch/x86/include/asm/kvm-x86-ops.h
+++ b/arch/x86/include/asm/kvm-x86-ops.h
@@ -143,6 +143,7 @@ KVM_X86_OP(get_feature_msr)
KVM_X86_OP(check_emulate_instruction)
KVM_X86_OP(apic_init_signal_blocked)
KVM_X86_OP_OPTIONAL(enable_l2_tlb_flush)
+KVM_X86_OP(update_hv_reg_intercepts)
KVM_X86_OP_OPTIONAL(migrate_timers)
KVM_X86_OP(recalc_intercepts)
KVM_X86_OP(complete_emulated_msr)
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 74899a27e29e..fe8fddab6039 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1803,6 +1803,13 @@ struct kvm_x86_ops {
bool (*apic_init_signal_blocked)(struct kvm_vcpu *vcpu);
int (*enable_l2_tlb_flush)(struct kvm_vcpu *vcpu);
+ /*
+ * Bring the L1 (vmcs01/vmcb01) descriptor-table intercepts in line
+ * with hv_vcpu->reg_intercept_control. Never touches L2 controls;
+ * safe to call at any point, including with a vCPU in guest mode.
+ */
+ int (*update_hv_reg_intercepts)(struct kvm_vcpu *vcpu);
+
void (*migrate_timers)(struct kvm_vcpu *vcpu);
void (*recalc_intercepts)(struct kvm_vcpu *vcpu);
int (*complete_emulated_msr)(struct kvm_vcpu *vcpu, int err);
diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h
index fbfbe0d52f9c..069c59788bf6 100644
--- a/arch/x86/kvm/hyperv.h
+++ b/arch/x86/kvm/hyperv.h
@@ -116,6 +116,21 @@ struct kvm_vcpu_hv {
u32 vp_id;
} nested;
+ /*
+ * HvRegisterCrInterceptControl value the higher VTL armed for the
+ * lower VTL (userspace VSM): descriptor-table write bits divert
+ * the emulated write into a KVM_EXIT_HYPERV_REG_INTERCEPT exit.
+ */
+ u64 reg_intercept_control;
+ struct {
+ u32 reg;
+ u8 insn_len;
+ bool pending;
+ u64 base;
+ u32 limit;
+ u64 rip;
+ } reg_intercept;
+
/*
* RAX/RCX of the lower VTL captured at VTL entry, restored on a
* non-fast VTL return when this plane has no VP assist page to
@@ -197,6 +212,26 @@ static inline struct kvm_vcpu_hv *to_hv_vcpu(struct kvm_vcpu *vcpu)
return vcpu->arch.hyperv;
}
+/*
+ * True when a higher VTL armed descriptor-table write intercepts
+ * (HvRegisterCrInterceptControl) on this vCPU, i.e. when the vendor code
+ * must trap the L1 guest's descriptor-table loads for hv_divert_dt_write().
+ */
+static inline bool kvm_hv_dt_intercepts_armed(struct kvm_vcpu *vcpu)
+{
+ struct kvm_vcpu_hv *hv_vcpu = to_hv_vcpu(vcpu);
+
+ return hv_vcpu && hv_vcpu->reg_intercept_control;
+}
+
+static inline void kvm_hv_disarm_dt_intercepts(struct kvm_vcpu *vcpu)
+{
+ struct kvm_vcpu_hv *hv_vcpu = to_hv_vcpu(vcpu);
+
+ if (hv_vcpu)
+ hv_vcpu->reg_intercept_control = 0;
+}
+
static inline struct kvm_vcpu_hv_synic *to_hv_synic(struct kvm_vcpu *vcpu)
{
struct kvm_vcpu_hv *hv_vcpu = to_hv_vcpu(vcpu);
@@ -434,6 +469,11 @@ static inline bool kvm_hv_hcall_feature(struct kvm *kvm, u64 feature)
{
return false;
}
+static inline bool kvm_hv_dt_intercepts_armed(struct kvm_vcpu *vcpu)
+{
+ return false;
+}
+static inline void kvm_hv_disarm_dt_intercepts(struct kvm_vcpu *vcpu) {}
static inline void kvm_hv_setup_tsc_page(struct kvm *kvm,
struct pvclock_vcpu_time_info *hv_clock) {}
static inline void kvm_hv_request_tsc_page_update(struct kvm *kvm) {}
diff --git a/arch/x86/kvm/svm/nested.c b/arch/x86/kvm/svm/nested.c
index 75b02a84f2df..1836828436a9 100644
--- a/arch/x86/kvm/svm/nested.c
+++ b/arch/x86/kvm/svm/nested.c
@@ -172,6 +172,22 @@ void nested_vmcb02_recalc_intercepts(struct vcpu_svm *svm)
for (i = 0; i < MAX_INTERCEPT; i++)
vmcb02->control.intercepts[i] |= vmcb12_ctrl->intercepts[i];
+ /*
+ * KVM arms the descriptor-table load intercepts in vmcb01 to divert
+ * L1's loads into Hyper-V register-intercept exits (see
+ * hv_divert_dt_write()); they do not apply to L2. Drop them from
+ * vmcb02 unless L1 wants the intercept itself, matching the
+ * SECONDARY_EXEC_DESC strip in VMX's prepare_vmcs02().
+ */
+ if (!vmcb12_is_intercept(vmcb12_ctrl, INTERCEPT_LOAD_IDTR))
+ vmcb_clr_intercept(&vmcb02->control, INTERCEPT_LOAD_IDTR);
+ if (!vmcb12_is_intercept(vmcb12_ctrl, INTERCEPT_LOAD_GDTR))
+ vmcb_clr_intercept(&vmcb02->control, INTERCEPT_LOAD_GDTR);
+ if (!vmcb12_is_intercept(vmcb12_ctrl, INTERCEPT_LOAD_LDTR))
+ vmcb_clr_intercept(&vmcb02->control, INTERCEPT_LOAD_LDTR);
+ if (!vmcb12_is_intercept(vmcb12_ctrl, INTERCEPT_LOAD_TR))
+ vmcb_clr_intercept(&vmcb02->control, INTERCEPT_LOAD_TR);
+
/* If SMI is not intercepted, ignore guest SMI intercept as well */
if (!intercept_smi)
vmcb_clr_intercept(&vmcb02->control, INTERCEPT_SMI);
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index a63822defc3b..79e78c7f56b3 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -35,6 +35,7 @@
#include "svm.h"
#include "svm_ops.h"
#include "cpuid.h"
+#include "hyperv.h"
#include "trace.h"
#include "lapic.h"
@@ -2161,6 +2162,25 @@ static int sev_check_source_vcpus(struct kvm *dst, struct kvm *src)
return 0;
}
+/*
+ * The destination becomes an SEV guest, which cannot have the Hyper-V
+ * descriptor-table register intercepts (svm_update_hv_reg_intercepts()
+ * refuses to arm them). Refuse the move while any destination vCPU has
+ * them armed; the VMM can disarm first.
+ */
+static int sev_check_dest_reg_intercepts(struct kvm *dst)
+{
+ struct kvm_vcpu *vcpu;
+ unsigned long i;
+
+ kvm_for_each_vcpu(i, vcpu, dst) {
+ if (kvm_hv_dt_intercepts_armed(vcpu))
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd)
{
struct kvm_sev_info *dst_sev = to_kvm_sev_info(kvm);
@@ -2210,6 +2230,10 @@ int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd)
if (ret)
goto out_source_vcpu;
+ ret = sev_check_dest_reg_intercepts(kvm);
+ if (ret)
+ goto out_source_vcpu;
+
/*
* Allocate a new have_run_cpus for the destination, i.e. don't copy
* the set of CPUs from the source. If a CPU was used to run a vCPU in
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 5f07ddfafca4..b87c12944108 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -1142,6 +1142,65 @@ static void svm_recalc_intercepts(struct kvm_vcpu *vcpu)
svm_recalc_msr_intercepts(vcpu);
}
+static int svm_update_hv_reg_intercepts(struct kvm_vcpu *vcpu)
+{
+ struct vcpu_svm *svm = to_svm(vcpu);
+
+ /*
+ * Trap descriptor-table loads while a higher VTL has register
+ * intercepts armed on them (reg_intercept_control), and stop
+ * trapping when they are disarmed; the emulator diverts a trapped
+ * load into a register-intercept exit (hv_divert_dt_write()).
+ * VMX arms the all-or-nothing SECONDARY_EXEC_DESC; the SVM
+ * intercepts are per-register but armed as a block to match, and
+ * an exit the emulator does not divert applies the load normally.
+ *
+ * The intercepts are armed in vmcb01 only: an L2 descriptor load
+ * is never diverted, and nested_vmcb02_recalc_intercepts() keeps
+ * these bits out of vmcb02 unless L1 asked for them itself.
+ */
+ if (kvm_hv_dt_intercepts_armed(vcpu)) {
+ /*
+ * The diverted load needs full emulation, which encrypted
+ * guests cannot have: on SEV-ES and SNP the emulator cannot
+ * read the registers, so svm_check_emulate_instruction()
+ * retries at the same RIP with the intercept still armed and
+ * every descriptor-table load livelocks the vCPU; on plain
+ * SEV there are no instruction bytes to decode and a valid
+ * lgdt takes a spurious #UD. Refuse to arm.
+ */
+ if (is_sev_guest(vcpu))
+ return -EINVAL;
+
+ svm_set_intercept(svm, INTERCEPT_LOAD_IDTR);
+ svm_set_intercept(svm, INTERCEPT_LOAD_GDTR);
+ svm_set_intercept(svm, INTERCEPT_LOAD_LDTR);
+ svm_set_intercept(svm, INTERCEPT_LOAD_TR);
+ } else {
+ svm_clr_intercept(svm, INTERCEPT_LOAD_IDTR);
+ svm_clr_intercept(svm, INTERCEPT_LOAD_GDTR);
+ svm_clr_intercept(svm, INTERCEPT_LOAD_LDTR);
+ svm_clr_intercept(svm, INTERCEPT_LOAD_TR);
+ }
+ return 0;
+}
+
+/*
+ * Re-apply the descriptor-table intercepts from reg_intercept_control.
+ * Arming is refused up front for guests that cannot have it (the
+ * enable-cap and sev_vm_move_enc_context_from() both check), so a failure
+ * here is a broken invariant: disarm, rather than leave the controls and
+ * reg_intercept_control disagreeing.
+ */
+static void svm_refresh_hv_reg_intercepts(struct kvm_vcpu *vcpu)
+{
+ if (!WARN_ON_ONCE(svm_update_hv_reg_intercepts(vcpu)))
+ return;
+
+ kvm_hv_disarm_dt_intercepts(vcpu);
+ svm_update_hv_reg_intercepts(vcpu);
+}
+
static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event)
{
struct vcpu_svm *svm = to_svm(vcpu);
@@ -1308,6 +1367,14 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event)
svm_hv_init_vmcb(vmcb);
+ /*
+ * The descriptor-table intercepts follow reg_intercept_control,
+ * which a reset does not touch: re-apply them, or a SHUTDOWN (which
+ * clears the VMCB and resets the vCPU) leaves them disarmed while
+ * the higher VTL still believes them armed.
+ */
+ svm_refresh_hv_reg_intercepts(vcpu);
+
kvm_make_request(KVM_REQ_RECALC_INTERCEPTS, vcpu);
vmcb_mark_all_dirty(vmcb);
@@ -2087,6 +2154,17 @@ static int npf_interception(struct kvm_vcpu *vcpu)
return rc;
}
+static int desc_interception(struct kvm_vcpu *vcpu)
+{
+ /*
+ * Descriptor-table write exits are armed only while a higher VTL
+ * has register intercepts on them (reg_intercept_control); the
+ * emulator diverts the write into a register-intercept exit via
+ * hv_divert_dt_write() instead of applying it.
+ */
+ return kvm_emulate_instruction(vcpu, 0);
+}
+
static int db_interception(struct kvm_vcpu *vcpu)
{
struct kvm_run *kvm_run = vcpu->run;
@@ -3461,6 +3539,10 @@ static int (*const svm_exit_handlers[])(struct kvm_vcpu *vcpu) = {
[SVM_EXIT_WRITE_CR3] = cr_interception,
[SVM_EXIT_WRITE_CR4] = cr_interception,
[SVM_EXIT_WRITE_CR8] = cr8_write_interception,
+ [SVM_EXIT_IDTR_WRITE] = desc_interception,
+ [SVM_EXIT_GDTR_WRITE] = desc_interception,
+ [SVM_EXIT_LDTR_WRITE] = desc_interception,
+ [SVM_EXIT_TR_WRITE] = desc_interception,
[SVM_EXIT_READ_DR0] = dr_interception,
[SVM_EXIT_READ_DR1] = dr_interception,
[SVM_EXIT_READ_DR2] = dr_interception,
@@ -4931,6 +5013,13 @@ static void svm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
if (guest_cpuid_is_intel_compatible(vcpu))
guest_cpu_cap_clear(vcpu->common, X86_FEATURE_V_VMSAVE_VMLOAD);
+ /*
+ * Re-arm the descriptor-table intercepts; setting guest CPUID does
+ * not change reg_intercept_control, but keep the controls in sync
+ * the same way vmx_vcpu_after_set_cpuid() does.
+ */
+ svm_refresh_hv_reg_intercepts(vcpu);
+
if (is_sev_guest(vcpu))
sev_vcpu_after_set_cpuid(svm);
}
@@ -5681,6 +5770,7 @@ struct kvm_x86_ops svm_x86_ops __initdata = {
.apic_init_signal_blocked = svm_apic_init_signal_blocked,
.recalc_intercepts = svm_recalc_intercepts,
+ .update_hv_reg_intercepts = svm_update_hv_reg_intercepts,
.complete_emulated_msr = svm_complete_emulated_msr,
.vcpu_deliver_sipi_vector = svm_vcpu_deliver_sipi_vector,
diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c
index 14d21d02bff4..6efbb7062201 100644
--- a/arch/x86/kvm/vmx/main.c
+++ b/arch/x86/kvm/vmx/main.c
@@ -211,6 +211,18 @@ static void vt_recalc_intercepts(struct kvm_vcpu *vcpu)
vmx_recalc_intercepts(vcpu);
}
+static int vt_update_hv_reg_intercepts(struct kvm_vcpu *vcpu)
+{
+ /*
+ * TDX doesn't allow descriptor-table loads to be intercepted, so
+ * arming fails; disarming has nothing to undo and succeeds.
+ */
+ if (is_td_vcpu(vcpu))
+ return kvm_hv_dt_intercepts_armed(vcpu) ? -EINVAL : 0;
+
+ return vmx_update_hv_reg_intercepts(vcpu);
+}
+
static int vt_complete_emulated_msr(struct kvm_vcpu *vcpu, int err)
{
if (is_td_vcpu(vcpu))
@@ -1147,6 +1159,7 @@ struct kvm_x86_ops vt_x86_ops __initdata = {
.check_emulate_instruction = vt_op(check_emulate_instruction),
.apic_init_signal_blocked = vt_op(apic_init_signal_blocked),
+ .update_hv_reg_intercepts = vt_op(update_hv_reg_intercepts),
.migrate_timers = vmx_migrate_timers,
.recalc_intercepts = vt_op(recalc_intercepts),
diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c
index c980ecf9202a..a33b6bd0e8a4 100644
--- a/arch/x86/kvm/vmx/nested.c
+++ b/arch/x86/kvm/vmx/nested.c
@@ -5177,6 +5177,7 @@ void __nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 vm_exit_reason,
* Update any VMCS fields that might have changed while vmcs02 was the
* active VMCS. The tracking is per-vCPU, not per-VMCS.
*/
+ vmx_update_hv_reg_intercepts(vcpu);
vmcs_write32(VM_EXIT_MSR_STORE_COUNT, vmx->msr_autostore.nr);
vmcs_write32(VM_EXIT_MSR_LOAD_COUNT, vmx->msr_autoload.host.nr);
vmcs_write32(VM_ENTRY_MSR_LOAD_COUNT, vmx->msr_autoload.guest.nr);
diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
index 2d85205cebb4..4e7eec9a0625 100644
--- a/arch/x86/kvm/vmx/vmx.c
+++ b/arch/x86/kvm/vmx/vmx.c
@@ -3595,6 +3595,42 @@ bool vmx_is_valid_cr4(struct kvm_vcpu *vcpu, unsigned long cr4)
return true;
}
+/*
+ * SECONDARY_EXEC_DESC has two consumers: UMIP emulation (descriptor-table
+ * reads must #GP when CR4.UMIP is set and the CPU cannot do it) and the
+ * Hyper-V VTL register intercepts (descriptor-table writes divert into
+ * KVM_EXIT_HYPERV_REG_INTERCEPT, see hv_divert_dt_write()). Every arm or
+ * disarm of the control for one consumer must preserve the other's claim.
+ */
+static bool vmx_need_desc_exiting(struct kvm_vcpu *vcpu)
+{
+ return (vmx_umip_emulated() &&
+ kvm_is_cr4_bit_set(vcpu, X86_CR4_UMIP)) ||
+ kvm_hv_dt_intercepts_armed(vcpu);
+}
+
+int vmx_update_hv_reg_intercepts(struct kvm_vcpu *vcpu)
+{
+ struct vcpu_vmx *vmx = to_vmx(vcpu);
+
+ if (!(vmcs_config.cpu_based_2nd_exec_ctrl & SECONDARY_EXEC_DESC))
+ return 0;
+
+ /*
+ * Only vmcs01 carries the register intercepts: prepare_vmcs02()
+ * strips SECONDARY_EXEC_DESC unless vmcs12 or L2's UMIP emulation
+ * wants it, so an L2 descriptor load is never diverted. While
+ * vmcs02 is loaded, leave it alone; nested_vmx_vmexit() refreshes
+ * vmcs01 after switching back.
+ */
+ if (vmx->loaded_vmcs != &vmx->vmcs01)
+ return 0;
+
+ secondary_exec_controls_changebit(vmx, SECONDARY_EXEC_DESC,
+ vmx_need_desc_exiting(vcpu));
+ return 0;
+}
+
void vmx_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4)
{
unsigned long old_cr4 = kvm_read_cr4(vcpu);
@@ -3618,8 +3654,16 @@ void vmx_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4)
if (cr4 & X86_CR4_UMIP) {
secondary_exec_controls_setbit(vmx, SECONDARY_EXEC_DESC);
hw_cr4 &= ~X86_CR4_UMIP;
- } else if (!is_guest_mode(vcpu) ||
- !nested_cpu_has2(get_vmcs12(vcpu), SECONDARY_EXEC_DESC)) {
+ } else if (is_guest_mode(vcpu)) {
+ if (!nested_cpu_has2(get_vmcs12(vcpu), SECONDARY_EXEC_DESC))
+ secondary_exec_controls_clearbit(vmx,
+ SECONDARY_EXEC_DESC);
+ } else if (!kvm_hv_dt_intercepts_armed(vcpu)) {
+ /*
+ * Clearing CR4.UMIP must not disarm the control
+ * while a higher VTL holds register intercepts on
+ * the descriptor tables.
+ */
secondary_exec_controls_clearbit(vmx, SECONDARY_EXEC_DESC);
}
}
@@ -5698,7 +5742,14 @@ static int handle_desc(struct kvm_vcpu *vcpu)
*/
BUILD_BUG_ON(KVM_POSSIBLE_CR4_GUEST_BITS & X86_CR4_UMIP);
- WARN_ON_ONCE(!kvm_is_cr4_bit_set(vcpu, X86_CR4_UMIP));
+ /*
+ * Descriptor-table exits are also armed when a higher VTL registers
+ * DT intercepts (reg_intercept_control); only an exit with neither
+ * UMIP emulation nor an armed intercept indicates a stale
+ * SECONDARY_EXEC_DESC control.
+ */
+ WARN_ON_ONCE(!kvm_is_cr4_bit_set(vcpu, X86_CR4_UMIP) &&
+ !kvm_hv_dt_intercepts_armed(vcpu));
return kvm_emulate_instruction(vcpu, 0);
}
@@ -7916,6 +7967,14 @@ void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)
/* Refresh #PF interception to account for MAXPHYADDR changes. */
vmx_update_exception_bitmap(vcpu);
+
+ /*
+ * Bring descriptor-table exiting in line with the VTL register
+ * intercepts and UMIP emulation. The secondary controls set above
+ * leave SECONDARY_EXEC_DESC alone, so this is a consistency refresh
+ * rather than a repair.
+ */
+ vmx_update_hv_reg_intercepts(vcpu);
}
static __init u64 vmx_get_perf_capabilities(void)
diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h
index 63e7b9256152..2e1dc101a5a6 100644
--- a/arch/x86/kvm/vmx/x86_ops.h
+++ b/arch/x86/kvm/vmx/x86_ops.h
@@ -53,6 +53,7 @@ void vmx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode,
void vmx_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu);
bool vmx_has_emulated_msr(struct kvm *kvm, u32 index);
void vmx_recalc_intercepts(struct kvm_vcpu *vcpu);
+int vmx_update_hv_reg_intercepts(struct kvm_vcpu *vcpu);
void vmx_prepare_switch_to_guest(struct kvm_vcpu *vcpu);
void vmx_update_exception_bitmap(struct kvm_vcpu *vcpu);
int vmx_get_feature_msr(u32 msr, u64 *data);
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 0933291298e0..0fda04f6114a 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2368,6 +2368,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
case KVM_CAP_HYPERV_CPUID:
case KVM_CAP_HYPERV_ENFORCE_CPUID:
case KVM_CAP_SYS_HYPERV_CPUID:
+ case KVM_CAP_HYPERV_REG_INTERCEPT:
#endif
case KVM_CAP_HCALL_FAULT_EXIT:
case KVM_CAP_PCI_SEGMENT:
@@ -3501,9 +3502,10 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
/*
* KVM_ENABLE_CAP is accepted on plane vCPU fds, but only the caps
* below have been audited for plane_level > 0. The nested paths
- * behind eVMCS and direct TLB flush have not; refuse those rather
- * than implicitly extending every current and future vCPU cap to
- * planes.
+ * behind eVMCS and direct TLB flush have not, and the register
+ * intercept only ever diverts plane 0 (hv_divert_dt_write() skips
+ * higher planes); refuse those rather than implicitly extending
+ * every current and future vCPU cap to planes.
*/
if (vcpu->plane_level) {
switch (cap->cap) {
@@ -3547,6 +3549,26 @@ static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
}
return r;
}
+ case KVM_CAP_HYPERV_REG_INTERCEPT: {
+ struct kvm_vcpu_hv *hv_vcpu;
+ int r;
+
+ r = kvm_hv_vcpu_init(vcpu);
+ if (r)
+ return r;
+ hv_vcpu = to_hv_vcpu(vcpu);
+ hv_vcpu->reg_intercept_control = cap->args[0];
+ /*
+ * Refresh the descriptor-table exiting controls. The vendor
+ * op touches only the L1 controls, so it is safe here even
+ * when the vCPU last exited from a nested guest and
+ * vcpu_load() restored vmcs02/vmcb02.
+ */
+ r = kvm_x86_call(update_hv_reg_intercepts)(vcpu);
+ if (r)
+ hv_vcpu->reg_intercept_control = 0;
+ return r;
+ }
case KVM_CAP_HYPERV_DIRECT_TLBFLUSH:
if (!kvm_x86_ops.enable_l2_tlb_flush)
return -ENOTTY;
@@ -5879,14 +5901,57 @@ static void emulator_get_idt(struct x86_emulate_ctxt *ctxt, struct desc_ptr *dt)
kvm_x86_call(get_idt)(emul_to_vcpu(ctxt), dt);
}
+/*
+ * Divert a descriptor-table write into a register-intercept exit when the
+ * higher VTL armed one: the write is not applied, and userspace delivers
+ * the TLFS register-intercept message to the higher VTL. The intercept
+ * contract covers only the L1 guest; a descriptor load emulated on behalf
+ * of a nested (L2) guest, however its exit arrived, is applied normally.
+ */
+static bool hv_divert_dt_write(struct x86_emulate_ctxt *ctxt, u32 hv_reg,
+ const struct desc_ptr *dt, u64 ctl_bit)
+{
+#ifdef CONFIG_KVM_HYPERV
+ struct kvm_vcpu *vcpu = emul_to_vcpu(ctxt);
+ struct kvm_vcpu_hv *hv_vcpu = to_hv_vcpu(vcpu);
+
+ if (!hv_vcpu || vcpu->plane_level || is_guest_mode(vcpu) ||
+ !(hv_vcpu->reg_intercept_control & ctl_bit))
+ return false;
+
+ hv_vcpu->reg_intercept.reg = hv_reg;
+ hv_vcpu->reg_intercept.base = dt->address;
+ hv_vcpu->reg_intercept.limit = dt->size;
+ hv_vcpu->reg_intercept.rip = ctxt->eip;
+ hv_vcpu->reg_intercept.insn_len = ctxt->_eip - ctxt->eip;
+ hv_vcpu->reg_intercept.pending = true;
+ return true;
+#else
+ return false;
+#endif
+}
+
+#define HV_CR_INTERCEPT_GDTR_WRITE BIT_ULL(15)
+#define HV_CR_INTERCEPT_IDTR_WRITE BIT_ULL(16)
+#define HV_X64_REGISTER_IDTR_ID 0x00070000
+#define HV_X64_REGISTER_GDTR_ID 0x00070001
+
static void emulator_set_gdt(struct x86_emulate_ctxt *ctxt, struct desc_ptr *dt)
{
+ if (hv_divert_dt_write(ctxt, HV_X64_REGISTER_GDTR_ID, dt,
+ HV_CR_INTERCEPT_GDTR_WRITE))
+ return;
kvm_x86_call(set_gdt)(emul_to_vcpu(ctxt), dt);
}
static void emulator_set_idt(struct x86_emulate_ctxt *ctxt, struct desc_ptr *dt)
{
- kvm_x86_call(set_idt)(emul_to_vcpu(ctxt), dt);
+ struct kvm_vcpu *vcpu = emul_to_vcpu(ctxt);
+
+ if (hv_divert_dt_write(ctxt, HV_X64_REGISTER_IDTR_ID, dt,
+ HV_CR_INTERCEPT_IDTR_WRITE))
+ return;
+ kvm_x86_call(set_idt)(vcpu, dt);
}
static unsigned long emulator_get_cached_segment_base(
@@ -6630,6 +6695,44 @@ int x86_decode_emulated_instruction(struct kvm_vcpu *vcpu, int emulation_type,
}
EXPORT_SYMBOL_FOR_KVM_INTERNAL(x86_decode_emulated_instruction);
+#ifdef CONFIG_KVM_HYPERV
+/*
+ * Complete a descriptor-table write that hv_divert_dt_write() diverted
+ * during the emulation that just finished: rewind RIP and build the
+ * KVM_EXIT_HYPERV_REG_INTERCEPT exit. Consuming the record before
+ * x86_emulate_instruction() returns means it never outlives the emulation
+ * call: no stale rewind can fire on a later KVM_RUN, and disarming the cap
+ * or migrating the vCPU can never see a half-delivered intercept.
+ */
+static int complete_hv_reg_intercept(struct kvm_vcpu *vcpu)
+{
+ struct kvm_vcpu_hv *hv_vcpu = to_hv_vcpu(vcpu);
+
+ hv_vcpu->reg_intercept.pending = false;
+
+ /*
+ * The intercepted instruction must not retire: the emulator advanced
+ * RIP when it diverted the write, but the higher VTL owns
+ * completion. It either applies the write and advances RIP itself,
+ * disarms the intercept so re-execution succeeds, or queues an
+ * exception that must fault AT the instruction (the
+ * kernel-patch-protection LIDT probe matches the faulting RIP
+ * against the probe site).
+ */
+ kvm_rip_write(vcpu, hv_vcpu->reg_intercept.rip);
+ vcpu->run->exit_reason = KVM_EXIT_HYPERV;
+ vcpu->run->hyperv.type = KVM_EXIT_HYPERV_REG_INTERCEPT;
+ vcpu->run->hyperv.plane = vcpu->plane_level;
+ vcpu->run->hyperv.u.reg_intercept.reg = hv_vcpu->reg_intercept.reg;
+ vcpu->run->hyperv.u.reg_intercept.insn_len =
+ hv_vcpu->reg_intercept.insn_len;
+ vcpu->run->hyperv.u.reg_intercept.base = hv_vcpu->reg_intercept.base;
+ vcpu->run->hyperv.u.reg_intercept.limit = hv_vcpu->reg_intercept.limit;
+ vcpu->run->hyperv.u.reg_intercept.rip = hv_vcpu->reg_intercept.rip;
+ return 0;
+}
+#endif
+
int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
int emulation_type, void *insn, int insn_len)
{
@@ -6861,6 +6964,15 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
} else
vcpu->arch.emulate_regs_need_sync_to_vcpu = true;
+#ifdef CONFIG_KVM_HYPERV
+ {
+ struct kvm_vcpu_hv *hv_vcpu = to_hv_vcpu(vcpu);
+
+ if (hv_vcpu && hv_vcpu->reg_intercept.pending)
+ r = complete_hv_reg_intercept(vcpu);
+ }
+#endif
+
return r;
}
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 694409459cb8..b260d4d8aa22 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -100,6 +100,7 @@ struct kvm_hyperv_exit {
#define KVM_EXIT_HYPERV_HCALL 2
#define KVM_EXIT_HYPERV_SYNDBG 3
#define KVM_EXIT_HYPERV_VTL_INTERRUPT 4
+#define KVM_EXIT_HYPERV_REG_INTERCEPT 5
__u32 type;
__u32 plane;
union {
@@ -117,6 +118,15 @@ struct kvm_hyperv_exit {
#define KVM_HYPERV_HCALL_UD (1ULL << 0)
__u64 flags;
} hcall;
+ struct {
+ __u32 reg;
+ __u8 insn_len;
+ __u8 pad[3];
+ __u64 base;
+ __u32 limit;
+ __u32 pad2;
+ __u64 rip;
+ } reg_intercept;
struct {
__u32 msr;
__u32 pad2;
@@ -1032,6 +1042,7 @@ struct kvm_enable_cap {
#define KVM_CAP_PLANES 254
#define KVM_CAP_HYPERV_FORWARD_HCALL 255
#define KVM_CAP_PLANES_SHARE 256
+#define KVM_CAP_HYPERV_REG_INTERCEPT 257
struct kvm_irq_routing_irqchip {
__u32 irqchip;
--
2.50.1