Re: [PATCH v3 06/12] KVM: x86: Forward VMware hypercalls to userspace
From: David Woodhouse
Date: Fri Nov 13 2026 - 03:07:41 EST
On Thu, 2026-11-12 at 20:05 -0500, Zack Rusin wrote:
>
> > And I wonder if the cleaner answer in both cases is just to give
> > userspace a way to cancel the pending hypercall completion and use
> > KVM_SET_VCPU_EVENTS instead of piling all this stuff into the hcall
> > structs? It could even just be a KVM_VCPUEVENT_ flag to make it cancel
> > the hypercall completion?
>
> I agree that this would be cleaner, and if KVM grows such a flag,
> VMware userspace could use it too. But it's new API in
> KVM_SET_VCPU_EVENTS, and we'd need to agree which completions it may
> cancel: simply clearing the callback leaves pending I/O state behind
> for PIO and MMIO, and KVM_EXIT_HYPERCALL also covers TDX completions
> with different semantics. So I'd rather not make this series depend
> on it because it significantly increases its scope.
I won't necessarily defend my suggestion above to the hilt on its
technical merit, but I'll observe that as a counter-argument, yours
*really* isn't a good one. We don't design a userspace ABI based on the
fact that we're impatient.
To your concern, I actually think we could get away with aborting *any*
pending complete_userspace_io callback. Userspace *already* needed to
know about those¹ because it can't migrate while those are pending; it
has to do at least one more run, even if it's with the immediate_exit
flag set.
If it's deemed clearer, I'd also be OK with an explicit
KVM_VCPUEVENT_CANCEL_HCALL which cancels *only* hypercalls. But I don't
really see the point; userspace shouldn't be getting confused when it
was actually some other kind of completion that was active anyway.
¹ https://lore.kernel.org/all/20250111012450.1262638-1-seanjc@xxxxxxxxxx/
Looks something like this in my Hyper-V hcall series (except I already
shouted at the agent for stupid struct layout and made it go run pahole
on everything it's working on).
From bc25df8c84af07f7ddf74dd762ba35adbe5fba75 Mon Sep 17 00:00:00 2001
From: David Woodhouse <dwmw@xxxxxxxxxxxx>
Date: Fri, 13 Nov 2026 04:39:31 +0000
Subject: [PATCH] KVM: x86: Let userspace cancel a pending hypercall completion
A VMM implementing Hyper-V hypercalls behind KVM_EXIT_HYPERV_HCALL can
only complete them: write a status, advance RIP. The TLFS specifies
some hypercalls to raise an exception at the hypercall instruction
instead, and the exit has no way to say that. The KVM_EXIT_VMWARE
series has the same problem for its backdoor exits, so solve it without
touching the exit format at all: cancel the pending completion, and
queue the exception through the machinery KVM_SET_VCPU_EVENTS already
has.
KVM_VCPUEVENT_CANCEL_HCALL on KVM_SET_VCPU_EVENTS drops the pending
completion: no result write, no RIP advance, so the guest re-executes
the hypercall on its next entry unless the same call queued an
exception to be delivered there. Hyper-V hypercalls are restartable by
design, so both outcomes are architecturally sound. If nothing
cancellable is pending - nothing at all, or a completion KVM armed for
its own purposes, like the MMIO and PIO emulation continuations - the
ioctl fails with EBUSY and changes nothing. Only the Hyper-V hypercall
userspace exits are cancellable.
KVM_SET_VCPU_EVENTS offers no way to probe for a new flag, so
KVM_CHECK_EXTENSION(KVM_CAP_VCPU_EVENTS) grows from 1 into a feature
mask: KVM_VCPU_EVENTS_BASE keeps existing truth-value checks working,
and KVM_VCPU_EVENTS_CANCEL_HCALL advertises the new flag.
Link: https://lore.kernel.org/all/876c4f4fc442116c9bff8abd8c34f99fcfa65531.camel@xxxxxxxxxxxxx/
Signed-off-by: David Woodhouse <dwmw@xxxxxxxxxxxx>
---
Documentation/virt/kvm/api.rst | 18 ++++++++++++++++++
arch/x86/include/asm/kvm_host.h | 2 ++
arch/x86/include/uapi/asm/kvm.h | 7 +++++++
arch/x86/kvm/hyperv.c | 6 ++++++
arch/x86/kvm/x86.c | 29 +++++++++++++++++++++++++++--
5 files changed, 60 insertions(+), 2 deletions(-)
diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index e95c9cda085d..69c5523b8265 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -1302,6 +1302,24 @@ If KVM_CAP_X86_TRIPLE_FAULT_EVENT is enabled, KVM_VCPUEVENT_VALID_TRIPLE_FAULT
can be set in flags field to signal that the triple_fault field contains
a valid state and shall be written into the VCPU.
+On x86, KVM_CHECK_EXTENSION(KVM_CAP_VCPU_EVENTS) returns a feature mask:
+KVM_VCPU_EVENTS_BASE for the two ioctls themselves, and
+KVM_VCPU_EVENTS_CANCEL_HCALL when KVM_SET_VCPU_EVENTS accepts
+KVM_VCPUEVENT_CANCEL_HCALL in the flags field.
+
+KVM_VCPUEVENT_CANCEL_HCALL cancels a pending, cancellable hypercall
+completion, i.e. a KVM_EXIT_HYPERV_HCALL exit that userspace is holding.
+KVM forgets the completion: the hypercall's result is never written and
+its instruction is never retired, so the vCPU re-executes the hypercall
+instruction on its next entry unless the same call queued an exception
+to be delivered there instead. Hyper-V hypercalls are restartable by
+design, so either is architecturally sound; this is how a VMM makes a
+hypercall fault (e.g. #UD, or #PF with the address in
+exception_payload) rather than return a status. If no completion is
+pending, or the pending completion is not cancellable (for example an
+MMIO or PIO emulation continuation), the ioctl fails with EBUSY and
+changes nothing.
+
ARM64:
^^^^^^
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 1ee4b127004d..c2e7e90016fb 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -857,6 +857,8 @@ struct kvm_vcpu_arch {
bool emulate_regs_need_sync_to_vcpu;
bool emulate_regs_need_sync_from_vcpu;
int (*complete_userspace_io)(struct kvm_vcpu *vcpu);
+ /* KVM_SET_VCPU_EVENTS may cancel the pending completion. */
+ bool cui_cancellable;
unsigned long cui_linear_rip;
int cui_rdmsr_imm_reg;
diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h
index 00ce265391e1..a886830f663c 100644
--- a/arch/x86/include/uapi/asm/kvm.h
+++ b/arch/x86/include/uapi/asm/kvm.h
@@ -330,6 +330,13 @@ struct kvm_reinject_control {
#define KVM_VCPUEVENT_VALID_PAYLOAD 0x00000010
#define KVM_VCPUEVENT_VALID_TRIPLE_FAULT 0x00000020
+/* Cancels a pending, cancellable hypercall completion on KVM_SET_VCPU_EVENTS */
+#define KVM_VCPUEVENT_CANCEL_HCALL 0x00000040
+
+/* KVM_CHECK_EXTENSION(KVM_CAP_VCPU_EVENTS) feature bits */
+#define KVM_VCPU_EVENTS_BASE 0x00000001
+#define KVM_VCPU_EVENTS_CANCEL_HCALL 0x00000002
+
/* Interrupt shadow states */
#define KVM_X86_SHADOW_INT_MOV_SS 0x01
#define KVM_X86_SHADOW_INT_STI 0x02
diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
index 5d2e3617c952..549da771d620 100644
--- a/arch/x86/kvm/hyperv.c
+++ b/arch/x86/kvm/hyperv.c
@@ -2712,6 +2712,12 @@ int kvm_hv_hypercall(struct kvm_vcpu *vcpu)
vcpu->run->hyperv.u.hcall.params[0] = hc.ingpa;
vcpu->run->hyperv.u.hcall.params[1] = hc.outgpa;
vcpu->arch.complete_userspace_io = kvm_hv_hypercall_complete_userspace;
+ /*
+ * Userspace may revoke this completion through KVM_SET_VCPU_EVENTS
+ * instead of completing it; the guest then re-executes the hypercall
+ * instruction.
+ */
+ vcpu->arch.cui_cancellable = true;
return 0;
}
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 26e7beaff8a0..20d7940a7e74 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2237,7 +2237,6 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
case KVM_CAP_IOEVENTFD_NO_LENGTH:
case KVM_CAP_SET_IDENTITY_MAP_ADDR:
- case KVM_CAP_VCPU_EVENTS:
#ifdef CONFIG_KVM_HYPERV
case KVM_CAP_HYPERV:
case KVM_CAP_HYPERV_VAPIC:
@@ -2305,6 +2304,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
case KVM_CAP_EXIT_HYPERCALL:
r = KVM_EXIT_HYPERCALL_VALID_MASK;
break;
+ case KVM_CAP_VCPU_EVENTS:
+ r = KVM_VCPU_EVENTS_BASE | KVM_VCPU_EVENTS_CANCEL_HCALL;
+ break;
case KVM_CAP_SET_GUEST_DEBUG2:
return KVM_GUESTDBG_VALID_MASK;
#ifdef CONFIG_KVM_XEN
@@ -3047,9 +3049,21 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu *vcpu,
| KVM_VCPUEVENT_VALID_SHADOW
| KVM_VCPUEVENT_VALID_SMM
| KVM_VCPUEVENT_VALID_PAYLOAD
- | KVM_VCPUEVENT_VALID_TRIPLE_FAULT))
+ | KVM_VCPUEVENT_VALID_TRIPLE_FAULT
+ | KVM_VCPUEVENT_CANCEL_HCALL))
return -EINVAL;
+ /*
+ * Refuse the cancel unless a cancellable completion is pending: the
+ * flag is only valid on the vCPU whose KVM_EXIT_HYPERV_HCALL exit
+ * userspace is holding, and a completion KVM armed for its own
+ * purposes (MMIO and PIO emulation continuations) is not userspace's
+ * to revoke.
+ */
+ if ((events->flags & KVM_VCPUEVENT_CANCEL_HCALL) &&
+ (!vcpu->arch.complete_userspace_io || !vcpu->arch.cui_cancellable))
+ return -EBUSY;
+
if (events->flags & KVM_VCPUEVENT_VALID_PAYLOAD) {
if (!vcpu->kvm->arch.exception_payload_enabled)
return -EINVAL;
@@ -3147,6 +3161,16 @@ static int kvm_vcpu_ioctl_x86_set_vcpu_events(struct kvm_vcpu *vcpu,
kvm_clear_request(KVM_REQ_TRIPLE_FAULT, vcpu);
}
+ /*
+ * Drop the pending completion: nothing retires the hypercall
+ * instruction, so the vCPU re-executes it on its next entry unless
+ * this call queued an exception to be delivered there instead.
+ */
+ if (events->flags & KVM_VCPUEVENT_CANCEL_HCALL) {
+ vcpu->arch.complete_userspace_io = NULL;
+ vcpu->arch.cui_cancellable = false;
+ }
+
kvm_make_request(KVM_REQ_EVENT, vcpu);
return 0;
@@ -9046,6 +9070,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)
if (unlikely(vcpu->arch.complete_userspace_io)) {
int (*cui)(struct kvm_vcpu *) = vcpu->arch.complete_userspace_io;
vcpu->arch.complete_userspace_io = NULL;
+ vcpu->arch.cui_cancellable = false;
r = cui(vcpu);
if (r <= 0)
goto out;
--
2.50.1
Attachment:
smime.p7s
Description: S/MIME cryptographic signature