[PATCH v2 5/6] KVM: Send reschedule IPIs instead of SMP function calls for non-waiting requests
From: Sean Christopherson
Date: Thu Nov 19 2026 - 18:57:24 EST
When kicking multiple vCPUs for a non-waiting request, send a reschedule
IPI to each vCPU instead of batching IPIs via a single SMP function call so
that KVM can make non-waiting requests with IRQs disabled. I.e. handle
multi-vCPU kicks the same way KVM handles single-vCPU kicks, for the exact
same reason (and more).
Avoiding SMP function calls for non-waiting requests fixes a potential
deadlock if KVM VMX triggers KVM_REQ_UNBLOCK to all vCPUs when starting
device assignmennt:
WARNING: kernel/smp.c:943 at smp_call_function_many_cond
kvm_make_all_cpus_request [kvm]
kvm_arch_irq_bypass_add_producer [kvm]
__connect [irqbypass]
irq_bypass_register_producer [irqbypass]
vfio_msi_set_vector_signal [vfio_pci_core]
Deliberately avoid using the per-CPU cpu_kick_mask for non-waiting to fix
an even more urgent bug on arm64, where KVM could fail to wait for a vCPU,
e.g. for a remote TLB flush, if an IRQ triggers a multi-vCPU kick in the
middle of another waiting kick.
kvm_flush_remote_tlbs()
kvm_make_all_cpus_request(kvm, KVM_REQ_TLB_FLUSH)
cpumask_clear(cpus)
kvm_make_vcpu_request(vcpu0, ...)
kvm_vcpu_exiting_guest_mode(vcpu0)
cmpxchg(&vcpu0->mode, IN_GUEST_MODE, EXITING_GUEST_MODE)
__cpumask_set_cpu(vcpu0->cpu, cpus)
<hardirq>
irqfd_wakeup()
...
kvm_make_all_cpus_request(kvm, KVM_REQ_IRQ_PENDING)
cpumask_clear(cpus) <-- wipes vcpu0->cpu from cpu_kick_mask
kvm_make_vcpu_request(vcpu0, ...)
kvm_request_needs_ipi(vcpu0, ...) <-- false (EXITING_GUEST_MODE)
kvm_kick_many_cpus(cpus, true) <-- misses vcpu0->cpu
Send reschedule IPIs immediately to avoid having to re-walk the set of CPUs,
and obviously because using the per-CPU mask is unsafe. As a bonus, it's
generally advantageous to kick vCPUs as soon as possible, e.g. so that the
target vCPU(s) can immediately process pending IRQs.
Fixes: 77e1b8332d1d ("KVM: x86: Decouple device assignment from IRQ bypass")
Cc: stable@xxxxxxxxxxxxxxx
Reported-by: Satish Kharat <satishkh@xxxxxxxxx>
Closes: https://lore.kernel.org/all/20261116-b4-kvm-irq-bypass-irqs-20261116-v1-1-6cc721a062fd@xxxxxxxxx
Reported-by: Sashiko Bot <sashiko-bot@xxxxxxxxxx>
Closes: https://lore.kernel.org/all/sashiko-outbox-219682@xxxxxxxxxx
Cc: Oliver Upton <oupton@xxxxxxxxxx>
Cc: Marc Zyngier <maz@xxxxxxxxxx>
Signed-off-by: Sean Christopherson <seanjc@xxxxxxxxxx>
---
virt/kvm/kvm_main.c | 21 ++++++++++++++++-----
1 file changed, 16 insertions(+), 5 deletions(-)
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 07cc0dd10b71..48fef91b8530 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -226,15 +226,17 @@ static __always_inline struct kvm_cpu_kick kvm_init_cpu_kick(unsigned int req)
kick.me = get_cpu();
- kick.cpus = this_cpu_cpumask_var_ptr(cpu_kick_mask);
- cpumask_clear(kick.cpus);
+ if (req & KVM_REQUEST_WAIT) {
+ kick.cpus = this_cpu_cpumask_var_ptr(cpu_kick_mask);
+ cpumask_clear(kick.cpus);
+ }
return kick;
}
static bool kvm_finish_cpu_kick(struct kvm_cpu_kick *kick)
{
- if (kick->need_kick)
+ if (kick->need_kick && kick->cpus)
smp_call_function_many(kick->cpus, ack_kick, NULL,
!!(kick->req & KVM_REQUEST_WAIT));
@@ -261,13 +263,22 @@ static void kvm_make_vcpu_request(struct kvm_vcpu *vcpu,
* satisfied if the vCPU migrates. Entering READING_SHADOW_PAGE_TABLES
* after this point is also OK, as the requirement is only that KVM wait
* for vCPUs that were reading SPTEs _before_ any changes were
- * finalized. See kvm_vcpu_kick() for more details on handling requests.
+ * finalized.
+ *
+ * See kvm_vcpu_kick() for more details on handling requests, including
+ * the use of a reschedule IPI for non-waiting requests. If waiting is
+ * required (kick->cpus is non-NULL), batch the SMP function call until
+ * after all to-be-kicked CPUs are gathered, to avoid waiting multiple
+ * times.
*/
if (kvm_request_needs_ipi(vcpu, kick->req)) {
cpu = READ_ONCE(vcpu->cpu);
if (cpu != -1 && cpu != kick->me && cpu_online(cpu)) {
kick->need_kick = true;
- __cpumask_set_cpu(cpu, kick->cpus);
+ if (kick->cpus)
+ __cpumask_set_cpu(cpu, kick->cpus);
+ else
+ smp_send_reschedule(cpu);
}
}
}
--
2.56.1.708.g77570a6a1d-goog