Re: [PATCH] RISC-V: KVM: Flush VS-stage TLB before reusing a host CPU
From: guoyaxing
Date: Mon Oct 26 2026 - 06:27:55 EST
在 2026/10/24 23:49, Anup Patel 写道:
On Sat, Oct 10, 2026 at 2:36 PM guoyaxing <guoyaxing@xxxxxxxxxx> wrote:
在 2026/10/10 15:50, Anup Patel 写道:
On Sat, Oct 10, 2026 at 4:12 AM guoyaxing <guoyaxing@xxxxxxxxxx> wrote:
在 2026/10/10 02:51, Anup Patel 写道:
On Mon, Aug 17, 2026 at 12:25 PM Yaxing Guo <guoyaxing@xxxxxxxxxx> wrote:Hi Anup,
Guest Linux uses mm_cpumask to decide whether an sfence.vma needs to be
local or remote. Under KVM that mask tracks guest CPUs, not the host CPUs
that previously backed a vCPU.
This can miss stale VS-stage TLB entries when a vCPU migrates across host
CPUs. For example:
- vcpu0 runs on host CPU1 and fills a VS-stage TLB entry.
- vcpu0 migrates to host CPU0.
- the guest updates the mapping on vcpu0 and issues a local sfence.vma.
- later, the guest task runs on vcpu1 while vcpu1 is backed by host CPU1.
When the vcpu0 is brought back to host CPU1, the kvm_riscv_local_tlb_sanitize()
will ensure TLB entries associated with Guest/VM are nuked.
By default, kvm_riscv_local_hfence_gvma_vmid_all() will only do HFENCE.GVMA
for the associated VMID because most RISC-V H-extension implemenation will
tag VS-stage TLB entries with VMID and HFENCE.GVMA must remove both
G-stage and VS-stage TLB entries for given VMID.
If your HW does not tag VS-stage TLB enteries with VMID then it buggy
/ inefficient
so you have to set kvm_riscv_vsstage_tlb_no_gpa in
kvm_riscv_setup_vendor_features()
as work-around.
NACK to this patch from myside.
Regards,
Anup
Thanks for the review and clarification.
Regarding “When the vcpu0 is brought back to host CPU1”: the failure
we hit is not caused by vcpu0 being brought back onto host CPU1. In our
case, after vcpu0 has already migrated away, neither vCPU migrates
again. The guest process is later scheduled inside the multi-vCPU guest
onto vcpu1, which is still running on host CPU1. So the stale TLB is
re-hit without any further KVM vCPU migration, and
kvm_riscv_local_tlb_sanitize() is not involved on that path.
My commit message was a bit too terse on this point. The concrete
sequence we observed was described earlier in the RFC discussion:
https://lore.kernel.org/kvm/0d15b098-b8f7-4080-9f33-e8c49c2331e1@xxxxxxxxxx/
In short, the issue is: because vcpu0 migrated away, cpu1 still has
stale TLB entries while cpu0 has the new mapping. After that, neither
vcpu migrated again, but the process was scheduled from vcpu0 to vcpu1
(which is on cpu1), causing it to hit the stale TLB. Consider the
following scenario:
1. vcpu0 and vcpu1 are both running on cpu1. At this point, vcpu0 is
on cpu1, and a process on it reads a page, which gets cached in the TLB
(let's call this the "stale TLB entry").
2. vcpu0 migrates to cpu0, and then the process (on vcpu0, now on
cpu0) does a CoW (copy-on-write) and establishes a new mapping.
3. Neither vcpu0 nor vcpu1 migrates again. At this point, the process
gets scheduled inside the VM onto vcpu1 (which is on cpu1), reads the
same page, and hits the stale TLB entry.
This cannot happen because in step#2 when mapping was updated the
Guest OS will do TLB flush for all its VCPUs which means vcpu1 on cpu0
will also receive TLB flush and the stale TLB will be removed. This means
when process is moved back to vcpu1 on cpu0, HW will fetch updated
PTE and new TLB entry will be populated.
Hi Anup,
Thanks for the follow-up. I may be missing something, so I wanted to
double-check this against the guest TLB shootdown path.
If I follow the Linux guest CoW path correctly, when the mapping is
updated in step#2 it goes through:
wp_page_copy()
-> ptep_clear_flush()
-> flush_tlb_page()
-> __flush_tlb_range(mm, mm_cpumask(mm), ...)
So the remote SFENCE.VMA appears to be targeted only at CPUs in
mm_cpumask(mm), rather than at all VCPUs of the guest. mm_cpumask is
updated in switch_mm()/set_mm() for guest CPUs that have actually run
that mm.
In our case, before the CoW the process had only ever run on vcpu0. From
the guest's point of view it did not migrate: the host migration of
vcpu0 from cpu1 to cpu0 is not visible to guest Linux, which only sees
vCPUs. So at CoW time mm_cpumask still contains only vcpu0 (or at least
does not contain vcpu1), and __flush_tlb_range() would not IPI vcpu1.
The local SFENCE on vcpu0 would only cover host cpu0, where vcpu0 is
currently running. The stale VS-stage TLB entry left on host cpu1 from
when vcpu0 previously ran there would then remain.
Later, still without any further vCPU migration, the process is
scheduled inside the guest onto vcpu1, which remains on host cpu1, and
can hit that stale entry.
This also seems more likely when the ASID allocator is enabled, because
set_mm_asid() does not always local_flush_tlb_all() on switch_mm(). So
even the first time the process lands on vcpu1, a stale ASID-tagged
entry on host cpu1 may still be present.
In other words, I am not sure this case is covered by a full guest TLB
flush to all VCPUs on the mapping update. Guest shootdown looks
mm_cpumask-based, and host pCPU migration of a vCPU does not by itself
add other vCPUs that never ran the mm into that mask.
This makes sense now considering ASID allocator in Guest Linux is global
and not local. In any case, KVM RISC-V must not assume that the Guest OS
has local ASID allocator hence the kvm_riscv_local_tlb_sanitize() must also
santize TLB for other VCPUs of the same Guest which can potentially run
on the previous host CPU where current VCPU ran.
Having said the above, there are multiple issues with the current patch:
1) The kvm_riscv_local_hfence_gvma_vmid_all() is sufficient for most
implementations to sanitize both G-stage and VS-stage TLB entries
because both G-stage and VS-stage TLB entries are generally tagged
with VMID. The kvm_riscv_vsstage_tlb_no_gpa work-around is for
implmentations which do not tag VS-stage TLB entries with VMID
and this patch remove this work-around.
2) The other VCPUs on the previous host CPU may be already running
and may still hit stale TLB entry before kvm_riscv_local_tlb_sanitize()
gets a chance to sanitize TLB on previous host CPU.
3) The per-CPU pointer in struct kvm_arch is racy because we
may have two VCPUs on same host CPU which are migrated to
other host CPUs. Instead, it is much simpler to have an atomic
cpumask in each VCPU where any part of KVM can mark a host
CPU in the cpumask of the VCPU to force TLB sanitization whenever
the VCPU runs on that particular host CPU.
How about the following change ?
diff --git a/arch/riscv/include/asm/kvm_host.h
b/arch/riscv/include/asm/kvm_host.h
index a30600579231..c975eee7240c 100644
--- a/arch/riscv/include/asm/kvm_host.h
+++ b/arch/riscv/include/asm/kvm_host.h
@@ -9,6 +9,7 @@
#ifndef __RISCV_KVM_HOST_H__
#define __RISCV_KVM_HOST_H__
+#include <linux/cpumask.h>
#include <linux/types.h>
#include <linux/kvm.h>
#include <linux/kvm_types.h>
@@ -184,6 +185,9 @@ struct kvm_vcpu_arch {
/* ISA feature bits (similar to MISA) */
DECLARE_BITMAP(isa, RISCV_ISA_EXT_MAX);
+ /* Sanitize TLB host CPU mask */
+ cpumask_t tlb_sanitize_mask;
+
/* Vendor, Arch, and Implementation details */
unsigned long mvendorid;
unsigned long marchid;
diff --git a/arch/riscv/kvm/tlb.c b/arch/riscv/kvm/tlb.c
index 2ae34632cdcb..b1387a00c8b0 100644
--- a/arch/riscv/kvm/tlb.c
+++ b/arch/riscv/kvm/tlb.c
@@ -207,11 +207,31 @@ void kvm_riscv_local_hfence_vvma_all(unsigned long vmid)
void kvm_riscv_local_tlb_sanitize(struct kvm_vcpu *vcpu)
{
- unsigned long vmid;
+ struct kvm_vcpu *tvcpu;
+ unsigned long i, vmid;
- if (!kvm_riscv_gstage_vmid_bits() ||
- vcpu->arch.last_exit_cpu == vcpu->cpu)
- return;
+ if (!cpumask_test_cpu(vcpu->cpu, &vcpu->arch.tlb_sanitize_mask)) {
+ if (!kvm_riscv_gstage_vmid_bits() ||
+ vcpu->arch.last_exit_cpu == vcpu->cpu)
+ return;
+ }
+
+ /*
+ * The VCPU might have left stale TLBs on the previous host CPU so
+ * mark previous host CPU in the tlb_sanitize_mask of other VCPUs.
+ */
+ if (vcpu->arch.last_exit_cpu > -1 && vcpu->arch.last_exit_cpu !=
vcpu->cpu) {
+ kvm_for_each_vcpu(i, tvcpu, vcpu->kvm) {
+ if (tvcpu == vcpu)
+ continue;
+ cpumask_set_cpu(vcpu->arch.last_exit_cpu,
+ &tvcpu->arch.tlb_sanitize_mask);
+ kvm_vcpu_kick(tvcpu);
+ }
+ }
+
+ /* Clear current host CPU from the tlb_sanitize_mask */
+ cpumask_clear_cpu(vcpu->cpu, &vcpu->arch.tlb_sanitize_mask);
/*
* On RISC-V platforms with hardware VMID support, we share same
diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
index dd786f758eb7..c00dc0b191dc 100644
--- a/arch/riscv/kvm/vcpu.c
+++ b/arch/riscv/kvm/vcpu.c
@@ -107,6 +107,8 @@ static void kvm_riscv_reset_vcpu(struct kvm_vcpu
*vcpu, bool kvm_sbi_reset)
vcpu->arch.last_exit_cpu = -1;
+ cpumask_clear(&vcpu->arch.tlb_sanitize_mask);
+
kvm_riscv_vcpu_context_reset(vcpu, kvm_sbi_reset);
kvm_riscv_vcpu_fp_reset(vcpu);
Regards,
Anup
Hi Anup,
Thanks for the explanation and for suggesting the per-vCPU sanitization
mask. Building on your proposal, I would like to suggest two changes:
1. Separate the invalidation strategy for vCPU migration and a mask hit.
Keep HFENCE.GVMA for vCPU migration, identified by
last_exit_cpu != vcpu->cpu, when hardware VMIDs are available. Use
HFENCE.VVMA when the current host CPU is marked in tlb_sanitize_mask.
A mask hit caused by another vCPU of the same VM running on this CPU
requires VS-stage sanitization; it does not by itself require G-stage
invalidation.
For example, if vCPU0 and vCPU1 both last ran on CPU0, switching from
vCPU1 back to vCPU0 only needs the mask-triggered VVMA. This avoids
requesting G-stage invalidation when the shared G-stage mappings have
not changed. If both migration and a mask hit occur, both fences run.
2. Move mask updates to kvm_arch_vcpu_put().
Mark the outgoing host CPU in the masks of the other vCPUs of the
same VM when a vCPU is put. This also covers a switch between vCPUs
on one host CPU without either vCPU migrating. Marking only when
migration is detected in kvm_riscv_local_tlb_sanitize() would miss
that case.
The marking now happens on the outgoing CPU before another vCPU can
enter guest mode there. The migration-time marking and kicks are
therefore removed. The trade-off is an O(nr_vcpus) walk on each
qualifying put().
How about the following change ?
diff --git a/arch/riscv/kvm/tlb.c b/arch/riscv/kvm/tlb.c
index 2ac3c0a20fce..ae8a40b3429c 100644
--- a/arch/riscv/kvm/tlb.c
+++ b/arch/riscv/kvm/tlb.c
@@ -160,32 +160,15 @@
void kvm_riscv_local_tlb_sanitize(struct kvm_vcpu *vcpu)
{
- struct kvm_vcpu *tvcpu;
- unsigned long i, vmid;
-
- if (!cpumask_test_cpu(vcpu->cpu, &vcpu->arch.tlb_sanitize_mask)) {
- if (!kvm_riscv_gstage_vmid_bits() ||
- vcpu->arch.last_exit_cpu == vcpu->cpu)
- return;
- }
-
- /*
- * The VCPU might have left stale TLBs on the previous host CPU so
- * mark previous host CPU in the tlb_sanitize_mask of other VCPUs.
- */
- if (vcpu->arch.last_exit_cpu > -1 &&
- vcpu->arch.last_exit_cpu != vcpu->cpu) {
- kvm_for_each_vcpu(i, tvcpu, vcpu->kvm) {
- if (tvcpu == vcpu)
- continue;
- cpumask_set_cpu(vcpu->arch.last_exit_cpu,
- &tvcpu->arch.tlb_sanitize_mask);
- kvm_vcpu_kick(tvcpu);
- }
- }
-
- /* Clear current host CPU from the tlb_sanitize_mask */
- cpumask_clear_cpu(vcpu->cpu, &vcpu->arch.tlb_sanitize_mask);
+ unsigned long vmid;
+ bool vcpu_migrated, sanitize_vsstage;
+
+ vcpu_migrated = vcpu->arch.last_exit_cpu != vcpu->cpu;
+ sanitize_vsstage = cpumask_test_and_clear_cpu(vcpu->cpu,
+ &vcpu->arch.tlb_sanitize_mask);
+ if (!sanitize_vsstage &&
+ (!kvm_riscv_gstage_vmid_bits() || !vcpu_migrated))
+ return;
/*
* On RISC-V platforms with hardware VMID support, we share same
@@ -199,13 +182,16 @@
*/
vmid = READ_ONCE(vcpu->kvm->arch.vmid.vmid);
- kvm_riscv_local_hfence_gvma_vmid_all(vmid);
+ if (vcpu_migrated && kvm_riscv_gstage_vmid_bits())
+ kvm_riscv_local_hfence_gvma_vmid_all(vmid);
/*
- * Flush VS-stage TLB entries for implementation where VS-stage
- * TLB does not cahce guest physical address and VMID.
+ * Flush VS-stage entries marked when another VCPU left this CPU, or when
+ * required by the split-TLB vendor workaround on this VCPU's migration.
*/
- if (static_branch_unlikely(&kvm_riscv_vsstage_tlb_no_gpa))
+ if (sanitize_vsstage ||
+ (vcpu_migrated &&
+ static_branch_unlikely(&kvm_riscv_vsstage_tlb_no_gpa)))
kvm_riscv_local_hfence_vvma_all(vmid);
}
diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c
index f2e52f3b0786..6209c66b24e1 100644
--- a/arch/riscv/kvm/vcpu.c
+++ b/arch/riscv/kvm/vcpu.c
@@ -644,8 +644,18 @@
void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
{
+ struct kvm_vcpu *tvcpu;
+ unsigned long i;
+ int cpu = vcpu->cpu;
void *nsh;
struct kvm_vcpu_csr *csr = &vcpu->arch.guest_csr;
+
+ if (cpu >= 0 && vcpu->arch.last_exit_cpu == cpu) {
+ kvm_for_each_vcpu(i, tvcpu, vcpu->kvm) {
+ if (tvcpu != vcpu)
+ cpumask_set_cpu(cpu, &tvcpu->arch.tlb_sanitize_mask);
+ }
+ }
vcpu->cpu = -1;
Would these changes make sense to you, or is there anythingI might be overlooking?
Regards,
Yaxing