[PATCH] KVM: x86/mmu: Wait for freed TDP MMU pages only before destroying caches

From: Qiliang Yuan

Date: Wed Nov 18 2026 - 10:45:03 EST


The TDP MMU frees page table pages from RCU callbacks, so that lockless
walkers never see a freed table. kvm_mmu_uninit_tdp_mmu() ends with
rcu_barrier(), which commit 7cca2d0b7e7d ("KVM: x86/mmu: Protect TDP MMU
page table memory with RCU") added so that these callbacks run before
the VM is torn down.

The callbacks only free the page table page and return its kvm_mmu_page
to mmu_page_header_cache; they do not touch the VM. Still, destroying
any VM waits for the RCU callbacks pending on every CPU, which accounts
for most of the time it takes to destroy a small VM and limits how fast
sandboxes that run one short-lived VM per task can be recycled.

Move the barrier to mmu_destroy_caches(), the only place that needs the
callbacks to have run, before mmu_page_header_cache is destroyed.

Each thread creates VMs with a 1 MiB memslot and one vCPU that touches
eight guest pages, and destroys them by closing the vCPU and VM fds; 400
VMs per run in a 32 vCPU guest with nested KVM on AMD, median of five
runs:

destroy time per VM VMs per second
threads before (ms) after (ms) change before after change
1 11.9 0.76 -94% 61 509 +734%
4 15.5 0.61 -96% 197 1105 +461%
16 23.6 2.87 -88% 429 968 +126%

Signed-off-by: Qiliang Yuan <qiliang.yuan@xxxxxxxxx>
---
arch/x86/kvm/mmu/mmu.c | 5 +++++
arch/x86/kvm/mmu/tdp_mmu.c | 7 -------
2 files changed, 5 insertions(+), 7 deletions(-)

diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index f89e9f8a6cc2f..eec12168abc13 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -7769,6 +7769,11 @@ void kvm_mmu_invalidate_mmio_sptes(struct kvm *kvm, u64 gen)

static void mmu_destroy_caches(void)
{
+ /*
+ * Let the RCU callbacks that free TDP MMU pages of destroyed VMs
+ * return their headers to mmu_page_header_cache before destroying it.
+ */
+ rcu_barrier();
kmem_cache_destroy(pte_list_desc_cache);
pte_list_desc_cache = NULL;
kmem_cache_destroy(mmu_page_header_cache);
diff --git a/arch/x86/kvm/mmu/tdp_mmu.c b/arch/x86/kvm/mmu/tdp_mmu.c
index 44dad106fad19..7bfb84a1a2ff1 100644
--- a/arch/x86/kvm/mmu/tdp_mmu.c
+++ b/arch/x86/kvm/mmu/tdp_mmu.c
@@ -44,13 +44,6 @@ void kvm_mmu_uninit_tdp_mmu(struct kvm *kvm)
KVM_MMU_WARN_ON(atomic64_read(&kvm->arch.tdp_mmu_pages));
#endif
WARN_ON(!list_empty(&kvm->arch.tdp_mmu_roots));
-
- /*
- * Ensure that all the outstanding RCU callbacks to free shadow pages
- * can run before the VM is torn down. Putting the last reference to
- * zapped roots will create new callbacks.
- */
- rcu_barrier();
}

static void __tdp_mmu_free_sp(struct kvm_mmu_page *sp)

---
base-commit: 20eef29b7c8465ac98d7dd6d6f2ac07a8d8056ca
change-id: 20261118-feat-kvm-tdp-mmu-rcu-barrier-b7b65a600e6c

Best regards,
--
Qiliang Yuan <qiliang.yuan@xxxxxxxxx>