Re: [RFC] KVM: x86/mmu: Prefetch forward run of pages on TDP page faults

[Sean Christopherson]

On Tue, Aug 25, 2026, Sean Christopherson wrote:
> On Tue, Aug 25, 2026, Sean Christopherson wrote:
> > Somewhat off the cuff and *very* lightly tested, but this seems to do what I want.
> > If it provides comparable performance, I'll write a changelog (or two?  e.g. to
> > have direct MMUs switch in a separate patch), and let Sashiko and other bots rip
> > apart my idea.
> > 
> > Note!  This has a hard dependency on in-flight prefaulting fixes[*].  Without
> > those, prefaulting will hang the vCPU if the root is invalidated.
> > [*] https://lore.kernel.org/all/20260806214050.78058-1-seanjc@xxxxxxxxxx
> > 
> > Note #2!  The below deliberately ignores A/D-disabled MMUs.  I can't think of
> > any reason why it matters whether or not KVM can precisely detect accessed SPTEs,
> > all of the aging stuff is already extremely fuzzy.
> 
> And of course I posted an untested version (I ripped out the direct MMU prefetching
> as an afterthough, and dropped a printk).  This version should actually compile.

This breaks dirty_log_test and dirty_log_page_splitting_test, because KVM creates
writable SPTEs in direct MMUs whenever possible.  Because nothing can be simple,
the below in turn breaks pre_fault_memory_test, but I suspect that's a test flaw.

Note, this would also short-circuit async #PF completion when dirty logging is
enabled.  I think that's a good thing?  If not, we could teach kvm_mmu_do_page_fault()
to differentiate between async #PF and unprompted prefetching.

diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index 88b6aa1f840f..540d6583995c 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -874,19 +874,11 @@ static void unaccount_nx_huge_page(struct kvm *kvm, struct kvm_mmu_page *sp)
 	untrack_possible_nx_huge_page(kvm, sp, KVM_SHADOW_MMU);
 }
 
-static struct kvm_memory_slot *gfn_to_memslot_dirty_bitmap(struct kvm_vcpu *vcpu,
-							   gfn_t gfn,
-							   bool no_dirty_log)
+static bool kvm_is_memslot_usable_for_prefetch(struct kvm_memory_slot *slot,
+					       unsigned int access)
 {
-	struct kvm_memory_slot *slot;
-
-	slot = kvm_vcpu_gfn_to_memslot(vcpu, gfn);
-	if (!slot || slot->flags & KVM_MEMSLOT_INVALID)
-		return NULL;
-	if (no_dirty_log && kvm_slot_dirty_track_enabled(slot))
-		return NULL;
-
-	return slot;
+	return slot && !(slot->flags & KVM_MEMSLOT_INVALID) &&
+	       (!(access & ACC_WRITE_MASK) || !kvm_slot_dirty_track_enabled(slot));
 }
 
 /*
@@ -3181,8 +3173,8 @@ static bool kvm_mmu_prefetch_sptes(struct kvm_vcpu *vcpu, gfn_t gfn, u64 *sptep,
 	if (WARN_ON_ONCE(nr_pages > PTE_PREFETCH_NUM))
 		return false;
 
-	slot = gfn_to_memslot_dirty_bitmap(vcpu, gfn, access & ACC_WRITE_MASK);
-	if (!slot)
+	slot = kvm_vcpu_gfn_to_memslot(vcpu, gfn);
+	if (!kvm_is_memslot_usable_for_prefetch(slot, access))
 		return false;
 
 	nr_pages = kvm_prefetch_pages(slot, gfn, pages, nr_pages);
@@ -4946,6 +4938,8 @@ static int kvm_mmu_do_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa,
 		 */
 		fault.gfn = gpa_to_gfn(fault.addr) & ~kvm_gfn_direct_bits(vcpu->kvm);
 		fault.slot = kvm_vcpu_gfn_to_memslot(vcpu, fault.gfn);
+		if (prefetch && !kvm_is_memslot_usable_for_prefetch(fault.slot, ACC_ALL))
+			return RET_PF_WRITE_PROTECTED;
 	}
 
 	/*