Re: [RFC] KVM: x86/mmu: Prefetch forward run of pages on TDP page faults

[Marangoni, Marco]

Thanks for the replies!

On Tue, Aug 25, 2026, Sean Christopherson <seanjc@xxxxxxxxxx> wrote:
>
> On Tue, Aug 25, 2026, Marco Marangoni wrote:
> > It's worth mentioning that I also evaluated using the existing
> > KVM_PRE_FAULT_MEMORY ioctl, but this doesn't work well for our use-case, as
> > it requires the vCPU to be paused.
>
> What about if/when KVM Userfault[*] comes along?  I.e. pre-fault memory when the
> vCPU exits to userspace.

KVM Userfault + KVM_PRE_FAULT_MEMORY is a valid suggestion, however if possible we'd like to have _both_ async page faults and prefetching.
I haven't tested async PF together with prefetching, but tested separately, both improvements yield great results, so it would be a shame to have to choose.

On Tue, Aug 25, 2026, James Houghton <jthoughton@xxxxxxxxxx> wrote:
> I think part of the problem in this case is that UFFDIO_COPY will
> install 4K pages (IIRC), I think a more natural way to fix this
> problem is to:
> 
> 1. MADV_COLLAPSE after doing UFFDIO_COPY.
> 2. Make UFFDIO_COPY install PMDs when it is able to do so.
> 
> These don't solve the exact same problem, but really userfaultfd
> should already try to install PMDs when it can (#2). If we have #2, #1
> is mostly a no-op.
> 
> What do you think?

Directly installing PMDs after an UFFD_COPY is something I already investigated. I didn't mention it originally, since it touches exclusively the MM module.
For some context, with that approach, in the same benchmarks, fault latency on nested is reduced by 92%, and by 44% on metal, which is significantly better than my proposal (which "only" improves by 75% and 22% respectively).
However, that approach only works when the copy is done in multiples of 2MiB, and for some Firecracker use-cases, that's a no-go (I can elaborate further if necessary, but the main problem is an explosion in incremental snapshots size when managing memory in big chunks).
I might pursue this proposal in a separate patch, however I'd love to work out a solution that can be applied when userfaultfd works with smaller chunk sizes.