Re: Some options abort RFC: KVM: SVM: Rework ASID management

[Yosry Ahmed]

On Mon, Nov 2, 2026 at 3:20 AM Jackie Yu <ufo19890607@xxxxxxxxx> wrote:
>
> Hi Sean, Yosry, and KVM folks,
>
> Sean, thank you for the reminder. Sorry about the earlier formatting.
> We will use plain text for future mailing-list replies.
>
> Yosry, thanks for clarifying. We understand that by "missing
> emulation" you meant the INVLPGB handling needed with distinct
> per-vCPU ASIDs, not a bug in an existing emulation path in our tree.
>
> We agree that intercepting each guest INVLPGB and issuing a separate
> invalidation for every ASID in a large VM may undermine the benefit
> we are seeking. Using per-VM ASIDs consistently, rather than
> switching ASID allocation models based on nested usage, sounds like
> a promising direction. Our production guests do not run nested VMs,
> so direct execution before nested use would cover our use case.
>
> Your suggestion to change the INVLPGB interception policy when
> nested virtualization is first used is helpful. If EFER.SVME can be
> set before a guest runs a nested VM, the first VMRUN seems like a
> more precise trigger. Would KVM catch that VMRUN before entering L2
> and enable the necessary intercepts across the VM before any vCPU
> can execute an INVLPGB requiring nested handling?

Yeah we can do this. For example, we can have a static branch in the
VMRUN emulation path that flips between a synchronized path that
enables the INVLPGB intercept, and the normal path. The static branch
might be an overkill.

> Could those
> intercepts then remain enabled for the VM's lifetime, rather than
> switching back when the guest leaves nested mode?

Probably? I think we can only switch back when all vCPUs clear
EFER.SVME. I doubt it's worth optimizing.

> We realize that
> interception would still require correct nested instruction handling.
>
> For context, our experimental KVM4 configuration used a VM-shared
> ASID and separate, out-of-tree guest INVLPGB passthrough. The
> madvise() latency measurements were:
>
>   Workers             INVLPGB off      INVLPGB on
>   64 mean              21.624 ms        12.505 ms
>   64 P99               32.760 ms        21.518 ms
>   128 mean             29.553 ms        20.459 ms
>   128 P99              39.628 ms        28.160 ms
>
> These were single sequential A/B runs with 64 allowed CPUs; the
> 128-worker run was oversubscribed. The results measure madvise()
> syscall latency, not IPI counts, and do not quantify any additional
> flushes from vCPU switches on a physical CPU under per-VM ASIDs.
>
> We can share the benchmark source and run further comparisons to
> help evaluate that trade-off. Would the maintainers consider
> consistent per-VM ASIDs, with direct guest INVLPGB execution before
> nested use and interception once nested use begins, a direction
> worth pursuing?

As I mentioned, I (not the maintainer) have no problem with per-VM
ASIDs. We probably want to make sure we don't regress workloads due to
the extra TLB flushes when multiple vCPUs run on the same pCPU tho.

Other than that, I don't want to rework my series to go back to per-VM
ASIDs. So if Sean and Paolo are okay with per-VM ASIDs, I think this
should be done as a follow up work on top. Just my 2c.