Re: [RFC PATCH v2 02/25] KVM: SVM: Passthrough the number of supported ASIDs

[Yosry Ahmed]

On Mon, Jun 15, 2026 at 5:42 PM Yosry Ahmed <yosry@xxxxxxxxxx> wrote:
>
> KVM currently hardcodes the number of supported ASIDs in CPUID to 8. A
> KVM guest (L1) would then do a full TLB flush (i.e.
> TLB_CONTROL_FLUSH_ALL_ASID) every time it runs out of ASIDs on a vCPU
> and updates the generation (see new_asid()).
>
> This is currently harmless, as KVM (L0) uses the same ASID for both L1
> and L2, and flushes that ASID on nested transitions. However, following
> changes will add proper ASID emulation and a separate ASID for L2,
> minimizing the TLB flushes on nested transitions. At that point, a full
> TLB flush from a KVM guest (L1) would flush both L1 and L2 ASIDs, so
> should be avoided as much as possible.
>
> Passthrough the number of ASIDs in hardware instead of hardcoding 8, to
> reduce the chances of an L1 guest flushing its own TLB entries
> unnecessarily on a nested VMRUN.
>
> In practice, there is no harm in exposing a large number of ASIDs to the
> guest, even larger than what hardware supports, as KVM never actually
> uses the value of ASID from vmcb12. Even with a separate L2 ASID, KVM
> would allocate a (supported) ASID for L2, and just flush that same ASID
> every time L1 changes the ASID in vmcb12.
>
> That being said, avoid the temptation of just advertising the maximum
> possible number of ASIDs (i.e. 0xFFFFFFFF), in case any peculiar guest
> OS does not handle that properly.
>
> Note: QEMU currently hardcodes the number of ASIDs to 16, so this change
> doesn't help QEMU VMs (without making a similar change in QEMU).

I had a discussion with Jim internally about what our VMM should
advertise as the number of ASIDs, and based on that I think passing
through the hardware value may not be the best idea, at least not
without some disclaimers.

Jim pointed out that AMD CPUs will ignore unsupported ASID bits, or at
least earlier ones did, even though it's not mentioned in the APM
(last time I checked). This means that technically KVM can ignore the
higher bits of the ASID set by L1 beyond supported ones (e.g. if it
uses a hashtable to multiplex L1 ASIDs onto several physical ASIDs).
KVM doesn't currently do so with or without this series, but it could.

The problem happens if a migration pool contains multiple AMD CPUs
with a different number of ASIDs in hardware. If the VMM advertises
the greater value of the two, it would technically be advertising an
unsupported number of ASIDs on one of the CPUs. If KVM ignores the
upper bits based on what it supports, not what userspace advertises in
CPUID, then userspace would be shooting itself in the foot.

I think technically speaking userspace would be in the wrong for using
a number of ASIDs above what KVM_GET_SUPPORTED_CPUID provides, and it
should use a number of ASIDs that is supported by all CPUs in a
migration pool. However, this makes the VMM's life difficult, say if
AMD decides to reduce the number of ASIDs in a newer CPU just for
kicks.

All of this is hypothetical in nature, but it's a very valid concern imo.

I think there are several ways we can handle this so that VMMs can
reasonably start advertising more ASIDs to guests without these
complications:

1. We can just advertise a larger ASID value instead of 8 (say 16K or
32K), rather than passthrough what's in hardware. I assume this would
mean that KVM is more-or-less committing to supporting that number of
ASIDs regardless of underlying hardware (i.e. KVM wouldn't reduce it
in the future). In this case, a VMM can just advertise the same value.

2. We can advertise the maximum number of ASIDs (2^32 - 1?), which is
the same as option (1), except that we're biting the bullet and
preventing the possibility of KVM ignoring upper bits completely.
Might be too aggressive.

3. Document (somewhere) that KVM should use the userspace advertised
value as the source of truth for ignorable bits, even if KVM
technically supports a lower value. As I am writing this I am
realizing it doesn't make much sense as it means that the value
supported by KVM is meaningless. Maybe that comes naturally with the
"don't break userspace" rule, but it could also fall under the
category of userspace shooting itself in the foot and KVM letting it.

4. Do nothing, let this be userspace's problem, and hope that AMD
doesn't reduce the number of ASIDs in a future CPU.

I think option (1) is probably the most reasonable, we advertise 32K
ASIDs, which is what recent AMD CPUs advertise anyway, but it's
more-or-less KVM committing to 32K ASIDs even if a newer AMD CPU
doesn't support them.

Sean, WDYT?