Re: [RFC PATCH v2 02/25] KVM: SVM: Passthrough the number of supported ASIDs

[Jim Mattson]

On Tue, Jul 14, 2026 at 6:20 AM Yosry Ahmed <yosry@xxxxxxxxxx> wrote:
>
> On Mon, Jun 15, 2026 at 5:42 PM Yosry Ahmed <yosry@xxxxxxxxxx> wrote:
> >
> > KVM currently hardcodes the number of supported ASIDs in CPUID to 8. A
> > KVM guest (L1) would then do a full TLB flush (i.e.
> > TLB_CONTROL_FLUSH_ALL_ASID) every time it runs out of ASIDs on a vCPU
> > and updates the generation (see new_asid()).
> >
> > This is currently harmless, as KVM (L0) uses the same ASID for both L1
> > and L2, and flushes that ASID on nested transitions. However, following
> > changes will add proper ASID emulation and a separate ASID for L2,
> > minimizing the TLB flushes on nested transitions. At that point, a full
> > TLB flush from a KVM guest (L1) would flush both L1 and L2 ASIDs, so
> > should be avoided as much as possible.
> >
> > Passthrough the number of ASIDs in hardware instead of hardcoding 8, to
> > reduce the chances of an L1 guest flushing its own TLB entries
> > unnecessarily on a nested VMRUN.
> >
> > In practice, there is no harm in exposing a large number of ASIDs to the
> > guest, even larger than what hardware supports, as KVM never actually
> > uses the value of ASID from vmcb12. Even with a separate L2 ASID, KVM
> > would allocate a (supported) ASID for L2, and just flush that same ASID
> > every time L1 changes the ASID in vmcb12.
> >
> > That being said, avoid the temptation of just advertising the maximum
> > possible number of ASIDs (i.e. 0xFFFFFFFF), in case any peculiar guest
> > OS does not handle that properly.
> >
> > Note: QEMU currently hardcodes the number of ASIDs to 16, so this change
> > doesn't help QEMU VMs (without making a similar change in QEMU).
>
> I had a discussion with Jim internally about what our VMM should
> advertise as the number of ASIDs, and based on that I think passing
> through the hardware value may not be the best idea, at least not
> without some disclaimers.
>
> Jim pointed out that AMD CPUs will ignore unsupported ASID bits, or at
> least earlier ones did, even though it's not mentioned in the APM
> (last time I checked). This means that technically KVM can ignore the
> higher bits of the ASID set by L1 beyond supported ones (e.g. if it
> uses a hashtable to multiplex L1 ASIDs onto several physical ASIDs).
> KVM doesn't currently do so with or without this series, but it could.
>
> The problem happens if a migration pool contains multiple AMD CPUs
> with a different number of ASIDs in hardware. If the VMM advertises
> the greater value of the two, it would technically be advertising an
> unsupported number of ASIDs on one of the CPUs. If KVM ignores the
> upper bits based on what it supports, not what userspace advertises in
> CPUID, then userspace would be shooting itself in the foot.
>
> I think technically speaking userspace would be in the wrong for using
> a number of ASIDs above what KVM_GET_SUPPORTED_CPUID provides, and it
> should use a number of ASIDs that is supported by all CPUs in a
> migration pool. However, this makes the VMM's life difficult, say if
> AMD decides to reduce the number of ASIDs in a newer CPU just for
> kicks.
>
> All of this is hypothetical in nature, but it's a very valid concern imo.
>
> I think there are several ways we can handle this so that VMMs can
> reasonably start advertising more ASIDs to guests without these
> complications:
>
> 1. We can just advertise a larger ASID value instead of 8 (say 16K or
> 32K), rather than passthrough what's in hardware. I assume this would
> mean that KVM is more-or-less committing to supporting that number of
> ASIDs regardless of underlying hardware (i.e. KVM wouldn't reduce it
> in the future). In this case, a VMM can just advertise the same value.
>
> 2. We can advertise the maximum number of ASIDs (2^32 - 1?), which is
> the same as option (1), except that we're biting the bullet and
> preventing the possibility of KVM ignoring upper bits completely.
> Might be too aggressive.
>
> 3. Document (somewhere) that KVM should use the userspace advertised
> value as the source of truth for ignorable bits, even if KVM
> technically supports a lower value. As I am writing this I am
> realizing it doesn't make much sense as it means that the value
> supported by KVM is meaningless. Maybe that comes naturally with the
> "don't break userspace" rule, but it could also fall under the
> category of userspace shooting itself in the foot and KVM letting it.
>
> 4. Do nothing, let this be userspace's problem, and hope that AMD
> doesn't reduce the number of ASIDs in a future CPU.
>
> I think option (1) is probably the most reasonable, we advertise 32K
> ASIDs, which is what recent AMD CPUs advertise anyway, but it's
> more-or-less KVM committing to 32K ASIDs even if a newer AMD CPU
> doesn't support them.

I think Bulldozer (family 15H) first introduced support for 32K ASIDs.
KVM still supports CPUs older than that, doesn't it?

The tricky part about advertising more ASIDs than the host supports on
pre-Bulldozer CPUs is that those CPUs don't have FlushByASID. So, do
you flush all ASIDs when one is reused?