Re: [PATCH 4/4] KVM: x86: Add vCPU stat for APICv interrupt injections causing #VMEXIT

[Alejandro Jimenez]

On 6/3/24 20:14, Sean Christopherson wrote:
> On Mon, Apr 29, 2024, Alejandro Jimenez wrote:
> > Even when APICv/AVIC is active, certain guest accesses to its local APIC(s)
> > cannot be fully accelerated, and cause a #VMEXIT to allow the VMM to
> > emulate the behavior and side effects. Expose a counter stat for these
> > specific #VMEXIT types.
> >
> > Suggested-by: Paolo Bonzini <pbonzini@xxxxxxxxxx>
> > Signed-off-by: Alejandro Jimenez <alejandro.j.jimenez@xxxxxxxxxx>
> > ---
> >   arch/x86/include/asm/kvm_host.h | 1 +
> >   arch/x86/kvm/svm/avic.c         | 7 +++++++
> >   arch/x86/kvm/vmx/vmx.c          | 2 ++
> >   arch/x86/kvm/x86.c              | 1 +
> >   4 files changed, 11 insertions(+)
> >
> > diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> > index e7e3213cefae..388979dfe9f3 100644
> > --- a/arch/x86/include/asm/kvm_host.h
> > +++ b/arch/x86/include/asm/kvm_host.h
> > @@ -1576,6 +1576,7 @@ struct kvm_vcpu_stat {
> >   	u64 guest_mode;
> >   	u64 notify_window_exits;
> >   	u64 apicv_active;
> > +	u64 apicv_unaccelerated_inj;
>
> The stat name doesn't match the changelog or the code.  The AVIC updates in
> avic_incomplete_ipi_interception() are unaccelerated _injection_, they're
> unaccelarated _delivery_.  And in those cases, the fact that delivery wasn't
> accelerated is relatively uninteresting in most cases.

Yeah, this was my flawed attempt to interpret/implement Paolo's comment in the RFC thread:

"... for example I'd add an interrupt_injections stat for unaccelerated injections causing a vmexit or otherwise hitting lapic.c"

so I incorrectly bundled together APIC accesses that result in #VMEXIT and end up requiring additional emulation (while managing to miss the handle_apic_access() case).


> And avic_unaccelerated_access_interception() and handle_apic_write() don't
> necessarily have anything to do with injection.

apicv_unaccelerated_acccess is perhaps a better name (assuming stat is updated in handle_apic_access() as well)?

> On the flip side, the slow paths for {svm,vmx}_deliver_interrupt() are very
> explicitly unnaccelerated injection.

Now that you highlight this, I think it might be closer to Paolo's idea. i.e. a stat for the slow path on these can be contrasted/compared with the kvm_apicv_accept_irq tracepoint that is hit on the fast path.
My initial reaction would be to update a stat for the fast path, as a confirmation that apicv is active which is how/why I typically use the kvm_apicv_accept_irq tracepoint, but that becomes redundant by having the apicv_active stat on PATCH 1.

So, if you don't think it is useful to have a general apicv_unaccelerated_acccess counter, I can drop this patch.

Thank you,

Alejandro

> It's not entirely clear from the changelog what the end goal of this stat is.
> A singular stat for all APICv/AVIC access VM-Exits seems uninteresting, as such
> a stat essentially just captures that the guest is active.  Maaaybe someone could
> glean info from comparing two VMs, but even that is dubious.  E.g. if a guest is
> doing something function and generating a lot of avic_incomplete_ipi_interception()
> exits, those will likely be in the noise due to the total volume of other AVIC
> exits.
>
> >   };
> >   
> >   struct x86_instruction_info;
> > diff --git a/arch/x86/kvm/svm/avic.c b/arch/x86/kvm/svm/avic.c
> > index 4b74ea91f4e6..274041d3cf66 100644
> > --- a/arch/x86/kvm/svm/avic.c
> > +++ b/arch/x86/kvm/svm/avic.c
> > @@ -517,6 +517,8 @@ int avic_incomplete_ipi_interception(struct kvm_vcpu *vcpu)
> >   			kvm_apic_write_nodecode(vcpu, APIC_ICR);
> >   		else
> >   			kvm_apic_send_ipi(apic, icrl, icrh);
> > +
> > +		++vcpu->stat.apicv_unaccelerated_inj;
> >   		break;
> >   	case AVIC_IPI_FAILURE_TARGET_NOT_RUNNING:
> >   		/*
> > @@ -525,6 +527,8 @@ int avic_incomplete_ipi_interception(struct kvm_vcpu *vcpu)
> >   		 * vcpus. So, we just need to kick the appropriate vcpu.
> >   		 */
> >   		avic_kick_target_vcpus(vcpu->kvm, apic, icrl, icrh, index);
> > +
> > +		++vcpu->stat.apicv_unaccelerated_inj;
> >   		break;
> >   	case AVIC_IPI_FAILURE_INVALID_BACKING_PAGE:
> >   		WARN_ONCE(1, "Invalid backing page\n");
> > @@ -704,6 +708,9 @@ int avic_unaccelerated_access_interception(struct kvm_vcpu *vcpu)
> >   
> >   	trace_kvm_avic_unaccelerated_access(vcpu->vcpu_id, offset,
> >   					    trap, write, vector);
> > +
> > +	++vcpu->stat.apicv_unaccelerated_inj;
> > +
> >   	if (trap) {
> >   		/* Handling Trap */
> >   		WARN_ONCE(!write, "svm: Handling trap read.\n");
> > diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
> > index f10b5f8f364b..a7487f12ded1 100644
> > --- a/arch/x86/kvm/vmx/vmx.c
> > +++ b/arch/x86/kvm/vmx/vmx.c
> > @@ -5657,6 +5657,8 @@ static int handle_apic_write(struct kvm_vcpu *vcpu)
> >   {
> >   	unsigned long exit_qualification = vmx_get_exit_qual(vcpu);
> >   
> > +	++vcpu->stat.apicv_unaccelerated_inj;
> > +
> >   	/*
> >   	 * APIC-write VM-Exit is trap-like, KVM doesn't need to advance RIP and
> >   	 * hardware has done any necessary aliasing, offset adjustments, etc...
> > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> > index 03cb933920cb..c8730b0fac87 100644
> > --- a/arch/x86/kvm/x86.c
> > +++ b/arch/x86/kvm/x86.c
> > @@ -307,6 +307,7 @@ const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = {
> >   	STATS_DESC_IBOOLEAN(VCPU, guest_mode),
> >   	STATS_DESC_COUNTER(VCPU, notify_window_exits),
> >   	STATS_DESC_IBOOLEAN(VCPU, apicv_active),
> > +	STATS_DESC_COUNTER(VCPU, apicv_unaccelerated_inj),
> >   };
> >   
> >   const struct kvm_stats_header kvm_vcpu_stats_header = {
> > --
> > 2.39.3