Re: [PATCH v2] KVM: x86: inhibit APICv when KVM_GUESTDBG_BLOCKIRQ active

[Maxim Levitsky]

On Wed, 2021-11-03 at 15:00 +0100, Vitaly Kuznetsov wrote:
> Maxim Levitsky <mlevitsk@xxxxxxxxxx> writes:
> 
> > On Wed, 2021-11-03 at 14:28 +0100, Vitaly Kuznetsov wrote:
> > > Maxim Levitsky <mlevitsk@xxxxxxxxxx> writes:
> > > 
> > > > KVM_GUESTDBG_BLOCKIRQ relies on interrupts being injected using
> > > > standard kvm's inject_pending_event, and not via APICv/AVIC.
> > > > 
> > > > Since this is a debug feature, just inhibit it while it
> > > > is in use.
> > > > 
> > > > Fixes: 61e5f69ef0837 ("KVM: x86: implement KVM_GUESTDBG_BLOCKIRQ")
> > > > 
> > > > Signed-off-by: Maxim Levitsky <mlevitsk@xxxxxxxxxx>
> > > > ---
> > > >  arch/x86/include/asm/kvm_host.h | 1 +
> > > >  arch/x86/kvm/svm/avic.c         | 3 ++-
> > > >  arch/x86/kvm/vmx/vmx.c          | 3 ++-
> > > >  arch/x86/kvm/x86.c              | 3 +++
> > > >  4 files changed, 8 insertions(+), 2 deletions(-)
> > > > 
> > > > diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> > > > index 88fce6ab4bbd7..8f6e15b95a4d8 100644
> > > > --- a/arch/x86/include/asm/kvm_host.h
> > > > +++ b/arch/x86/include/asm/kvm_host.h
> > > > @@ -1034,6 +1034,7 @@ struct kvm_x86_msr_filter {
> > > >  #define APICV_INHIBIT_REASON_IRQWIN     3
> > > >  #define APICV_INHIBIT_REASON_PIT_REINJ  4
> > > >  #define APICV_INHIBIT_REASON_X2APIC	5
> > > > +#define APICV_INHIBIT_REASON_BLOCKIRQ	6
> > > >  
> > > >  struct kvm_arch {
> > > >  	unsigned long n_used_mmu_pages;
> > > > diff --git a/arch/x86/kvm/svm/avic.c b/arch/x86/kvm/svm/avic.c
> > > > index 8052d92069e01..affc0ea98d302 100644
> > > > --- a/arch/x86/kvm/svm/avic.c
> > > > +++ b/arch/x86/kvm/svm/avic.c
> > > > @@ -904,7 +904,8 @@ bool svm_check_apicv_inhibit_reasons(ulong bit)
> > > >  			  BIT(APICV_INHIBIT_REASON_NESTED) |
> > > >  			  BIT(APICV_INHIBIT_REASON_IRQWIN) |
> > > >  			  BIT(APICV_INHIBIT_REASON_PIT_REINJ) |
> > > > -			  BIT(APICV_INHIBIT_REASON_X2APIC);
> > > > +			  BIT(APICV_INHIBIT_REASON_X2APIC) |
> > > > +			  BIT(APICV_INHIBIT_REASON_BLOCKIRQ);
> > > >  
> > > >  	return supported & BIT(bit);
> > > >  }
> > > > diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
> > > > index 71f54d85f104c..e4fc9ff7cd944 100644
> > > > --- a/arch/x86/kvm/vmx/vmx.c
> > > > +++ b/arch/x86/kvm/vmx/vmx.c
> > > > @@ -7565,7 +7565,8 @@ static void hardware_unsetup(void)
> > > >  static bool vmx_check_apicv_inhibit_reasons(ulong bit)
> > > >  {
> > > >  	ulong supported = BIT(APICV_INHIBIT_REASON_DISABLE) |
> > > > -			  BIT(APICV_INHIBIT_REASON_HYPERV);
> > > > +			  BIT(APICV_INHIBIT_REASON_HYPERV) |
> > > > +			  BIT(APICV_INHIBIT_REASON_BLOCKIRQ);
> > > >  
> > > >  	return supported & BIT(bit);
> > > >  }
> > > > diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> > > > index ac83d873d65b0..dccf927baa4dd 100644
> > > > --- a/arch/x86/kvm/x86.c
> > > > +++ b/arch/x86/kvm/x86.c
> > > > @@ -10747,6 +10747,9 @@ int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu,
> > > >  	if (vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP)
> > > >  		vcpu->arch.singlestep_rip = kvm_get_linear_rip(vcpu);
> > > >  
> > > > +	kvm_request_apicv_update(vcpu->kvm,
> > > > +				 !(vcpu->guest_debug & KVM_GUESTDBG_BLOCKIRQ),
> > > > +				 APICV_INHIBIT_REASON_BLOCKIRQ);
> > > >  	/*
> > > >  	 * Trigger an rflags update that will inject or remove the trace
> > > >  	 * flags.
> > > 
> > > This fixes the problem for me!
> > > 
> > > Reviewed-by: Vitaly Kuznetsov <vkuznets@xxxxxxxxxx>
> > 
> > Cool!
> > 
> > Now that I think about it, since guest debug flags are per-vcpu, this code won't
> > work if there are multiple vCPUs and you enable KVM_GUESTDBG_BLOCKIRQ on all of them
> > and then disable on this flag on just one of vCPUs, because this code will re-enable APICv/AVIC in this case.
> > A counter is needed, like you did in synic/autoeoi case.
> > 
> 
> Right, I completely forgot about this peculiarity!

I am sending v3 of this patch, I decided to avoid a counter and instead just use 
kvm_for_each_vcpu to see if other vCPUs have that bit set, on the ground of
this feature beeing only a debug feature.

Best regards,
	Maxim Levitsky

>