RE: [PATCH v3 1/4] KVM: Recover IRTE to remapped mode if the interrupt is not single-destination

[Wu, Feng]

> -----Original Message-----
> From: Yang Zhang [mailto:yang.zhang.wz@xxxxxxxxx]
> Sent: Thursday, January 21, 2016 1:00 PM
> To: Wu, Feng <feng.wu@xxxxxxxxx>; pbonzini@xxxxxxxxxx;
> rkrcmar@xxxxxxxxxx
> Cc: linux-kernel@xxxxxxxxxxxxxxx; kvm@xxxxxxxxxxxxxxx
> Subject: Re: [PATCH v3 1/4] KVM: Recover IRTE to remapped mode if the
> interrupt is not single-destination
> 
> On 2016/1/21 12:42, Wu, Feng wrote:
> >
> >
> >> -----Original Message-----
> >> From: kvm-owner@xxxxxxxxxxxxxxx [mailto:kvm-owner@xxxxxxxxxxxxxxx]
> On
> >> Behalf Of Yang Zhang
> >> Sent: Thursday, January 21, 2016 11:35 AM
> >> To: Wu, Feng <feng.wu@xxxxxxxxx>; pbonzini@xxxxxxxxxx;
> >> rkrcmar@xxxxxxxxxx
> >> Cc: linux-kernel@xxxxxxxxxxxxxxx; kvm@xxxxxxxxxxxxxxx
> >> Subject: Re: [PATCH v3 1/4] KVM: Recover IRTE to remapped mode if the
> >> interrupt is not single-destination
> >>
> >> On 2016/1/21 11:14, Wu, Feng wrote:
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Yang Zhang [mailto:yang.zhang.wz@xxxxxxxxx]
> >>>> Sent: Thursday, January 21, 2016 11:06 AM
> >>>> To: Wu, Feng <feng.wu@xxxxxxxxx>; pbonzini@xxxxxxxxxx;
> >>>> rkrcmar@xxxxxxxxxx
> >>>> Cc: linux-kernel@xxxxxxxxxxxxxxx; kvm@xxxxxxxxxxxxxxx
> >>>> Subject: Re: [PATCH v3 1/4] KVM: Recover IRTE to remapped mode if
> the
> >>>> interrupt is not single-destination
> >>>>
> >>>> On 2016/1/20 9:42, Feng Wu wrote:
> >>>>> When the interrupt is not single destination any more, we need
> >>>>> to change back IRTE to remapped mode explicitly.
> >>>>>
> >>>>> Signed-off-by: Feng Wu <feng.wu@xxxxxxxxx>
> >>>>> ---
> >>>>>     arch/x86/kvm/vmx.c | 11 ++++++++++-
> >>>>>     1 file changed, 10 insertions(+), 1 deletion(-)
> >>>>>
> >>>>> diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
> >>>>> index e2951b6..13d14d4 100644
> >>>>> --- a/arch/x86/kvm/vmx.c
> >>>>> +++ b/arch/x86/kvm/vmx.c
> >>>>> @@ -10764,8 +10764,17 @@ static int vmx_update_pi_irte(struct kvm
> >>>> *kvm, unsigned int host_irq,
> >>>>>     		 */
> >>>>>
> >>>>>     		kvm_set_msi_irq(e, &irq);
> >>>>> -		if (!kvm_intr_is_single_vcpu(kvm, &irq, &vcpu))
> >>>>> +		if (!kvm_intr_is_single_vcpu(kvm, &irq, &vcpu)) {
> >>>>> +			/*
> >>>>> +			 * Make sure the IRTE is in remapped mode if
> >>>>> +			 * we don't handle it in posted mode.
> >>>>> +			 */
> >>>>> +			pi_set_sn(vcpu_to_pi_desc(vcpu));
> >>>>> +			ret = irq_set_vcpu_affinity(host_irq, NULL);
> >>>>> +			pi_clear_sn(vcpu_to_pi_desc(vcpu));
> >>>>> +
> >>>>>     			continue;
> >>>>> +		}
> >>>>>
> >>>>>     		vcpu_info.pi_desc_addr =
> __pa(vcpu_to_pi_desc(vcpu));
> >>>>>     		vcpu_info.vector = irq.vector;
> >>>>>
> >>>>
> >>>> I am still feel weird with this change: according the semantic of VT-d
> >>>> posted interrupt, the interrupt will injected to guest through posted
> >>>> notification and /proc/interrupts shows the same meaning. But now,
> >>>> without being aware of user, the interrupt changes to legacy way and it
> >>>> appears on different entry on /proc/interrupts. It looks weird.
> >>>
> >>> I don't think it has problem here, IMO, this is exactly how it works.
> >>> There should be different entry for the interrupts in VT-d PI mode
> >>> and leagcy mode.
> >>
> >> I am not saying any problem here. Just feel weird. From a normal user's
> >> point, he has turned on the VT-d pi and according the semantic of VT-d
> >> pi, he should not observe the interrupt through legacy mode, but now he
> >> do see it. Maybe print out a message here will be helpful, like what you
> >> did for disabled lapic found during irq injection.
> >
> > Even VT-d PI is on, not all interrupts can be handled by it, the reason the
> 
> No, we can handle it but we don't do it due to the complexity.For
> example, we can use wake up vector to delivery the interrupt which still
> is in PI mode but doesn't require any mode change.

I mean, multi-cast and broadcast interrupts cannot be handled in PI mode.

> 
> > interrupts is changed back to legacy mode is because the user changes
> > the affinity, and it cannot be handle in PI mode, and hence legacy mode
> > is used. It is the user's behavior that cause this mode change, seems it is
> > not so weird to me. But add some message here is good idea, just like
> 
> Why user's behavior can change the mode? 

Like you mentioned before, if the interrupt is changed from single-destination
to multiple-destination by guest. And this is the reason of adding the rollback
logic here, right?

Thanks,
Feng

> According the current design,
> there is no way for user to turn on/off dynamically.Why we need to
> rollback to legacy mode is we don't want to handle multi-destination
> interrupt in PI mode but it doesn't mean we cannot do it like i said before.
> 
> 
> --
> best regards
> yang