Re: [PATCH v2 1/2] KVM: LAPIC: Optimize timer latency consider world switch time

[Sean Christopherson]

On Tue, Jun 11, 2019 at 09:38:18AM +0800, Wanpeng Li wrote:
> On Tue, 11 Jun 2019 at 09:21, Sean Christopherson
> <sean.j.christopherson@xxxxxxxxx> wrote:
> >
> > On Fri, May 31, 2019 at 02:40:13PM +0800, Wanpeng Li wrote:
> > > From: Wanpeng Li <wanpengli@xxxxxxxxxxx>
> > >
> > > Advance lapic timer tries to hidden the hypervisor overhead between the
> > > host emulated timer fires and the guest awares the timer is fired. However,
> > > even though after more sustaining optimizations, kvm-unit-tests/tscdeadline_latency
> > > still awares ~1000 cycles latency since we lost the time between the end of
> > > wait_lapic_expire and the guest awares the timer is fired. There are
> > > codes between the end of wait_lapic_expire and the world switch, furthermore,
> > > the world switch itself also has overhead. Actually the guest_tsc is equal
> > > to the target deadline time in wait_lapic_expire is too late, guest will
> > > aware the latency between the end of wait_lapic_expire() and after vmentry
> > > to the guest. This patch takes this time into consideration.
> > >
> > > The vmentry_lapic_timer_advance_ns module parameter should be well tuned by
> > > host admin, setting bit 0 to 1 to finally cache parameter in KVM. This patch
> > > can reduce average cyclictest latency from 3us to 2us on Skylake server.
> > > (guest w/ nohz=off, idle=poll, host w/ preemption_timer=N, the cyclictest
> > > latency is not too sensitive when preemption_timer=Y for this optimization in
> > > my testing), kvm-unit-tests/tscdeadline_latency can reach 0.
> > >
> > > Cc: Paolo Bonzini <pbonzini@xxxxxxxxxx>
> > > Cc: Radim KrÄmÃÅ <rkrcmar@xxxxxxxxxx>
> > > Cc: Sean Christopherson <sean.j.christopherson@xxxxxxxxx>
> > > Signed-off-by: Wanpeng Li <wanpengli@xxxxxxxxxxx>
> > > ---
> > > NOTE: rebase on https://lkml.org/lkml/2019/5/20/449
> > > v1 -> v2:
> > >  * rename get_vmentry_advance_delta to get_vmentry_advance_cycles
> > >  * cache vmentry_advance_cycles by setting param bit 0
> > >  * add param max limit
> > >
> > >  arch/x86/kvm/lapic.c   | 38 +++++++++++++++++++++++++++++++++++---
> > >  arch/x86/kvm/lapic.h   |  3 +++
> > >  arch/x86/kvm/vmx/vmx.c |  2 +-
> > >  arch/x86/kvm/x86.c     |  9 +++++++++
> > >  arch/x86/kvm/x86.h     |  2 ++
> > >  5 files changed, 50 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
> > > index fcf42a3..60587b5 100644
> > > --- a/arch/x86/kvm/lapic.c
> > > +++ b/arch/x86/kvm/lapic.c
> > > @@ -1531,6 +1531,38 @@ static inline void adjust_lapic_timer_advance(struct kvm_vcpu *vcpu,
> > >       apic->lapic_timer.timer_advance_ns = timer_advance_ns;
> > >  }
> > >
> > > +#define MAX_VMENTRY_ADVANCE_NS 1000
> > > +
> > > +u64 compute_vmentry_advance_cycles(struct kvm_vcpu *vcpu)
> >
> > This can be static, unless get_vmentry_advance_cycles() is moved to
> > lapic.h, in which case compute_vmentry_advance_cycles() would need to be
> > exported.
> 
> Thanks for the review, Sean. I think Paolo has already drop this one.
> https://lkml.org/lkml/2019/5/31/210

I couldn't tell if Paolo's response was "no, don't do that" or "let's be
careful".  :-)