Re: [PATCH] KVM: Boosting vCPUs that are delivering interrupts

From: Christian Borntraeger
Date: Mon Jul 15 2019 - 06:53:51 EST




On 12.07.19 09:10, Wanpeng Li wrote:
> From: Wanpeng Li <wanpengli@xxxxxxxxxxx>
>
> Inspired by commit 9cac38dd5d (KVM/s390: Set preempted flag during vcpu wakeup
> and interrupt delivery), except the lock holder, we want to also boost vCPUs
> that are delivering interrupts. Actually most smp_call_function_many calls are
> synchronous ipi calls, the ipi target vCPUs are also good yield candidates.
> This patch sets preempted flag during wakeup and interrupt delivery time.
>
> Testing on 80 HT 2 socket Xeon Skylake server, with 80 vCPUs VM 80GB RAM:
> ebizzy -M
>
> vanilla boosting improved
> 1VM 23000 21232 -9%
> 2VM 2800 8000 180%
> 3VM 1800 3100 72%
>
> Testing on my Haswell desktop 8 HT, with 8 vCPUs VM 8GB RAM, two VMs,
> one running ebizzy -M, the other running 'stress --cpu 2':
>
> w/ boosting + w/o pv sched yield(vanilla)
>
> vanilla boosting improved
> 1570 4000 55%
>
> w/ boosting + w/ pv sched yield(vanilla)
>
> vanilla boosting improved
> 1844 5157 79%
>
> w/o boosting, perf top in VM:
>
> 72.33% [kernel] [k] smp_call_function_many
> 4.22% [kernel] [k] call_function_i
> 3.71% [kernel] [k] async_page_fault
>
> w/ boosting, perf top in VM:
>
> 38.43% [kernel] [k] smp_call_function_many
> 6.31% [kernel] [k] async_page_fault
> 6.13% libc-2.23.so [.] __memcpy_avx_unaligned
> 4.88% [kernel] [k] call_function_interrupt
This certainly made sense for s390 so I guess that this also makes sense
for others.
Nnote we (s390) do not use kvm_vcpu_kick, so this should not cause
any issue for s390.


>
> Cc: Paolo Bonzini <pbonzini@xxxxxxxxxx>
> Cc: Radim KrÄmÃÅ <rkrcmar@xxxxxxxxxx>
> Cc: Christian Borntraeger <borntraeger@xxxxxxxxxx>
> Signed-off-by: Wanpeng Li <wanpengli@xxxxxxxxxxx>
> ---
> virt/kvm/kvm_main.c | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index b4ab59d..2c46705 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -2404,8 +2404,10 @@ void kvm_vcpu_kick(struct kvm_vcpu *vcpu)
> int me;
> int cpu = vcpu->cpu;
>
> - if (kvm_vcpu_wake_up(vcpu))
> + if (kvm_vcpu_wake_up(vcpu)) {
> + vcpu->preempted = true;
> return;
> + }
>
> me = get_cpu();
> if (cpu != me && (unsigned)cpu < nr_cpu_ids && cpu_online(cpu))
>