Re: [PATCH v3 00/16] kvm: x86: Support AMD SVM AVIC w/ in-kernel irqchip mode

[Suthikulpanit, Suravee]

Ping:

Hi All,

Are there other concerns or suggestions for this series?

Thanks,
Suravee

On 9/13/19 2:00 PM, Suthikulpanit, Suravee wrote:
> The 'commit 67034bb9dd5e ("KVM: SVM: Add irqchip_split() checks before
> enabling AVIC")' was introduced to fix miscellaneous boot-hang issues
> when enable AVIC. This is mainly due to AVIC hardware doest not #vmexit
> on write to LAPIC EOI register resulting in-kernel PIC and IOAPIC to
> wait and do not inject new interrupts (e.g. PIT, RTC).
> 
> This limits AVIC to only work with kernel_irqchip=split mode, which is
> not currently enabled by default, and also required user-space to
> support split irqchip model, which might not be the case.
> 
> The goal of this series is to enable AVIC to work in both irqchip modes,
> by allowing AVIC to be deactivated temporarily during runtime, and fallback
> to legacy interrupt injection mode (w/ vINTR and interrupt windows)
> when needed, and then re-enabled subsequently.
> 
> Similar approach is also used to handle Hyper-V SynIC in the
> 'commit 5c919412fe61 ("kvm/x86: Hyper-V synthetic interrupt controller")',
> where APICv is permanently disabled at runtime (currently broken for
> AVIC, and fixed by this series).
> 
> This series contains three parts:
>    * Part 1: patch 1-2
>      Introduce APICv state enum and logic for keeping track of the state
>      for each vm.
>   
>    * Part 2: patch 3-11
>      Add support for activate/deactivate APICv at runtime
> 
>    * Part 3: patch 12-16:
>      Provide workaround for AVIC EOI and allow enable AVIC w/
>      kernel_irqchip=on
> 
> Pre-requisite Patch:
>    * commit b9c6ff94e43a ("iommu/amd: Re-factor guest virtual APIC
>      (de-)activation code")
>      (https://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu.git/commit/
>       ?h=next&id=b9c6ff94e43a0ee053e0c1d983fba1ac4953b762)
> 
> This series has been tested against v5.3-rc5 as following:
>    * Booting Linux and Windows Server 2019 VMs upto 240 vcpus
>      and FreeBSD upto 128 vcpus w/ qemu option "kernel-irqchip=on"
>      and "-no-hpet".
>    * Pass-through Intel 10GbE NIC and run netperf in the VM.
> 
> Changes from V2: (https://lkml.org/lkml/2019/8/15/672)
>    * Rebase to v5.3-rc5
>    * Misc changes recommended by Alex and Vitaly.
>    * Rename APICV_DEACTIVATED to APICV_SUSPENDED
>    * Disable AVIC when guest booting w/ SVM support since AVIC
>      does not currently support guest w/ nested virt.
>    * Add tracepoint for APICV activate/deactivate request. (per Alex)
>    * Consolidate changes for handling EOI for kvm PIT emulation and
>      IOAPIC RTC handling in V2 into ioapic_lazy_update_eoi() in
>      patch 17/18 of v3 serie.
>    * Remove patches for providing per-vm apicv_state debug information.
> 
> Changes from V1: (https://lkml.org/lkml/2019/3/22/1042)
>    * Introduce APICv state enumeration
>    * Introduce KVM debugfs for APICv state
>    * Add synchronization logic for APICv state to prevent potential
>      race condition (per Jan's suggestion)
>    * Add support for activate/deactivate posted interrupt
>      (per Jan's suggestion)
>    * Remove callback functions for handling APIC ID, DFR and LDR update
>      (per Paolo's suggestion)
>    * Add workaround for handling EOI for in-kernel PIT and IOAPIC.
> 
> Suravee Suthikulpanit (16):
>    kvm: x86: Modify kvm_x86_ops.get_enable_apicv() to use struct kvm
>      parameter
>    kvm: x86: Introduce KVM APICv state
>    kvm: lapic: Introduce APICv update helper function
>    kvm: x86: Add support for activate/de-activate APICv at runtime
>    kvm: x86: Add APICv activate/deactivate request trace points
>    kvm: x86: svm: Add support to activate/deactivate posted interrupts
>    svm: Add support for setup/destroy virutal APIC backing page for AVIC
>    svm: Add support for activate/deactivate AVIC at runtime
>    kvm: x86: hyperv: Use APICv deactivate request interface
>    svm: Disable AVIC when launching guest with SVM support
>    svm: Temporary deactivate AVIC during ExtINT handling
>    kvm: x86: Introduce struct kvm_x86_ops.apicv_eoi_accelerate
>    kvm: lapic: Clean up APIC predefined macros
>    kvm: ioapic: Refactor kvm_ioapic_update_eoi()
>    kvm: x86: ioapic: Lazy update IOAPIC EOI
>    svm: Allow AVIC with in-kernel irqchip mode
> 
>   arch/x86/include/asm/kvm_host.h |  28 +++++-
>   arch/x86/kvm/hyperv.c           |  12 ++-
>   arch/x86/kvm/ioapic.c           | 149 +++++++++++++++++++-----------
>   arch/x86/kvm/lapic.c            |  35 ++++---
>   arch/x86/kvm/lapic.h            |   2 +
>   arch/x86/kvm/svm.c              | 198 +++++++++++++++++++++++++++++++++++++---
>   arch/x86/kvm/trace.h            |  30 ++++++
>   arch/x86/kvm/vmx/vmx.c          |   2 +-
>   arch/x86/kvm/x86.c              | 136 ++++++++++++++++++++++++++-
>   9 files changed, 506 insertions(+), 86 deletions(-)
>