[PART1 RFC 0/9] KVM: x86: Introduce SVM AVIC support

[Suravee Suthikulpanit]

From: Suravee Suthikulpanit <Suravee.Suthikulpanit@xxxxxxx>

OVERVIEW
========
This patch set is the first of the two-part patch series to introduce 
the new AMD Advance Virtual Interrupt Controller (AVIC) support.

Basically, SVM AVIC hardware virtualizes local APIC registers of each
vCPU via the virtual APIC (vAPIC) backing page. This allows guest access
to certain APIC registers without the need to emulate the hardware behavior
in the hypervisor. More information about AVIC can be found in the
AMD64 Architecture Programmerâs Manual Volume 2 - System Programming.

  http://support.amd.com/TechDocs/24593.pdf

For SVM AVIC, we extend the existing kvm_amd driver to:
  * Check CPUID to detect AVIC support in the processor
  * Program new fields in VMCB to enable AVIC
  * Introduce new AVIC data structures and add code to manage them
  * Handle two new AVIC #VMEXITs
  * Add new interrupt intjection code using vAPIC backing page
    instead of the existing V_IRQ, V_INTR_PRIO, V_INTR_VECTOR,
    and V_IGN_TPR fileds

Currently, this patch series does not enable AVIC by default.
Users can enable SVM AVIC by specifying avic=1 during insmod kvm-amd.

Later, in part 2, we will introduce the IOMMU AVIC support, which
provides speed up for PCI device passthrough use case by allowing
the IOMMU hardware to inject interrupt directly into the guest via
the vAPIC backing page.

PERFORMANCE RESULTS
===================
Currently, AVIC is supported in the AMD family 15h models 6Xh
(Carrizo) processors. Therefore, it is used to collect the 
perforamance data shown below.

Generaly, SVM AVIC alone (w/o IOMMU AVIC) should provide speedup for
IPI interrupt since hypervisor does not require VMEXIT to inject
these interrupts. Also, it should speed up the case when hypervisor
wants to inject an interrupt into a running guest by setting the
corresponded IRR bit in the vAPIC backing page and trigger
AVIC_DOORBELL MSR.

IPI PERFORMANCE
===============
For IPI, I have collected some performance number on 2 and 4 CPU running
hackbech with the following detail:

  hackbench -p -l 100000
  Running in process mode with 10 groups using 40 file descriptors each (== 400 tasks)
  Each sender will pass 100000 messages of 100 bytes

                |    2 vcpus    |    4 vcpus 
 ------------------------------------------------
         Vanila |  273.76       |  190.21	
  AVIC disabled |  260.51 (~5%) |  184.40 (~5%)
          AVIC  |  239.03 (~10%)|  166.37 (~10%)

OVERALL PERFORMANCE
===================
Enabling AVIC should helps speeding up workloads, which generate
large amount of interrupts. However, it requires additional logics to:
  * Maintain AVIC-specific data structures during vCPU load/unload
    due to schedule in/out.
  * Track and manange interrupt pending in vAPIC backing page.

The goal is to minimize the overhead of AVIC in most cases, so that
we can achieve equivalent or improvement in overall performance when
enabling AVIC.

This is an on-going investigation and to be discussed.

CURRENT UNSUPPORT USE-CASES
===========================
    - Nested VM
    - VM Migration

GITHUB
======
Latest git tree can be found at:
    http://github.com/ssuthiku/linux.git    avic_part1_rfc

Any feedback and comments are very much appreciated.

Thank you,
Suravee

Suravee Suthikulpanit (9):
  KVM: x86: Misc LAPIC changes to exposes helper functions
  svm: Introduce new AVIC VMCB registers
  svm: clean up V_TPR, V_IRQ, V_INTR_PRIO, and V_INTR_MASKING
  KVM: x86: Detect and Initialize AVIC support
  svm: Add VMEXIT handlers for AVIC
  svm: Add interrupt injection via AVIC
  svm: Do not expose x2APIC when enable AVIC
  svm: Do not intercept CR8 when enable AVIC
  svm: Manage vcpu load/unload when enable AVIC

 arch/x86/include/asm/cpufeature.h |   1 +
 arch/x86/include/asm/kvm_host.h   |   4 +
 arch/x86/include/asm/msr-index.h  |   1 +
 arch/x86/include/asm/svm.h        |  38 +-
 arch/x86/include/uapi/asm/svm.h   |   9 +-
 arch/x86/kernel/cpu/scattered.c   |   1 +
 arch/x86/kvm/lapic.c              |  53 +--
 arch/x86/kvm/lapic.h              |   5 +
 arch/x86/kvm/svm.c                | 882 +++++++++++++++++++++++++++++++++++++-
 arch/x86/kvm/x86.c                |   4 +-
 10 files changed, 940 insertions(+), 58 deletions(-)

-- 
1.9.1