[PATCH v2 00/25] Add AMD Secure Nested Paging (SEV-SNP) Initialization Support

[Michael Roth]

This patchset is also available at:

  https://github.com/amdese/linux/commits/snp-host-init-v2

and is based on top of linux-next tag next-20240125

These patches were originally included in v10 of the SNP KVM/hypervisor
patches[1], but have been split off from the general KVM support for easier
review and eventual merging into the x86 tree. They are based on linux-next
to help stay in sync with both tip and kvm-next.

There is 1 KVM-specific patch here since it is needed to avoid regressions
when running legacy SEV guests while the RMP table is enabled.

== OVERVIEW ==

AMD EPYC systems utilizing Zen 3 and newer microarchitectures add support
for a new feature called SEV-SNP, which adds Secure Nested Paging support
on top of the SEV/SEV-ES support already present on existing EPYC systems.

One of the main features of SNP is the addition of an RMP (Reverse Map)
table to enforce additional security protections for private guest memory.
This series primarily focuses on the various host initialization
requirements for enabling SNP on the system, while the actual KVM support
for running SNP guests is added as a separate series based on top of these
patches.

The basic requirements to initialize SNP support on a host when the feature
has been enabled in the BIOS are:

  - Discovering and initializing the RMP table
  - Initializing various MSRs to enable the capability across CPUs
  - Various tasks to maintain legacy functionality on the system, such as:
    - Setting up hooks for handling RMP-related changes for IOMMU pages
    - Initializing SNP in the firmware via the CCP driver, and implement
      additional requirements needed for continued operation of legacy
      SEV/SEV-ES guests

Additionally some basic SEV ioctl interfaces are added to configure various
aspects of SNP-enabled firmwares via the CCP driver.

More details are available in the SEV-SNP Firmware ABI[2].

Feedback/review is very much appreciated!

-Mike

[1] https://lore.kernel.org/kvm/20231016132819.1002933-1-michael.roth@xxxxxxx/ 
[2] https://www.amd.com/en/developer/sev.html 

Changes since v1:

 * rebased on linux-next tag next-202401125
 * patch 26: crypto: ccp: Add the SNP_SET_CONFIG command
   - documentation updates (Boris)
 * patch 25: crypto: ccp: Add the SNP_COMMIT command
   - s/len/length/ in SNP_COMMIT kernel-doc (Boris)
   - s/length/len/ in struct for consistency with other cmds/structs
 * patch 24: crypto: ccp: Add the SNP_PLATFORM_STATUS command
   - add comments regarding the need for page reclaim of status page (Boris)
 * patch 21: crypto: ccp: Add panic notifier for SEV/SNP firmware shutdown on kdump
   - enable crash_kexec_post_notifiers by default for SNP (Ashish)
   - squash in refactorings from Boris, but keep wbinvd_on_all_cpus() so that
     non-SNP/non-panic case is still handled as it was prior to SNP.
 * patch 20: crypto: ccp: Add debug support for decrypting pages
   - not utilized in current code, dropped it for now (Sean)
 * patch 18: crypto: ccp: Handle legacy SEV commands when SNP is enabled
   - drop uneeded index variables used when rolling back descriptor mappings
     when failure occurs (Boris)
   - error message fixups (Boris)
   - fix indentation to align each argument description line to same column
     as recommended by kernel-doc documentation
   - attempt to unmap/reclaim all descriptors even if a previous unmap/reclaim
     failed
   - s/restored to/restored to hypervisor-owned/ in commit message
   - fix reclaim clobbering return value when a command failure occurs
   - reclaim cmd buffers and descriptors separately so that active cmd buffers
     can be marked !inuse even when reclaim for a descriptor fails
 * patch 17: crypto: ccp: Handle non-volatile INIT_EX data when SNP is enabled
   - drop usage of vmap() to access INIT_EX non-volatile data buffer, no longer
     needed now that rmpupdate() splits directmap rather than removes mappings
 * patch 15: x86/sev: Introduce snp leaked pages list
   - add Suggested-by: Vlastimil Babka <vbabka@xxxxxxx>
   - avoid the need to allocate memory when leaking unreclaimable pages
     (Vlastimil, Ashish)
   - use pr_warn() instead of pr_debug() when leaking (Sean)
 * patch 14: crypto: ccp: Provide API to issue SEV and SNP commands
   - s/SEV/SEV device/ in kernel-doc for sev_do_cmd() (Boris) 
 * patch 13: crypto: ccp: Add support to initialize the AMD-SP for SEV-SNP
   - account for non-page-aligned HV-Fixed ranges when calling SNP_INIT (Ashish)
   - various comment/commit fixups (Boris)
 * patch 12: crypto: ccp: Define the SEV-SNP commands
   - fix alignment of SEV_CMD_SNP_DOWNLOAD_FIRMWARE_EX enum
   - add "Incoming Migration Image" in all places that mention of IMI for
     purposes of documentation (Boris)
   - fix indentation to align each argument description line to same column
     in accordance with Documentation/doc-guide/kernel-doc.rst
   - rename sev_data_snp_addr.paddr_gctx to more appropriate
     sev_data_snp_addr.address, fix up affected patches
   - len/length in kernel-doc descriptions
 * patch 11: x86/sev: Invalidate pages from the direct map when adding them to the RMP table
   - replace old patch/handling with an implementation based on set_memory_4k()
     that splits directmap as-needed rather than adding/removing mappings in
     response to shared/private conversions.
   - lookup/confirm the mappings in advance of splitting, and avoid unecessary
     splits for 2M private ranges to reduce unecessary contention / TLB flushing
     (Dave, Boris, Tom, Mike, Vlastimil)
 * patch 10: x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction
   - add additional comments clarifying RMPUPDATE retry logic (Boris)
 * patch 07: x86/fault: Add helper for dumping RMP entries
   - fix dump logic for 2MB+ pages (Tom)
   - dump full 2MB range if particular 4K RMP entry is not populated (Boris)
   - drop EXPORT_SYMBOL_GPL() until actually needed by a module (Boris)
 * patch 04: x86/sev: Add the host SEV-SNP initialization support
   - macro alignment fixups (Boris)
   - commit message cleanups (Boris)
   - work in Zen5 patch (Boris)
   - squash __snp_init_rmptable in helper back in (Boris)
 * patch 03: iommu/amd: Don't rely on external callers to enable IOMMU SNP support
   - add Joerg's acked-by
   - error message cleanups (Boris)

Changes since being split off from v10 hypervisor patches:

 * Move all host initialization patches to beginning of overall series and
   post as a separate patchset. Include only KVM patches that are necessary
   for maintaining legacy SVM/SEV functionality with SNP enabled.
   (Paolo, Boris)
 * Don't enable X86_FEATURE_SEV_SNP until all patches are in place to maintain
   legacy SVM/SEV/SEV-ES functionality when RMP table and SNP firmware support
   are enabled. (Paolo)
 * Re-write how firmware-owned buffers are handled when dealing with legacy
   SEV commands. Allocate on-demand rather than relying on pre-allocated pool,
   use a descriptor format for handling nested/pointer params instead of
   relying on macros. (Boris)
 * Don't introduce sev-host.h, re-use sev.h (Boris)
 * Various renames, cleanups, refactorings throughout the tree (Boris)
 * Rework leaked pages handling (Vlastimil)
 * Fix kernel-doc errors introduced by series (Boris)
 * Fix warnings when onlining/offlining CPUs (Jeremi, Ashish)
 * Ensure X86_FEATURE_SEV_SNP is cleared early enough that AutoIBRS will still
   be enabled if RMP table support is not available/configured. (Tom)
 * Only read the RMP base/end MSR values once via BSP (Boris)
 * Handle IOMMU SNP setup automatically based on state machine rather than via
   external caller (Boris)

----------------------------------------------------------------
Ashish Kalra (5):
      iommu/amd: Don't rely on external callers to enable IOMMU SNP support
      x86/mtrr: Don't print errors if MtrrFixDramModEn is set when SNP enabled
      x86/sev: Introduce snp leaked pages list
      iommu/amd: Clean up RMP entries for IOMMU pages during SNP shutdown
      crypto: ccp: Add panic notifier for SEV/SNP firmware shutdown on kdump

Brijesh Singh (14):
      x86/cpufeatures: Add SEV-SNP CPU feature
      x86/sev: Add the host SEV-SNP initialization support
      x86/sev: Add RMP entry lookup helpers
      x86/fault: Add helper for dumping RMP entries
      x86/traps: Define RMP violation #PF error code
      x86/sev: Add helper functions for RMPUPDATE and PSMASH instruction
      crypto: ccp: Define the SEV-SNP commands
      crypto: ccp: Add support to initialize the AMD-SP for SEV-SNP
      crypto: ccp: Provide API to issue SEV and SNP commands
      crypto: ccp: Handle the legacy TMR allocation when SNP is enabled
      crypto: ccp: Handle legacy SEV commands when SNP is enabled
      KVM: SEV: Make AVIC backing, VMSA and VMCB memory allocation SNP safe
      crypto: ccp: Add the SNP_PLATFORM_STATUS command
      crypto: ccp: Add the SNP_SET_CONFIG command

Kim Phillips (1):
      x86/speculation: Do not enable Automatic IBRS if SEV SNP is enabled

Michael Roth (3):
      x86/fault: Dump RMP table information when RMP page faults occur
      x86/sev: Adjust directmap to avoid inadvertant RMP faults
      x86/cpufeatures: Enable/unmask SEV-SNP CPU feature

Tom Lendacky (2):
      crypto: ccp: Handle non-volatile INIT_EX data when SNP is enabled
      crypto: ccp: Add the SNP_COMMIT command

 Documentation/virt/coco/sev-guest.rst    |   51 ++
 arch/x86/Kbuild                          |    2 +
 arch/x86/include/asm/cpufeatures.h       |    1 +
 arch/x86/include/asm/disabled-features.h |    8 +-
 arch/x86/include/asm/iommu.h             |    1 +
 arch/x86/include/asm/kvm-x86-ops.h       |    1 +
 arch/x86/include/asm/kvm_host.h          |    1 +
 arch/x86/include/asm/msr-index.h         |   11 +-
 arch/x86/include/asm/sev.h               |   38 +
 arch/x86/include/asm/trap_pf.h           |   20 +-
 arch/x86/kernel/cpu/amd.c                |   21 +-
 arch/x86/kernel/cpu/common.c             |    7 +-
 arch/x86/kernel/cpu/mtrr/generic.c       |    3 +
 arch/x86/kernel/crash.c                  |    3 +
 arch/x86/kernel/sev.c                    |   10 +
 arch/x86/kvm/lapic.c                     |    5 +-
 arch/x86/kvm/svm/nested.c                |    2 +-
 arch/x86/kvm/svm/sev.c                   |   37 +-
 arch/x86/kvm/svm/svm.c                   |   17 +-
 arch/x86/kvm/svm/svm.h                   |    1 +
 arch/x86/mm/fault.c                      |    5 +
 arch/x86/virt/svm/Makefile               |    3 +
 arch/x86/virt/svm/sev.c                  |  547 +++++++++++++++
 drivers/crypto/ccp/sev-dev.c             | 1122 ++++++++++++++++++++++++++++--
 drivers/crypto/ccp/sev-dev.h             |    5 +
 drivers/iommu/amd/amd_iommu.h            |    1 -
 drivers/iommu/amd/init.c                 |  120 +++-
 include/linux/amd-iommu.h                |    6 +-
 include/linux/psp-sev.h                  |  319 ++++++++-
 include/uapi/linux/psp-sev.h             |   59 ++
 tools/arch/x86/include/asm/cpufeatures.h |    1 +
 31 files changed, 2303 insertions(+), 125 deletions(-)
 create mode 100644 arch/x86/virt/svm/Makefile
 create mode 100644 arch/x86/virt/svm/sev.c