[PATCH v11 00/23] KVM: x86: Cleaning up the KVM clock mess, part 2

[David Woodhouse]

From: David Woodhouse <dwmw@xxxxxxxxxxxx>

This is the second half of the KVM clock series which has been churning
since about 2024. The first half¹ is upstream as of 7.4-rc1 (yay!).
This is the remainder — the masterclock overhaul, and new API for
fixing migration.

The KVM clock is defined, in the sane case, as a simple arithmetic
function of the guest TSC. The masterclock patches stop KVM from
breaking that definition gratuitously: TSC synchronization no longer
depends on wallclock-sampling heuristics where exact matching will do,
vCPUs with matching frequency but offset TSCs no longer disable the
master clock, and clock updates stop perturbing a perfectly good
reference point for no reason.

The uAPI half then lets userspace preserve the TSC→kvmclock
relationship precisely across migration and live update, instead of
clamping to a UTC reference with KVM_SET_CLOCK and giving the guest an
additional discontinuity: KVM_[GS]ET_CLOCK_GUEST transfers the actual
pvclock information, and KVM_VCPU_TSC_SCALE exposes the precise TSC
frequency and scaling. Selftests prove the lot, including a simulated
live migration which preserves the clock relationship to the limits of
arithmetic precision.

Tree: https://git.infradead.org/?p=users/dwmw2/linux.git;a=shortlog;h=refs/heads/kvmclock11

Since v7² (the last time this half was posted):

 • Rebase onto part 1 as merged in 7.4-rc1.
 • KVM_GET_CLOCK_GUEST no longer writes guest memory; it computes the
   pvclock information directly from the master clock reference point,
   and returns -ENODATA when not in master clock mode. Invoking
   kvm_guest_time_update() from the GET path dirtied guest pages after
   the final dirty-log pass during live migration, which was unkind.
 • KVM_CAP_CLOCK_GUEST renumbered; 251 was taken in the meantime.
 • A TSC synchronization now clears the vCPU's pending backwards-TSC
   compensation, which is already folded into the matched offset it
   reuses; applying it again at the next vcpu_load doubled the delta.
 • Re-synchronizing the TSC after KVM_SET_TSC_KHZ joins the current
   generation for a vCPU which has never run (unless userspace already
   wrote its TSC), preserving the matched state of its siblings and
   with it PVCLOCK_TSC_STABLE_BIT; a vCPU whose scaling ratio differs
   from the generation's opens a new generation rather than inheriting
   an offset recorded in a different scaling domain.
 • The Hyper-V reenlightenment notifier invalidates the TSC page
   itself, now that the clock update consumes the pending masterclock
   requests which used to do it.
 • pvclock_migration_test now also covers vCPUs with mutually-offset
   TSCs: each vCPU's TSC is migrated individually, and a single
   KVM_SET_CLOCK_GUEST on vCPU0 restores the VM-wide clock.
 • New selftest: the first userspace TSC write is honoured exactly,
   rather than being snapped back by the 1-second slop heuristic.
 • Drop the open-coded fallback in kvm_xen_start_timer(); its excuse
   for not using get_kvmclock_ns() went away in part 1.
 • Use a single coherent time snapshot in kvm_vm_ioctl_set_clock()
   instead of sampling CLOCK_REALTIME and the kvmclock base separately.
 • Assorted hardening from pre-post review: KCSAN annotations for the
   benign lockless flag reads, documentation for -EINVAL and the
   same-vCPU requirement of KVM_[GS]ET_CLOCK_GUEST, and selftests that
   skip rather than fail on kernels and hosts without the prerequisites.

¹ https://lore.kernel.org/all/20260826213303.914988-1-seanjc@xxxxxxxxxx
² https://lore.kernel.org/all/20260728144954.355376-1-dwmw2@xxxxxxxxxxxxx


David Woodhouse (20):
  KVM: x86: Improve synchronization in kvm_synchronize_tsc()
  KVM: x86: Kill last_tsc_{nsec,write,offset} fields
  KVM: x86: Replace nr_vcpus_matched_tsc count with
    all_vcpus_matched_tsc bool
  KVM: x86: Allow KVM master clock mode when TSCs are offset from each
    other
  KVM: x86: Factor out kvm_use_master_clock()
  KVM: x86: Avoid gratuitous global clock updates
  KVM: x86: Avoid redundant masterclock updates from multiple vCPUs
  KVM: x86: Re-synchronize TSC after KVM_SET_TSC_KHZ
  KVM: x86: Cache host vclock_mode for masterclock eligibility checks
  KVM: x86: Remove pvclock_gtod_data and private timekeeping code
  KVM: x86: Activate master clock from kvm_arch_init_vm()
  KVM: selftests: Use UAPI pvclock-abi.h in xen_shinfo_test
  KVM: selftests: Add library helpers for clock and TSC-offset ioctls
  KVM: x86: Add KVM_VCPU_TSC_SCALE and fix the documentation on TSC
    migration
  KVM: selftests: Add master clock offset test
  KVM: selftests: Add Xen/generic CPUID timing leaf test
  KVM: selftests: Add Xen runstate migration test
  KVM: selftests: Test that the first user TSC write is honoured exactly
  KVM: x86/xen: Drop open-coded fallback clock read in
    kvm_xen_start_timer()
  KVM: x86: Use a single time snapshot in kvm_vm_ioctl_set_clock()

Jack Allister (3):
  UAPI: x86: Move pvclock-abi to UAPI for x86 platforms
  KVM: x86: Add KVM_[GS]ET_CLOCK_GUEST for accurate KVM clock migration
  KVM: selftests: Add KVM/PV clock selftest to prove timer correction

 Documentation/virt/kvm/api.rst                |  54 ++
 Documentation/virt/kvm/devices/vcpu.rst       | 130 +++-
 Documentation/virt/kvm/locking.rst            |   2 +-
 MAINTAINERS                                   |   4 +-
 arch/x86/include/asm/kvm_host.h               |  12 +-
 arch/x86/include/uapi/asm/kvm.h               |   7 +
 arch/x86/include/{ => uapi}/asm/pvclock-abi.h |  27 +-
 arch/x86/kvm/msrs.c                           |  20 +-
 arch/x86/kvm/x86.c                            | 711 ++++++++++++++----
 arch/x86/kvm/x86.h                            |   2 +-
 arch/x86/kvm/xen.c                            |  40 +-
 include/uapi/linux/kvm.h                      |   4 +
 scripts/xen-hypercalls.sh                     |   2 +-
 tools/arch/x86/include/asm/pvclock-abi.h      |  27 +-
 tools/testing/selftests/kvm/Makefile.kvm      |   6 +
 .../selftests/kvm/include/x86/processor.h     |  50 ++
 .../kvm/x86/masterclock_offset_test.c         | 215 ++++++
 .../kvm/x86/pvclock_migration_test.c          | 520 +++++++++++++
 .../testing/selftests/kvm/x86/pvclock_test.c  | 447 +++++++++++
 .../selftests/kvm/x86/tsc_user_set_test.c     |  86 +++
 .../selftests/kvm/x86/xen_cpuid_timing_test.c | 273 +++++++
 .../selftests/kvm/x86/xen_migration_test.c    | 222 ++++++
 .../selftests/kvm/x86/xen_shinfo_test.c       |  31 +-
 23 files changed, 2625 insertions(+), 267 deletions(-)
 rename arch/x86/include/{ => uapi}/asm/pvclock-abi.h (82%)
 create mode 100644 tools/testing/selftests/kvm/x86/masterclock_offset_test.c
 create mode 100644 tools/testing/selftests/kvm/x86/pvclock_migration_test.c
 create mode 100644 tools/testing/selftests/kvm/x86/pvclock_test.c
 create mode 100644 tools/testing/selftests/kvm/x86/tsc_user_set_test.c
 create mode 100644 tools/testing/selftests/kvm/x86/xen_cpuid_timing_test.c
 create mode 100644 tools/testing/selftests/kvm/x86/xen_migration_test.c


base-commit: dea4f6380f4ec96e8f3b0aaf2343764bd0d25ef4
-- 
2.43.0