[PATCH] x86/tdx, KVM: fix HKID leak when kexec is initiated with active TDs

[Nowicki, Robert]

When kexec is initiated while TDs are running, vCPU threads can be
mid-TDH.VP.ENTER on other CPUs when tdx_shutdown() fires. The TDX
module rejects TDH.MNG.VPFLUSHDONE for a VP in RUNNING state, leaving
the HKID in a leaked state:

  kvm_intel: tdh_mng_vpflushdone() failed. HKID 33 is leaked.

Fix this by introducing a quiescing flag set at the start of
tdx_shutdown(). KVM's tdx_vcpu_run() checks the flag and returns
EXIT_FASTPATH_NONE before attempting TDH.VP.ENTER. After setting the
flag, tdx_shutdown() calls on_each_cpu(tdx_seam_sync) with wait=1 to
ensure any CPU currently inside TDH.VP.ENTER has exited SEAM before
tdx_sys_disable() is called.

Fixes: 58171ae22e11 ("x86/tdx: Disable the TDX module during kexec and kdump")
Signed-off-by: Nowicki, Robert <robert.nowicki@xxxxxxxxx>
---
 arch/x86/include/asm/tdx.h  |  2 ++
 arch/x86/kvm/vmx/tdx.c      |  3 +++
 arch/x86/virt/vmx/tdx/tdx.c | 12 ++++++++++++
 3 files changed, 17 insertions(+)

diff --git a/arch/x86/include/asm/tdx.h b/arch/x86/include/asm/tdx.h
index a0a4a15142fc..68a87bdbca9a 100644
--- a/arch/x86/include/asm/tdx.h
+++ b/arch/x86/include/asm/tdx.h
@@ -173,6 +173,7 @@ static inline int pg_level_to_tdx_sept_level(enum pg_level level)
 }
 
 void tdx_sys_disable(void);
+bool tdx_kexec_quiescing(void);
 
 u64 tdh_vp_enter(struct tdx_vp *vp, struct tdx_module_args *args);
 u64 tdh_mng_addcx(struct tdx_td *td, struct page *tdcs_page);
@@ -206,6 +207,7 @@ static inline u32 tdx_get_nr_guest_keyids(void) { return 0; }
 static inline const char *tdx_dump_mce_info(struct mce *m) { return NULL; }
 static inline const struct tdx_sys_info *tdx_get_sysinfo(void) { return NULL; }
 static inline void tdx_sys_disable(void) { }
+static inline bool tdx_kexec_quiescing(void) { return false; }
 #endif	/* CONFIG_INTEL_TDX_HOST */
 
 #endif /* !__ASSEMBLER__ */
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index 50a5cfdbd33e..2d658db7700d 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -1053,6 +1053,9 @@ fastpath_t tdx_vcpu_run(struct kvm_vcpu *vcpu, u64 run_flags)
 	struct vcpu_tdx *tdx = to_tdx(vcpu);
 	struct vcpu_vt *vt = to_vt(vcpu);
 
+	if (unlikely(tdx_kexec_quiescing()))
+		return EXIT_FASTPATH_NONE;
+
 	/*
 	 * WARN if KVM wants to force an immediate exit, as the TDX module does
 	 * not guarantee entry into the guest, i.e. it's possible for KVM to
diff --git a/arch/x86/virt/vmx/tdx/tdx.c b/arch/x86/virt/vmx/tdx/tdx.c
index aaf22a87717a..71c7e4fadda3 100644
--- a/arch/x86/virt/vmx/tdx/tdx.c
+++ b/arch/x86/virt/vmx/tdx/tdx.c
@@ -236,6 +236,16 @@ static void tdx_cpu_flush_cache(void)
 	this_cpu_write(cache_state_incoherent, false);
 }
 
+
+static atomic_t tdx_shutdown_in_progress = ATOMIC_INIT(0);
+
+bool tdx_kexec_quiescing(void)
+{
+	return atomic_read(&tdx_shutdown_in_progress);
+}
+EXPORT_SYMBOL_GPL(tdx_kexec_quiescing);
+
+static void tdx_seam_sync(void *ign) { }
 static void tdx_shutdown_cpu(void *ign)
 {
 	/*
@@ -252,6 +262,8 @@ static void tdx_shutdown_cpu(void *ign)
 
 static void tdx_shutdown(void *ign)
 {
+	atomic_set(&tdx_shutdown_in_progress, 1);
+	on_each_cpu(tdx_seam_sync, NULL, 1);
 	tdx_sys_disable();
 	on_each_cpu(tdx_shutdown_cpu, NULL, 1);
 }
-- 
2.53.0

---------------------------------------------------------------------
Intel Technology Poland sp. z o.o.
ul. Slowackiego 173 | 80-298 Gdansk | Sad Rejonowy Gdansk Polnoc | VII Wydzial Gospodarczy Krajowego Rejestru Sadowego - KRS 101882 | NIP 957-07-52-316 | Kapital zakladowy 200.000 PLN.
Spolka oswiadcza, ze posiada status duzego przedsiebiorcy w rozumieniu ustawy z dnia 8 marca 2013 r. o przeciwdzialaniu nadmiernym opoznieniom w transakcjach handlowych.

Ta wiadomosc wraz z zalacznikami jest przeznaczona dla okreslonego adresata i moze zawierac informacje poufne. W razie przypadkowego otrzymania tej wiadomosci, prosimy o powiadomienie nadawcy oraz trwale jej usuniecie; jakiekolwiek przegladanie lub rozpowszechnianie jest zabronione.
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). If you are not the intended recipient, please contact the sender and delete all copies; any review or distribution by others is strictly prohibited.