RE: [PATCH V1 2/2] Drivers: hv: Fix bugs and issues around repeat hypercall

From: Michael Kelley

Date: Sun Nov 15 2026 - 18:48:18 EST


From: Mukesh R <mrathor@xxxxxxxxxxxxxxxxxxx> Sent: Friday, November 13, 2026 6:53 PM
>
> There are few issues/bugs around repeat hypercall:
> o If the hypercall fails, it could still have processed some entries.
> Fix places where it assumes none of the entries were processed and will
> try to undo entries already processed or print incorrect completions.
>
> o Hypercall returns back with timeouts only for HW confidential VMs. In
> that case it should be retried instead of returning back. Returning back
> results in error conditions and take down of VMs in most cases.
>
> o For non confidential VMs, the hypercall could be streamlined as it will
> never return back with timeouts.
>
> o Rename HV_STATUS_TIME_OUT to HV_STATUS_TIMEOUT to match hypervisor
> public header file. There is no reason for it to be different.
>
> o Minor formatting fix for checkpatch. Error printing missing in couple
> places when rep hypercall fails.
>
> Signed-off-by: Mukesh R <mrathor@xxxxxxxxxxxxxxxxxxx>

[snip]

> diff --git a/include/asm-generic/mshyperv.h b/include/asm-generic/mshyperv.h
> index 396bcb3db0bc..87c784b9daa6 100644
> --- a/include/asm-generic/mshyperv.h
> +++ b/include/asm-generic/mshyperv.h
> @@ -28,6 +28,9 @@
>
> #define VTPM_BASE_ADDRESS 0xfed40000
>
> +#define HV_REP_COUNT_MAX \
> + (HV_HYPERCALL_REP_COMP_MASK >> HV_HYPERCALL_REP_COMP_OFFSET)

Sashiko noted that this new #define is never used anywhere.

> +
> enum hv_partition_type {
> HV_PARTITION_TYPE_GUEST,
> HV_PARTITION_TYPE_ROOT,
> @@ -128,6 +131,11 @@ static inline bool hv_result_success(u64 status)
> return hv_result(status) == HV_STATUS_SUCCESS;
> }
>
> +static inline bool hv_result_timeout(u64 status)
> +{
> + return hv_result(status) == HV_STATUS_TIMEOUT;
> +}
> +
> static inline unsigned int hv_repcomp(u64 status)
> {
> /* Bits [43:32] of status have 'Reps completed' data. */
> @@ -135,45 +143,63 @@ static inline unsigned int hv_repcomp(u64 status)
> HV_HYPERCALL_REP_COMP_OFFSET;
> }
>
> -/*
> - * Rep hypercalls. Callers of this functions are supposed to ensure that
> - * rep_count, varhead_size, and rep_start comply with Hyper-V hypercall
> - * definition.
> - */
> -static inline u64 hv_do_rep_hypercall_ex(u16 code, u16 rep_count,
> - u16 varhead_size, u16 rep_start,
> - void *input, void *output)
> +static inline u64 hv_do_rep_hypercall_reg(u16 code, u16 rep_count,
> + u16 varhead_size, u16 rep_start,
> + void *input, void *output)
> {
> u64 control = code;
> - u64 status;
> - u16 rep_comp;
>
> control |= (u64)varhead_size << HV_HYPERCALL_VARHEAD_OFFSET;
> control |= (u64)rep_count << HV_HYPERCALL_REP_COMP_OFFSET;
> control |= (u64)rep_start << HV_HYPERCALL_REP_START_OFFSET;
>
> - do {

Dropped the looping behavior here may be much more significant
than just "streamlining" as described in the commit message. The looping
behavior was included when rep hypercalls were originally added in
commit 806c89273bab in kernel 4.14, long before any SNP/TDX/CCA
functionality was added. So it was present for "normal" VMs doing
hypercalls on x86/x64 the normal way. The looping ensured that
rep hypercalls could not return with no errors but with the completed
rep count less than the requested rep count.

However, in my mind it *is* debatable whether that scenario could
ever occur. The Hyper-V TLFS describes "Hypercall Continuation" [1]
as essentially transparent to the caller. If a rep hypercall stops
processing reps because of hitting the 50 microsecond time limit,
it should restart and continue with the next set of reps without
the caller having to loop. Commit 806c89273bab mentions
hypercall continuation, but does not describe why the explicit
looping was added or how that aligns with the TLFS statements.
Vitaly -- it was a long time ago, but do you have any recollection?

Have you gotten an explicit assurance from the Hyper-V team
that the transparent hypercall continuation really does work,
and the explicit looping has all along been unnecessary? This
assurance would be needed back to and including Windows
Server 2016, which is now the oldest Hyper-V version supported
by Linux. Also note that the rep hypercall code is architecture
independent, so the transparent hypercall continuation would
also need to work on arm64 when the hypercall is invoked via
the standard path of using the HVC instruction (the TLFS doesn't
cover arm64, so this is unclear).

If you have such assurance, then I'm good with the removal of
the looping. But please make the removal a separate patch, with
sufficient discussion in the commit message to justify why it's OK
to do the removal (including that evidently the original commit
misinterpreted how continuation worked). And instead of looping,
perhaps do a sanity check that the completed rep count matches
the requested rep count.

Without such assurance, I think the looping needs to stay. Callers
are not prepared for a rep hypercall to return with no errors, but
with the completed rep count less than the requested rep count.

[1] https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/tlfs/hypercall-interface#hypercall-continuation

> + return hv_do_hypercall(control, input, output);
> +}
> +
> +static inline u64 hv_do_rep_hypercall_cvm(u16 code, u16 rep_count,
> + u16 varhead_size, u16 rep_start,
> + void *input, void *output)
> +{
> + u64 status, control = code;
> + u16 rep_comp = 0;
> +
> + control |= (u64)varhead_size << HV_HYPERCALL_VARHEAD_OFFSET;
> + control |= (u64)rep_count << HV_HYPERCALL_REP_COMP_OFFSET;
> + control |= (u64)rep_start << HV_HYPERCALL_REP_START_OFFSET;
> +
> + for (;;) {
> status = hv_do_hypercall(control, input, output);
> - if (!hv_result_success(status))
> +
> + if (!hv_result_timeout(status))
> return status;
>
> rep_comp = hv_repcomp(status);
> -
> control &= ~HV_HYPERCALL_REP_START_MASK;
> control |= (u64)rep_comp << HV_HYPERCALL_REP_START_OFFSET;
>
> touch_nmi_watchdog();
> - } while (rep_comp < rep_count);
> + }
>
> return status;

And Sashiko noted that this "return status" is dead code. The only
way to exit this function is via the "return status" up in the loop.

> }
>
> -/* For the typical case where rep_start is 0 */
> +/*
> + * Rep hypercalls for HW confidential VMs return back when timeouts occur,
> + * whereas for regular VMs they return only on success or failure. In case
> + * of successful completion, entire rep_count is done. But a hypercall could
> + * fail with partial completions, hence callers must always check that
> + * in case of failure. Lastly, a rep_count value of 0 is allowed for some
> + * hypercalls.
> + */
> static inline u64 hv_do_rep_hypercall(u16 code, u16 rep_count, u16 varhead_size,
> void *input, void *output)
> {
> - return hv_do_rep_hypercall_ex(code, rep_count, varhead_size, 0,
> - input, output);
> + if (hv_isolation_type_hw())

I don't think this is the right test, so let me explain. You don't describe the
context for when HV_STATUS_TIMEOUT may be returned, and there's no
TLFS documentation of this status, but I'm assuming the context is hypercall
continuation. This isn't the typical "something isn't responding" timeout. I
suspect it means the hypervisor hit the 50 us limit, and returned to the guest
VM with some reps still to be done. In a normal VM when a hypercall is
initiated by the hypercall page on x86/x64 or the HVC instruction on arm64,
the transparent hypercall continuation mechanism makes a TIMEOUT status
unnecessary. But with SNP, TDX, and CCA, hypercall invocation may be
piggybacked on other transports, and transparent continuation isn't available.
So Hyper-V returns HV_STATUS_TIMEOUT with the expectation that the
invoking code will loop to do the remaining reps.

The key factor is whether the hypercall is piggybacked on other transports.
The tests in hv_isolation_type_hw() don't capture that. For example, if
a paravisor is present, hv_isolation_type_snp() will return "true" even
though hypercalls are invoked via the standard x86/x64 hypercall page.
The hypercalls trap to the paravisor. If the paravisor is functioning properly,
it should do transparent hypercall continuation just like the hypervisor would.
But this is something to be confirmed with the hypervisor and paravisor teams.

Piggybacking is done only when no paravisor is present, so the
hv_do_rep_hypercall_cvm() path should be taken only in that circumstance.
Your current code works, but it goes down the more expensive
hv_do_rep_hypercall_cvm() path when it doesn't need to. (I'm not completely
clear on how arm64 works with and without a paravisor, so my statement
applies only to SNP and TDX. Need to sort out the CCA case with Kameron.)

Given the non-trivial complexity in sorting all this out, I wonder if
distinguishing the _cvm and _reg cases is really worth it. The current
code for the _reg case is admittedly inefficient because it does work for
the looping case before likely deciding that no looping is needed. But
that could easily be fixed. When no looping is necessary, which is the
vast majority of cases, the looping code could end up being only a
couple of instructions slower than a separately coded non-looping
version (I did some builds and looked at the generated code). Since
there's overhead in deciding which to use, there may not be any
performance win from having both versions.

Michael

> + return hv_do_rep_hypercall_cvm(code, rep_count, varhead_size, 0,
> + input, output);
> + else
> + return hv_do_rep_hypercall_reg(code, rep_count, varhead_size, 0,
> + input, output);
> }
>