Re: [patch V5 part 2 15/18] x86/kvm/svm: Handle hardirqs proper on guest enter/exit

[Paolo Bonzini]

On 07/05/20 16:44, Thomas Gleixner wrote:
> Entering guest mode is more or less the same as returning to user
> space. From an instrumentation point of view both leave kernel mode and the
> transition to guest or user mode reenables interrupts on the host. In user
> mode an interrupt is served directly and in guest mode it causes a VM exit
> which then handles or reinjects the interrupt.
> 
> The transition from guest mode or user mode to kernel mode disables
> interrupts, which needs to be recorded in instrumentation to set the
> correct state again.
> 
> This is important for e.g. latency analysis because otherwise the execution
> time in guest or user mode would be wrongly accounted as interrupt disabled
> and could trigger false positives.
> 
> Add hardirq tracing to guest enter/exit functions in the same way as it
> is done in the user mode enter/exit code, respecting the RCU requirements.
> 
> Signed-off-by: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
> Cc: Paolo Bonzini <pbonzini@xxxxxxxxxx>
> Cc: Sean Christopherson <sean.j.christopherson@xxxxxxxxx>
> ---
> V5: Adjust comments and changelog

Apart from the subject being svm and not vmx, it looks great.  Thanks!

Paolo

> ---
>  arch/x86/kvm/vmx/vmx.c |   27 +++++++++++++++++++++++++--
>  1 file changed, 25 insertions(+), 2 deletions(-)
> 
> --- a/arch/x86/kvm/vmx/vmx.c
> +++ b/arch/x86/kvm/vmx/vmx.c
> @@ -6604,9 +6604,21 @@ static void vmx_vcpu_run(struct kvm_vcpu
>  	x86_spec_ctrl_set_guest(vmx->spec_ctrl, 0);
>  
>  	/*
> -	 * Tell context tracking that this CPU is about to enter guest mode.
> +	 * VMENTER enables interrupts (host state), but the kernel state is
> +	 * interrupts disabled when this is invoked. Also tell RCU about
> +	 * it. This is the same logic as for exit_to_user_mode().
> +	 *
> +	 * This ensures that e.g. latency analysis on the host observes
> +	 * guest mode as interrupt enabled.
> +	 *
> +	 * guest_enter_irqoff() informs context tracking about the
> +	 * transition to guest mode and if enabled adjusts RCU state
> +	 * accordingly.
>  	 */
> +	trace_hardirqs_on_prepare();
> +	lockdep_hardirqs_on_prepare(CALLER_ADDR0);
>  	guest_enter_irqoff();
> +	lockdep_hardirqs_on(CALLER_ADDR0);
>  
>  	/* L1D Flush includes CPU buffer clear to mitigate MDS */
>  	if (static_branch_unlikely(&vmx_l1d_should_flush))
> @@ -6623,9 +6635,20 @@ static void vmx_vcpu_run(struct kvm_vcpu
>  	vcpu->arch.cr2 = read_cr2();
>  
>  	/*
> -	 * Tell context tracking that this CPU is back.
> +	 * VMEXIT disables interrupts (host state), but tracing and lockdep
> +	 * have them in state 'on' as recorded before entering guest mode.
> +	 * Same as enter_from_user_mode().
> +	 *
> +	 * guest_exit_irqoff() restores host context and reinstates RCU if
> +	 * enabled and required.
> +	 *
> +	 * This needs to be done before the below as native_read_msr()
> +	 * contains a tracepoint and x86_spec_ctrl_restore_host() calls
> +	 * into world and some more.
>  	 */
> +	lockdep_hardirqs_off(CALLER_ADDR0);
>  	guest_exit_irqoff();
> +	trace_hardirqs_off_prepare();
>  
>  	/*
>  	 * We do not use IBRS in the kernel. If this vCPU has used the
>