Re: [PATCH] KVM: arm/arm64: don't set vtimer->cnt_ctl in kvm_arch_timer_handler

[Christoffer Dall]

On Tue, Dec 12, 2017 at 11:00:07PM -0800, Jia He wrote:
> In our Armv8a server (qualcomm Amberwing, non VHE), after applying
> Christoffer's timer optimizing patchset(Optimize arch timer register
> handling), the guest is hang during kernel booting.
> 
> The error root cause might be as follows:
> 1. in kvm_arch_timer_handler, it reset vtimer->cnt_ctl with current
> cntv_ctl register value. And then it missed some cases to update timer's
> irq (irq.level) when kvm_timer_irq_can_fire() is false

Why should it set the irq level to true when the timer cannot fire?

> 2. It causes kvm_vcpu_check_block return 0 instead of -EINTR
> 	kvm_vcpu_check_block
> 		kvm_cpu_has_pending_timer
> 			kvm_timer_is_pending
> 				kvm_timer_should_fire
> 3. Thus, the kvm hyp code can not break the loop in kvm_vcpu_block (halt
> poll process) and the guest is hang forever

This is just a polling loop which will expire after some time, so it
shouldn't halt the guest indefinitely, but merely slow it down for some
while, if we have a bug.  Is that the behavior you're seeing or are you
seeing the guest coming to a complete halt?

> 
> Fixes: b103cc3f10c0 ("KVM: arm/arm64: Avoid timer save/restore in vcpu entry/exit")
> Signed-off-by: Jia He <jia.he@xxxxxxxxxxxxxxxx>
> ---
>  virt/kvm/arm/arch_timer.c | 1 -
>  1 file changed, 1 deletion(-)
> 
> diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
> index f9555b1..bb86433 100644
> --- a/virt/kvm/arm/arch_timer.c
> +++ b/virt/kvm/arm/arch_timer.c
> @@ -100,7 +100,6 @@ static irqreturn_t kvm_arch_timer_handler(int irq, void *dev_id)
>  	vtimer = vcpu_vtimer(vcpu);
>  
>  	if (!vtimer->irq.level) {
> -		vtimer->cnt_ctl = read_sysreg_el0(cntv_ctl);

This fix is clearly not correct, as it would prevent forwarding timer
interrupts in some cases.

>  		if (kvm_timer_irq_can_fire(vtimer))
>  			kvm_timer_update_irq(vcpu, true, vtimer);
>  	}
> -- 
> 2.7.4
> 

I actually don't see how the above scenario you painted can happen.

If you're in the polling loop, that means that the timer state is loaded
on the vcpu, and that means you can take interrupts from the timer, and
when you take interrupts, you will set the irq.level.

And here's the first bit of logic in kvm_timer_is_pending():

	if (vtimer->irq.level || ptimer->irq.level)
		return true;

So that would break the loop.

I'm not able to reproduce on my side with a non-VHE platform.

What is the workload you're running to reproduce this, and what is the
exact kernel tree and kernel configuration you're using?

Thanks,
-Christoffer