Re: [PATCHv3 2/2] kvm: x86: implement KVM PM-notifier

[Sergey Senozhatsky]

On (21/06/07 15:54), Maxim Levitsky wrote:
[..]
> Also I would like to add my .02 cents on my observations on what happens when I suspend my system
> with guests running, which I do once in a while.
> I haven't dug deep into it yet as host suspend with VM running wasn't high on my priority list.
>  
> First of all after a host suspend/resume cycle (and that is true on all 3 machines I own),
> the host TSC is reset to 0 on all CPUs, thus while it is still synchronized, it jumps backward.
>  
> Host kernel has no issues coping with this.
>  
> Guests however complain about clocksource watchdog and mark the tsc clocksource as unstable,
> at least when invtsc is used (regardless of this patch, I wasn't able to notice a difference
> with and without it yet).
>  
>  
> [  287.515864] clocksource: timekeeping watchdog on CPU0: Marking clocksource 'tsc' as unstable because the skew is too large:
> [  287.516926] clocksource:                       'kvm-clock' wd_now: 4437767926 wd_last: 429c3c42f5 mask: ffffffffffffffff
> [  287.527100] clocksource:                       'tsc' cs_now: c33f6ce157 cs_last: c1be2ad19f mask: ffffffffffffffff
> [  287.528493] tsc: Marking TSC unstable due to clocksource watchdog
> [  287.556640] clocksource: Switched to clocksource kvm-clock
>  
>  
> This is from Intel system with stable TSC, but I have seen this on my AMD systems as well,
> but these have other issues which might affect this (see below).
>  
> AFAIK, we have code in kvm_arch_hardware_enable for this exact case but it might not work
> correctly or be not enough to deal with this.
>  
> Also I notice that this code sets kvm->arch.backwards_tsc_observed = true which 
> in turn disables the master clock which is not good as well.
>  
> I haven't yet allocated time to investigate this.
>  
>  
> Another bit of information which I didn't start a discussion (but I think I should), 
> which is relevant to AMD systems, is in 'unsynchronized_tsc' function.
> 
> On AMD guest it will mark the TSC as unstable in the guest as long as invtsc is not used.
> I patched that code out for myself, that is why I am mentioning it.

If you are going to fork this discussion the could you please Cc Suleiman
and me? I believe we are having a bunch of problems with the guest clocks
here on our side.