Re: [PATCH] x86/kvm: Show guest system/user cputime in cpustat

[Zhang, Yanmin]

On Thu, 2010-03-11 at 09:50 +0200, Avi Kivity wrote:
> On 03/11/2010 09:46 AM, Sheng Yang wrote:
> > On Thursday 11 March 2010 15:36:01 Avi Kivity wrote:
> >    
> >> On 03/11/2010 09:20 AM, Sheng Yang wrote:
> >>  

>     
> >>> Currently we can only get the cpu_stat of whole guest as one. This patch
> >>> enhanced cpu_stat with more detail, has guest_system and guest_user cpu
> >>> time statistics with a little overhead.
> >>>
> >>> Signed-off-by: Sheng Yang<sheng@xxxxxxxxxxxxxxx>
It seems per-process guest cpu utilization is more useful than per-cpu's.


> >>> ---
> >>>
> >>> This draft patch based on KVM upstream to show the idea. I would split it
> >>> into more kernel friendly version later.
> >>>
> >>> The overhead is, the cost of get_cpl() after each exit from guest.
> >>>        
> >> This can be very expensive in the nested virtualization case, so I
> >> wouldn't like this to be in normal paths.  I think detailed profiling
> >> like that can be left to 'perf kvm', which only has overhead if enabled
> >> at runtime.
> >>      
> > Yes, that's my concern too(though nested vmcs/vmcb read already too expensive,
> > they should be optimized...).
> 
> Any ideas on how to do that?  Perhaps use paravirt_ops to covert the 
> vmread into a memory read?  We store the vmwrites in the vmcs anyway.
Another method is to add sysctl entry, such like /proc/sys/kernel/collect_guest_utilization,
and we can set it off by default. Or add a /sys/kernel/debug/kvm/collect_guest_utilization.

> 
> > The other concern is, perf alike mechanism would
> > bring a lot more overhead compared to this.
> >    
> 
> Ordinarily users won't care if time is spent in guest kernel mode or 
> guest user mode.  They want to see which guest is imposing a load on a 
> system.  I consider a user profiling a guest from the host an advanced 
> and rarer use case, so it's okay to require tools and additional 
> overhead for this.
Here is the story why Sheng worked out the patch. Some guys work on
KVM performance. They want us to extend top to show guest utilization
info, such like guest kernel and guest userspace cpu utilization. With
the new tool, they could find which VM (mapping with qemu process id)
consumes too much cpu time in host space (including kernel and userspace),
and compare them with guest kernel/userspace. That information could provide
a first-hand high-level overview about all VMs running in the system
and help admin quickly find what the worst VM instance is.

So we need per-process (guest) cpu utilization than per-cpu guest utilization.

> 
> >> For example you can put the code to note the cpl in a tracepoint which
> >> is enabled dynamically.
> >>      
> > Yanmin have already implement "perf kvm" to support this. We are just arguing
> > if a normal top-alike mechanism is necessary.
perf kvm mostly is used to find hot functions which might cause more overhead.
Sheng's patch has less overhead.


> >
> > I am also considering to make it a feature that can be disabled. But seems it
> > make things complicate and result in uncertain cpustat output.
> >    
> 
> I'm not even sure that guest time was a good idea.


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/