Re: [kvm-devel] [BUG] Oops with KVM-27

[Luca Tettamanti]

Il Thu, Jun 14, 2007 at 11:26:29AM +0300, Avi Kivity ha scritto: 
> Luca Tettamanti wrote:
> >With GOOD_APIC apic_read_around is a nop, while apic_write_around is a
> >normal write. With !GOOD_APIC apic_write_around writes to the APIC reg
> >using xchg. With !GOOD_APIC and this patch:
> >
> >--- include/asm-i386/apic.h~	2007-04-26 05:08:32.000000000 +0200
> >+++ include/asm-i386/apic.h	2007-06-13 22:35:00.000000000 +0200
> >@@ -56,7 +56,8 @@
> > static __inline fastcall void native_apic_write_atomic(unsigned long reg,
> > 						       unsigned long v)
> > {
> >-	xchg((volatile unsigned long *)(APIC_BASE+reg), v);
> >+//	xchg((volatile unsigned long *)(APIC_BASE+reg), v);
> >+	*((volatile unsigned long *)(APIC_BASE+reg)) = v;
> > }
> > 
> > static __inline fastcall unsigned long native_apic_read(unsigned long reg)
> >
> >The kernel boots fine. 
> >  
> 
> Looking at the xchg emulation code, it seems fine, but clearly it 
> isn't.  Can you add logging to the kernel apic driver and to the qemu 
> device emulation (qemu/hw/apic.c, apic_mem_readl()/apic_mem_writel()) 
> and compare the results?

Done, but at this point I don't know what I'm looking at ;)

I'm attaching logs for working and non working kernels. I've made
apic_read_around() a nop in both cases since it doesn't influence the
outcome; the only difference is that working kernel writes directly to
the mapped APIC registers while the non-working one uses xchg.

As expected kernel side logs show zero differences (except for timer
calibration).

QEMU is a different story. The most obvious difference is that in
non-working case each write is preceded by a read at the same register
(due to xchg).

What is strange is that almost all the writes done using
apic_write_atomic are not hitting QEMU (see qemu.diff). In the log
there's only the read generated by the xchg.

For example, during timer calibration the kernel is doing:

write to 0x0b0
read from 0x390

but - in the non-working case - QEMU sees:

read from 0x0b0 (probably generated by xchg)
read from 0x390

but it doesn't see the write!

The write is not always lost:

apic_read: 0xffffd030 = 0x00050011
apic_write_atomic: 0xffffd370 = 0x000100fe
...
apic_read: fee00030 = 00050011
apic_read: fee00370 = 00010000
apic_write: fee00370 = 000100fe

Luca
-- 
> While we're on all of this, are we going to change "tained" to some
> other less alarmist word?
"screwed" -- Alexander Viro

Attachment: apic-logs.tar.bz2
Description: Binary data