kvm/qemu server "freeze" with Kernel BUG

From: samuel_wolf@xxxxxxxxxxx
Date: Thu Feb 03 2011 - 06:42:10 EST


I have running a Ubuntu 10.04 Server on a Xeon CPU with kvm/qemu.
Since a kernel update few weeks ago, sometimes my VM die and the VM host too.
I can ping the VM host and I get an answer but can not login via ssh.

qemu-kvm 0.12.3+noroms-0ubuntu9.3
Linux vmhost 2.6.32-28-server #55-Ubuntu SMP Mon Jan 10 23:57:16 UTC 2011 x86_64 GNU/Linux

Looking directly on the screen shows me:
Jan 29 16:43:24 vmhost kernel: [480258.278932] ------------[ cut here ]------------
Jan 29 16:43:24 vmhost kernel: [480258.279129] kernel BUG at /build/buildd/linux-2.6.32/arch/x86/kvm/mmu.c:635!
Jan 29 16:43:24 vmhost kernel: [480258.279397] invalid opcode: 0000 [#1] SMP
Jan 29 16:43:24 vmhost kernel: [480258.279682] last sysfs file: /sys/devices/pci0000:00/0000:00:1e.0/0000:05:05.0/local_cpus
Jan 29 16:43:24 vmhost kernel: [480258.280012] CPU 0
Jan 29 16:43:24 vmhost kernel: [480258.280170] Modules linked in: btrfs zlib_deflate crc32c libcrc32c ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat jfs xfs exportfs reiserfs ipt_MASQUERADE iptable_nat nf_na
t nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack bonding ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp kvm_intel kvm fbcon tileblit font bitblit softcursor vga16fb vgastate radeon
ttm drm_kms_helper drm i2c_algo_bit i3000_edac psmouse serio_raw ipmi_msghandler edac_core lp parport floppy e1000 e1000e 3w_9xxx
Jan 29 16:43:24 vmhost kernel: [480258.285521] Pid: 1670, comm: kvm Not tainted 2.6.32-27-server #49-Ubuntu
Jan 29 16:43:24 vmhost kernel: [480258.285780] RIP: 0010:[<ffffffffa021ffc0>] [<ffffffffa021ffc0>] rmap_remove+0x1b0/0x1c0 [kvm]
Jan 29 16:43:24 vmhost kernel: [480258.286210] RSP: 0018:ffff88020f191a68 EFLAGS: 00010292
Jan 29 16:43:24 vmhost kernel: [480258.286428] RAX: 000000000000004c RBX: ffff8801e927dfb0 RCX: 0000000000000000
Jan 29 16:43:24 vmhost kernel: [480258.286526] RDX: 0000000000000000 RSI: 0000000000000082 RDI: 0000000000000246
Jan 29 16:43:24 vmhost kernel: [480258.286526] RBP: ffff88020f191a88 R08: 00000000ffffffff R09: ffff8800000a18b0
Jan 29 16:43:24 vmhost kernel: [480258.286526] R10: ffff88020f191758 R11: ffff880214599800 R12: ffff88020e23c000
Jan 29 16:43:24 vmhost kernel: [480258.286526] R13: 00000000002035b7 R14: ffff88009e6ab9a0 R15: 00000000000001f6
Jan 29 16:43:24 vmhost kernel: [480258.286526] FS: 00007fce604fd700(0000) GS:ffff880009000000(0000) knlGS:0000000000000000
Jan 29 16:43:24 vmhost kernel: [480258.286526] CS: 0010 DS: 002b ES: 002b CR0: 0000000080050033
Jan 29 16:43:24 vmhost kernel: [480258.286526] CR2: 00007fc56c6ae0b0 CR3: 0000000213d03000 CR4: 00000000000026e0
Jan 29 16:43:24 vmhost kernel: [480258.286526] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jan 29 16:43:24 vmhost kernel: [480258.286526] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Jan 29 16:43:24 vmhost kernel: [480258.286526] Process kvm (pid: 1670, threadinfo ffff88020f190000, task ffff88020f345b80)
Jan 29 16:43:24 vmhost kernel: [480258.286526] Stack:
Jan 29 16:43:24 vmhost kernel: [480258.286526] ffff88009e6ab9a0 ffff88020e23c000 ffff8801e927dfb0 0000000000000000
Jan 29 16:43:24 vmhost kernel: [480258.286526] <0> ffff88020f191ac8 ffffffffa0220894 ffff88020f191ab8 ffff88020e23c000
Jan 29 16:43:24 vmhost kernel: [480258.286526] <0> ffff88009e6ab580 ffff88020e23f878 0000000000000001 00000000000e0000
Jan 29 16:43:24 vmhost kernel: [480258.286526] Call Trace:
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffffa0220894>] kvm_mmu_zap_page+0x84/0x380 [kvm]
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffffa0220dd8>] kvm_mmu_zap_all+0x38/0x70 [kvm]
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffffa0213576>] kvm_arch_flush_shadow+0x16/0x30 [kvm]
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffffa020a2c1>] __kvm_set_memory_region+0x6e1/0x770 [kvm]
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffffa0226109>] ? paging64_page_fault+0x1f9/0x210 [kvm]
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffffa021c094>] ? kvm_emulate_pio+0x174/0x1c0 [kvm]
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffffa020a393>] kvm_set_memory_region+0x43/0x70 [kvm]
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffffa020a3dd>] kvm_vm_ioctl_set_memory_region+0x1d/0x30 [kvm]
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffffa020bd42>] kvm_vm_ioctl+0x282/0x6e0 [kvm]
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffffa0266a8a>] ? vmx_handle_exit+0xca/0x250 [kvm_intel]
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffffa02621f2>] ? __vmx_load_host_state+0x1d2/0x240 [kvm_intel]
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffffa026226e>] ? vmx_vcpu_put+0xe/0x10 [kvm_intel]
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffffa020e9ab>] ? kvm_arch_vcpu_put+0x1b/0x50 [kvm]
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffffa020ac08>] ? vcpu_put+0x28/0x30 [kvm]
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffffa021c194>] ? kvm_arch_vcpu_ioctl_run+0xb4/0x1d0 [kvm]
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffff81059a06>] ? try_to_wake_up+0x2e6/0x470
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffff81059bc0>] ? wake_up_state+0x10/0x20
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffff81095010>] ? wake_futex+0x40/0x60
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffff810959c2>] ? futex_wake+0x112/0x130
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffff81153e12>] vfs_ioctl+0x22/0xa0
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffff81153fb1>] do_vfs_ioctl+0x81/0x410
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffff81097fab>] ? sys_futex+0x7b/0x170
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffff811543c1>] sys_ioctl+0x81/0xa0
Jan 29 16:43:24 vmhost kernel: [480258.286526] [<ffffffff810121b2>] system_call_fastpath+0x16/0x1b
Jan 29 16:43:24 vmhost kernel: [480258.286526] Code: 8b 13 48 89 de 48 c7 c7 70 5c 23 a0 31 c0 e8 ff 7c 33 e1 0f 0b eb fe 48 8b 13 48 89 de 48 c7 c7 50 5c 23 a0 31 c0 e8 e7 7c 33 e1 <0f> 0b eb fe 48 8b 17 48 89 10 eb a8 0f 1f 40 00 55 48 89 e5 41
Jan 29 16:43:24 vmhost kernel: [480258.286526] RIP [<ffffffffa021ffc0>] rmap_remove+0x1b0/0x1c0 [kvm]
Jan 29 16:43:24 vmhost kernel: [480258.286526] RSP <ffff88020f191a68>
Jan 29 16:43:24 vmhost kernel: [480258.816905] ---[ end trace 39ac185951650917 ]---
Jan 29 16:44:33 vmhost kernel: [480327.671241] BUG: soft lockup - CPU#1 stuck for 61s! [ksmd:53]
Jan 29 16:44:33 vmhost kernel: [480327.678895] Modules linked in: btrfs zlib_deflate crc32c libcrc32c ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat jfs xfs exportfs reiserfs ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack bonding ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp kvm_intel kvm fbcon tileblit font bitblit softcursor vga16fb vgastate radeon ttm drm_kms_helper drm i2c_algo_bit i3000_edac psmouse serio_raw ipmi_msghandler edac_core lp parport floppy e1000 e1000e 3w_9xxx
Jan 29 16:44:33 vmhost kernel: [480327.744839] CPU 1:
Jan 29 16:44:33 vmhost kernel: [480327.753302] Modules linked in: btrfs zlib_deflate crc32c libcrc32c ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat jfs xfs exportfs reiserfs ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack bonding ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp kvm_intel kvm fbcon tileblit font bitblit softcursor vga16fb vgastate radeon ttm drm_kms_helper drm i2c_algo_bit i3000_edac psmouse serio_raw ipmi_msghandler edac_core lp parport floppy e1000 e1000e 3w_9xxx
Jan 29 16:44:33 vmhost kernel: [480327.823706] Pid: 53, comm: ksmd Tainted: G D 2.6.32-27-server #49-Ubuntu
Jan 29 16:44:33 vmhost kernel: [480327.843983] RIP: 0010:[<ffffffff8103872e>] [<ffffffff8103872e>] __ticket_spin_lock+0xe/0x20
Jan 29 16:44:33 vmhost kernel: [480327.864312] RSP: 0018:ffff880216485c10 EFLAGS: 00000297
Jan 29 16:44:33 vmhost kernel: [480327.875527] RAX: 0000000000002b2a RBX: ffff880216485c10 RCX: 80000000d077b065
Jan 29 16:44:33 vmhost kernel: [480327.896984] RDX: 00007fce42dcb000 RSI: ffff88020f97ad80 RDI: ffff88020e23c000
Jan 29 16:44:33 vmhost kernel: [480327.919374] RBP: ffffffff81012cae R08: ffff88020f97b048 R09: 0000000000000040
Jan 29 16:44:33 vmhost kernel: [480327.940773] R10: ffff88020e0f70b0 R11: 0000000000000000 R12: 0000000000000000
Jan 29 16:44:33 vmhost kernel: [480327.962178] R13: ffff880000012080 R14: 0000000000015bc0 R15: 0000000000015bc0
Jan 29 16:44:33 vmhost kernel: [480327.984300] FS: 0000000000000000(0000) GS:ffff880009080000(0000) knlGS:0000000000000000
Jan 29 16:44:33 vmhost kernel: [480328.006146] CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
Jan 29 16:44:33 vmhost kernel: [480328.017186] CR2: 000000000236fd20 CR3: 0000000001001000 CR4: 00000000000026e0
Jan 29 16:44:33 vmhost kernel: [480328.040392] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Jan 29 16:44:33 vmhost kernel: [480328.063416] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Jan 29 16:44:33 vmhost kernel: [480328.086604] Call Trace:
Jan 29 16:44:33 vmhost kernel: [480328.098046] [<ffffffff8155a93e>] ? _spin_lock+0xe/0x20
Jan 29 16:44:33 vmhost kernel: [480328.109634] [<ffffffffa0208a8e>] ? kvm_mmu_notifier_change_pte+0x2e/0x60 [kvm]
Jan 29 16:44:33 vmhost kernel: [480328.131850] [<ffffffff8112e48e>] ? __mmu_notifier_change_pte+0x3e/0x90
Jan 29 16:44:33 vmhost kernel: [480328.143421] [<ffffffff8112f91b>] ? write_protect_page+0x15b/0x180
Jan 29 16:44:33 vmhost kernel: [480328.154502] [<ffffffff811300d6>] ? try_to_merge_one_page+0x96/0x120
Jan 29 16:44:33 vmhost kernel: [480328.165366] [<ffffffff811301f4>] ? try_to_merge_with_ksm_page+0x94/0xa0
Jan 29 16:44:33 vmhost kernel: [480328.176247] [<ffffffff811305c1>] ? try_to_merge_two_pages+0x181/0x1a0
Jan 29 16:44:33 vmhost kernel: [480328.186744] [<ffffffff8113071e>] ? cmp_and_merge_page+0x13e/0x1f0
Jan 29 16:44:33 vmhost kernel: [480328.197214] [<ffffffff81130800>] ? ksm_do_scan+0x30/0xb0
Jan 29 16:44:33 vmhost kernel: [480328.208074] [<ffffffff811308f2>] ? ksm_scan_thread+0x72/0x160
Jan 29 16:44:33 vmhost kernel: [480328.218351] [<ffffffff81084380>] ? autoremove_wake_function+0x0/0x40
Jan 29 16:44:33 vmhost kernel: [480328.228525] [<ffffffff81130880>] ? ksm_scan_thread+0x0/0x160
Jan 29 16:44:33 vmhost kernel: [480328.238402] [<ffffffff81084006>] ? kthread+0x96/0xa0
Jan 29 16:44:33 vmhost kernel: [480328.248160] [<ffffffff810131ea>] ? child_rip+0xa/0x20
Jan 29 16:44:33 vmhost kernel: [480328.257998] [<ffffffff81083f70>] ? kthread+0x0/0xa0
Jan 29 16:44:33 vmhost kernel: [480328.267262] [<ffffffff810131e0>] ? child_rip+0x0/0x20
Jan 29 16:45:01 vmhost CRON[32030]: (root) CMD (if [ -x /etc/munin/plugins/apt_all ]; then /etc/munin/plugins/apt_all update 7200 12 >/dev/null; elif [ -x /etc/munin/plugins/apt ]; then /etc/munin/plugins/apt update 7200 12 >/dev/null; fi)
Jan 29 16:45:39 vmhost kernel: [480393.171240] BUG: soft lockup - CPU#1 stuck for 61s! [ksmd:53]
Jan 29 16:45:39 vmhost kernel: [480393.180456] Modules linked in: btrfs zlib_deflate crc32c libcrc32c ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat jfs xfs exportfs reiserfs ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack bonding ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables bridge stp kvm_intel kvm fbcon tileblit font bitblit softcursor vga16fb vgastate radeon ttm drm_kms_helper drm i2c_algo_bit i3000_edac psmouse serio_raw ipmi_msghandler edac_core lp parport floppy e1000 e1000e 3w_9xxx
Jan 29 16:45:39 vmhost kernel: [480393.248579] CPU 1:


----------------------------------------------------------------
Postfach fast voll? Jetzt kostenlos E-Mail Adresse @t-online.de sichern und endlich Platz fÃr tausende Mails haben.
http://www.t-online.de/email-kostenlos


--
To unsubscribe from this list: send the line "unsubscribe linux-net" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html