Re: [BUG] Linux 2.6.28.4 freezing on a 32-bits x86 Thinkpad T43p

From: Avi Kivity
Date: Wed Feb 11 2009 - 15:11:42 EST


Mathieu Desnoyers wrote:
* Ingo Molnar (mingo@xxxxxxx) wrote:
* Mathieu Desnoyers <mathieu.desnoyers@xxxxxxxxxx> wrote:

Hi,

I've started experiencing freezes on my uniprocessor laptop with a
2.6.28.2/2.6.28.3 kernel with the LTTng patchset applied
(http://git.kernel.org/?p=linux/kernel/git/compudj/linux-2.6-lttng.git;a=shortlog;h=2.6.28.3-lttng-0.88).
Instrumentation is dynamically disabled when this happens, so it's
unlikely that the LTTng patches would be causing this problem.

It happens when I work in X. The keyboard and mouse stop responding, and
the machine stops answering to the network. It may take a few days to
reproduce, and happens randomly when I actively use the computer (e.g
surfing with firefox).

I managed to install a 50' serial cable through my appartment to capture
the following OOPS. It points to a NULL pointer dereference in
kernel/timer.c:cascade(). My config has hrtimers and no_hz activated.
I suspect a race with with timer base lock or interrupt disabling
protecting the timer base.

Any idea what is going on with the timers here ? In the meantime, I'll
try to enable more debugging options to get more information when the
problem reappears.
hm, it would be nice to know which timer got corrupted. It could possibly have gotten kfreed, reallocated, overwritten - and crashes things like this.

There's two ways to debug such things more directly:

1) enable CONFIG_PAGEALLOC=y. These days its plenty fast and its overhead cannot be noticed.

2) enable DEBUGOBJECTS - you also need 'debugobjects' on the boot line for this to be activated. This will report such corruptions sooner and in a more specific way.

3) any particular reason why you have:

# CONFIG_DEBUG_KERNEL is not set

There's a number of goodies in that menu. CONFIG_LIST_DEBUG=y for example.

It is highly unlikely that the timer list code is the culprit here - it has not changed in ages and it is very intensively used by all subsystems so breakages in it get found and reported very, very quickly.

btw., your stacktrace also has this:

[<c1010000>] kvm_mmu_pte_write+0xb0/0xa60
So in theory there could be some kvm induced memory corruption as well.

Hope this helps,

Ingo

Hi Ingo,

Thanks for the hints.

Here is a new backtrace, taken with a huge amount of debugging active,
which still points to an interrupt handler nested over kvm_mmu_pte_write
as the culprit. It's weird that the kvm code gets called on my modest
Pentium M laptop, which I think has no VT-x support at all. I am not
running any KVM VMs on this machine. The problem still happens on
2.6.28.4, and Slub redzones did not identify any memory corruption. This
could be due to kvm_mmu_pte_write which either should not be called at
all, or due to improper interrupt disabling in this function.


I think kvm_mmu_pte_write is just random crap on the stack here. Your cpu definitely has no VT support so that code cannot be enabled at all.

Note the address is 64KB aligned which further suggests it isn't a real EIP.


--
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/