Re: frequent lockups in 3.18rc4

From: Dave Jones
Date: Wed Dec 03 2014 - 14:48:41 EST


On Wed, Dec 03, 2014 at 11:25:29AM -0800, Linus Torvalds wrote:
> On Wed, Dec 3, 2014 at 11:00 AM, Dave Jones <davej@xxxxxxxxxx> wrote:
> >
> > So right after sending my last mail, I rebooted, and restarted the run
> > on the same kernel again.
> >
> > As I was writing this mail, this happened.
> >
> > [ 524.420897] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [trinity-c178:20182]
> >
> > and that's all that made it over the console. I couldn't log in via ssh,
> > and thought "ah-ha, so it IS bad". I walked over to reboot it, and
> > found I could actually log in on the console. check out this dmesg..
> >
> > [ 503.683055] Clocksource tsc unstable (delta = -95946009388 ns)
> > [ 503.692038] Switched to clocksource hpet
> > [ 524.420897] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [trinity-c178:20182]
>
> Interesting. That whole NMI watchdog thing happens pretty much 22s
> after the "TSC unstable" message.
>
> Have you ever seen that TSC issue before? The watchdog relies on
> comparing get_timestamp() differences, so if the timestamp was
> incorrect...

yeah, quite a lot.

# grep tsc\ unstable /var/log/messages* | wc -l
71

Usually happens pretty soon after boot, once I start the fuzzing run.
It sometimes occurs quite some time before the NMI issue though.

eg:

Dec 3 11:50:24 binary kernel: [ 4253.432642] Clocksource tsc unstable (delta = -243666538341 ns)
...
Dec 3 13:24:28 binary kernel: [ 9862.915562] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [trinity-c29:13237]


Dave
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/