Re: Leap second insertion causes futex to repeatedly timeout

[John Stultz]

On 07/01/2012 01:36 AM, Ben Blum wrote:
> On Sun, Jul 01, 2012 at 01:16:13AM -0700, john stultz wrote:
> > On Sat, Jun 30, 2012 at 5:57 PM, Jan Engelhardt <jengelh@xxxxxxx> wrote:
> > > This year's leap second insertion has had the strange effect on at least
> > > Linux versions 3.4.4 (my end) and 3.5-rc4 (Simon's box, Cc) that certain
> > > processes use up all CPU power, because of futexes repeatedly timing
> > > out. This seems to only affect certain processes.
> > >
> > > Simon observes - http://s85.org/owXfmLvt - that
> > > Firefox/Thunderbird/Chrome/Java are affected.
> > >
> > > As for me, it affects VirtualBox, mysqld and ksoftirqd. The processes
> > > continue to run and respond. Most weird: I can stop-start mysqld and the
> > > issue persists. (I would have expected it to go away because the leap
> > > second event would then be in the past that mysqld does not know about
> > > anymore.)
> > >
> > >
> > > Is this a kernel issue? glibc?
> > Some of the reports that the issue is resolved by calling:
> >         $ date -s "`date`"
> > suggests that it might be due to clock_was_set() not being called
> > after the leap second was added, causing some hrtimer confusion.
> >
> > Thomas: does that sound about right?
> >
> > I've got an initial patch to add the clock_was_set() calls where
> > needed, but so far have not been able to reproduce the issue (tried
> > firefox and some simpler futex tests).  I'll keep trying and hopefully
> > have something to send out tomorrow.
> >
> > Again, my apologies for the trouble.
> I can't vouch for whether this is the problem or not, but be very
> careful with clock_was_set()! See this commit:
>
> http://www.mail-archive.com/git-commits-head@xxxxxxxxxxxxxxx/msg15039.html
>
> In short, clock_was_set() calls on_each_cpu() which is not allowed to be
> called in atomic context. Watch out for xtime_lock.

Quite right.  The fix is a little awkward due to the need to call it 
outside of holding xtime_lock/timekeeper.lock.

I've just reproduced the issue w/ Thunderbird, and my fix seems to avoid 
the issue. Working up a patch now.

thanks
-john


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/