Re: New crashes walking proc with Saturday's git

[Chris Mason]

On Sun, Nov 23, 2014 at 4:11 PM, Chris Mason <clm@xxxxxx> wrote:
> On Sun, Nov 23, 2014 at 4:05 PM, Thomas Gleixner <tglx@xxxxxxxxxxxxx> 
> wrote:
> > On Sun, 23 Nov 2014, Chris Mason wrote:
> > >  On Sun, Nov 23, 2014 at 11:32 AM, Borislav Petkov <bp@xxxxxxxxx> 
> > > wrote:
> > >  > On Sun, Nov 23, 2014 at 11:16:51AM -0500, Chris Mason wrote:
> > >  > >  It must be:
> > >  > >
> > >  > >  commit 6e998916dfe327e785e7c2447959b2c1a3ea4930
> > >  > >  Author: Stanislaw Gruszka <sgruszka@xxxxxxxxxx>
> > >  > >  Date:   Wed Nov 12 16:58:44 2014 +0100
> > >  > >
> > >  > >     sched/cputime: Fix clock_nanosleep()/clock_gettime() 
> > > inconsistency
> > >  > >
> > >  > >  I'll do two runs to confirm, but it's the only related patch 
> > > between rc5
> > >  > > and
> > >  > >  now.
> > >
> > >  I've adding Ingo and Stanislaw to the cc.  With
> > >  6e998916dfe327e785e7c2447959b2c1a3ea4930 reverted, I'm no longer 
> > > crashing.
> > >
> > >  Repeating the stack trace for the new cc list.  I see the crash 
> > > with atop or
> > >  similar walkers of /proc racing against exiting programs.  Given 
> > > the NULL rip,
> > >  this line from the patch is probably broken, but it really feels 
> > > like we
> > >  should be falling over on p->sched_class and not on the 
> > > update_curr func.
> > >
> > >  +               p->sched_class->update_curr(rq);
> > >
> > >  I'm leaving my fork bomb running on two machines with the patch 
> > > reverted to
> > >  make sure.
> >
> > The sched_class instances which do not have update_curr are stop_task
> > and idle. Patch below.
> >
> > I'm sure nobody thought about the stats read code path here.
> >
> > [ 1053.759741]  [<ffffffff81208348>] do_task_stat+0x8b8/0xb00
> >
> > do_task_stat(()
> >  thread_group_cputime_adjusted()
> >    thread_group_cputime()
> >      task_cputime()
> >        task_sched_runtime()
> > 	if (task_current(rq, p) && task_on_rq_queued(p)) {
> >                 update_rq_clock(rq);
> >                 p->sched_class->update_curr(rq);
> >         }
> >
> > Now if the stats are read for a stomp machine task, aka 'migration/N'
> > and that task is current on its cpu. Ooops.
> >
> > I added the callback for idle tasks as well for completeness sake.
>
> This does make sense, but it doesn't match with the crash being much 
> more likely during the fork bomb.  The difference is crashing within 
> a few hours vs crashing within 5 minutes.
>
> But, maybe I just got lucky.  I'll try the patch.

11 minutes later and it's still alive.  I'll keep an eye on it and yell 
if it falls over.

-chris



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/