Re: [PATCH v3 0/3] qrwlock: Introducing a queue read/write lock implementation

[Waiman Long]

On 08/14/2013 11:57 AM, Ingo Molnar wrote:
> * Waiman Long<waiman.long@xxxxxx>  wrote:
>
> > On 08/14/2013 06:20 AM, Ingo Molnar wrote:
> > > * Waiman Long<waiman.long@xxxxxx>   wrote:
> > >
> > > > I would like to share with you a rwlock related system crash that I
> > > > encountered during my testing with hackbench on an 80-core DL980. The
> > > > kernel crash because of a "watchdog detected hard lockup on cpu 79". The
> > > > crashing CPU was running "write_lock_irq(&tasklist_lock)" in
> > > > forget_original_parent() of the exit code path when I interrupted the
> > > > hackbench which was spawning thousands of processes. Apparently, the
> > > > remote CPU was not able to get the lock for a sufficient long time due
> > > > to the unfairness of the rwlock which I think my version of queue rwlock
> > > > will be able to alleviate this issue.
> > > >
> > > > So far, I was not able to reproduce the crash. I will try to see if I
> > > > could more consistently reproduce it.
> > > Was it an actual crash/lockup, or a longish hang followed by a lock
> > > detector splat followed by the system eventually recovering back to
> > > working order?
> > >
> > > Thanks,
> > >
> > > 	Ingo
> > It was an actual crash initiated by the NMI handler. I think the
> > system was in a halt state after that.
> Could be a CONFIG_BOOTPARAM_HARDLOCKUP_PANIC_VALUE=1 kernel?
>
> Thanks,
>
> 	Ingo

My test system was a RHEL6.4 system. The 3.10 kernel config file was 
based on the original RHEL6.4 config file. So yes, the 
CONFIG_BOOTPARAM_HARDLOCKUP_PANIC_VALUE parameter was set.

I also found that when I bump the process count up to about 30k range, 
interrupting the main hack_bench process may not cause the other spawned 
process to die out. I will further investigate this phenomenon later 
next week.

Regards,
Longman
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/