Re: sched: hang in migrate_swap

[Peter Zijlstra]

On Wed, May 14, 2014 at 02:21:04PM +0400, Kirill Tkhai wrote:
> 
> 
> 14.05.2014, 14:14, "Peter Zijlstra" <peterz@xxxxxxxxxxxxx>:
> > On Wed, May 14, 2014 at 01:42:32PM +0400, Kirill Tkhai wrote:
> >
> >>  Peter, do we have to queue stop works orderly?
> >>
> >>  Is there is not a possibility, when two pair of works queued different on
> >>  different cpus?
> >>
> >>   kernel/stop_machine.c | 10 ++++++++--
> >>   1 file changed, 8 insertions(+), 2 deletions(-)
> >>  diff --git a/kernel/stop_machine.c b/kernel/stop_machine.c
> >>  index b6b67ec..29e221b 100644
> >>  --- a/kernel/stop_machine.c
> >>  +++ b/kernel/stop_machine.c
> >>  @@ -250,8 +250,14 @@ struct irq_cpu_stop_queue_work_info {
> >>   static void irq_cpu_stop_queue_work(void *arg)
> >>   {
> >>           struct irq_cpu_stop_queue_work_info *info = arg;
> >>  - cpu_stop_queue_work(info->cpu1, info->work1);
> >>  - cpu_stop_queue_work(info->cpu2, info->work2);
> >>  +
> >>  + if (info->cpu1 < info->cpu2) {
> >>  + cpu_stop_queue_work(info->cpu1, info->work1);
> >>  + cpu_stop_queue_work(info->cpu2, info->work2);
> >>  + } else {
> >>  + cpu_stop_queue_work(info->cpu2, info->work2);
> >>  + cpu_stop_queue_work(info->cpu1, info->work1);
> >>  + }
> >>   }
> >
> > I'm not sure, we already send the IPI to the first cpu of the pair, so
> > supposing we have 4 cpus, and get 4 pairs like:
> >
> > 0,1 1,2 2,3 3,0
> >
> > That would result in IPIs to 0, 1, 2, and 0 again, and since the IPI
> > function is serialized I don't immediately see a way for this to
> > deadlock.
> 
> It's about stop_two_cpus(), I have a distrust about other users of stop task:
> 
> queue_stop_cpus_work() queues work consequentially:
> 
> 0 1 2 4
> 
> stop_two_cpus() may queue:
> 
> 1 0
> 
> Looks like, stop thread on 0th and on 1th are waiting for wrong works.

so we serialize stop_cpus_work() vs stop_two_cpus() with an l/g lock.

Ah, but stop_cpus_work() only holds the global lock over queueing, it
doesn't wait for completion, that might indeed cause a problem.

Also, since its two different cpus queueing, the ordered queue doesn't
really matter, you can still interleave the all and two sets and get
into this state.

Attachment: pgpqyrp3L0J8v.pgp
Description: PGP signature