RE: [PATCH V2] smp: Give WARN()ing when callingsmp_call_function_many()/single() in serving irq

[Liu, Chuansheng]

> -----Original Message-----
> From: Lai Jiangshan [mailto:eag0628@xxxxxxxxx]
> Sent: Wednesday, February 27, 2013 10:51 PM
> To: Liu, Chuansheng
> Cc: mingo@xxxxxxxxxx; peterz@xxxxxxxxxxxxx; jbeulich@xxxxxxxx;
> paulmck@xxxxxxxxxxxxxxxxxx; akpm@xxxxxxxxxxxxxxxxxxxx;
> mina86@xxxxxxxxxx; srivatsa.bhat@xxxxxxxxxxxxxxxxxx;
> linux-kernel@xxxxxxxxxxxxxxx; Zhang, Jun; Wu, Fengguang
> Subject: Re: [PATCH V2] smp: Give WARN()ing when calling
> smp_call_function_many()/single() in serving irq
> 
> On Sat, Feb 16, 2013 at 10:10 PM, Chuansheng Liu
> <chuansheng.liu@xxxxxxxxx> wrote:
> > Currently the functions smp_call_function_many()/single() will
> > give a WARN()ing only in the case of irqs_disabled(), but that
> > check is not enough to guarantee execution of the SMP
> > cross-calls.
> >
> > In many other cases such as softirq handling/interrupt handling,
> > the two APIs still can not be called, just as the
> > smp_call_function_many() comments say:
> >
> >   * You must not call this function with disabled interrupts or from a
> >   * hardware interrupt handler or from a bottom half handler. Preemption
> >   * must be disabled when calling this function.
> >
> > There is a real case for softirq DEADLOCK case:
> >
> > CPUA                            CPUB
> >                                 spin_lock(&spinlock)
> >                                 Any irq coming, call the irq handler
> >                                 irq_exit()
> > spin_lock_irq(&spinlock)
> > <== Blocking here due to
> > CPUB hold it
> >                                   __do_softirq()
> >                                     run_timer_softirq()
> >                                       timer_cb()
> >                                         call
> smp_call_function_many()
> >                                           send IPI interrupt to
> CPUA
> >                                             wait_csd()
> >
> > Then both CPUA and CPUB will be deadlocked here.
> >
> > So we should give a warning in the nmi, hardirq or softirq context as well.
> >
> > Moreover, adding one new macro in_serving_irq() which indicates
> > we are processing nmi, hardirq or sofirq.
> 
> The code smells bad. in_serving_softirq() don't take spin_lock_bh() in account.
> 
> CPUA                    CPUB                             CPUC
>                         spin_lock(&lockA)
>                           Any irq coming, call
>                           the irq handler
>                           irq_exit()
> spin_lock_irq(&lockA)
> *Blocking* here
> due to CPUB hold it
> spin_lock_bh(&lockB)
>                             __do_softirq()
>                               run_timer_softirq()
>                                 spin_lock_bh(&lockB)
>                                 *Blocking* heredue to
>                                 CPUC hold it
>                                                          call
> smp_call_function_many()
>                                                          send IPI
> interrupt to CPUA
> 
> wait_csd()
> 
> *Blocking* here.
> 
> So it is still deadlock. but your code does not warn it.
In your case, even you change spin_lock_bh() to spin_lock(), the deadlock is still there. So no relation with _bh() at all,
Do not need warning for such deadlock case in smp_call_xxx() or for _bh() case.

> so in_softirq() is better than in_serving_softirq() in in_serving_irq(),
> and results in_serving_irq() is the same as in_interrupt().
> 
> so please remove in_serving_irq() and use in_interrupt() instead.
The original patch is using in_interrupt(). https://lkml.org/lkml/2013/2/6/34 

> And add:
> 
> Reviewed-by: Lai Jiangshan <laijs@xxxxxxxxxxxxxx>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/