Re: [PATCH 3/3] x86: Avoid calling arch_trigger_all_cpu_backtrace()at the same time

[DDD]

Frederic Weisbecker wrote:
> On Fri, Nov 12, 2010 at 09:50:55AM -0500, Don Zickus wrote:
> > From: Dongdong Deng <dongdong.deng@xxxxxxxxxxxxx>
> >
> > The spin_lock_debug/rcu_cpu_stall detector uses
> > trigger_all_cpu_backtrace() to dump cpu backtrace.
> > Therefore it is possible that trigger_all_cpu_backtrace()
> > could be called at the same time on different CPUs, which
> > triggers and 'unknown reason NMI' warning. The following case
> > illustrates the problem:
> >
> >       CPU1                    CPU2                     ...   CPU N
> >                        trigger_all_cpu_backtrace()
> >                        set "backtrace_mask" to cpu mask
> >                                |
> > generate NMI interrupts  generate NMI interrupts       ...
> >     \                          |                               /
> >      \                         |                              /
> >
> > The "backtrace_mask" will be cleaned by the first NMI interrupt
> > at nmi_watchdog_tick(), then the following NMI interrupts generated
> > by other cpus's arch_trigger_all_cpu_backtrace() will be taken as
> > unknown reason NMI interrupts.
> >
> > This patch uses a test_and_set to avoid the problem, and stop the
> > arch_trigger_all_cpu_backtrace() from calling to avoid dumping a
> > double cpu backtrace info when there is already a
> > trigger_all_cpu_backtrace() in progress.
> >
> > Signed-off-by: Dongdong Deng <dongdong.deng@xxxxxxxxxxxxx>
> > Reviewed-by: Bruce Ashfield <bruce.ashfield@xxxxxxxxxxxxx>
> > CC: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
> > CC: Ingo Molnar <mingo@xxxxxxxxxx>
> > CC: "H. Peter Anvin" <hpa@xxxxxxxxx>
> > CC: x86@xxxxxxxxxx
> > CC: linux-kernel@xxxxxxxxxxxxxxx
> > Signed-off-by: Don Zickus <dzickus@xxxxxxxxxx>
> > ---
> >  arch/x86/kernel/apic/hw_nmi.c |   24 ++++++++++++++++++++++++
> >  1 files changed, 24 insertions(+), 0 deletions(-)
> >
> > diff --git a/arch/x86/kernel/apic/hw_nmi.c b/arch/x86/kernel/apic/hw_nmi.c
> > index f349647..d892896 100644
> > --- a/arch/x86/kernel/apic/hw_nmi.c
> > +++ b/arch/x86/kernel/apic/hw_nmi.c
> > @@ -27,9 +27,27 @@ u64 hw_nmi_get_sample_period(void)
> >  /* For reliability, we're prepared to waste bits here. */
> >  static DECLARE_BITMAP(backtrace_mask, NR_CPUS) __read_mostly;
> >  
> > +/* "in progress" flag of arch_trigger_all_cpu_backtrace */
> > +static unsigned long backtrace_flag;
> > +
> >  void arch_trigger_all_cpu_backtrace(void)
> >  {
> >  	int i;
> > +	unsigned long flags;
> > +
> > +	/*
> > +	 * Have to disable irq here, as the
> > +	 * arch_trigger_all_cpu_backtrace() could be
> > +	 * triggered by "spin_lock()" with irqs on.
> > +	 */
> > +	local_irq_save(flags);
>
>
>
> I'm not sure I understand why you disable irqs here. It looks
> safe with the test_and_set_bit already.

Hi Frederic,

Yep, after we use test_and_set_bit to replace spin_lock,
the disable irqs ops obvious could be removed here.

I will redo this patch, and send it to Don.

Thanks,
Dongdong

> > +
> > +	if (test_and_set_bit(0, &backtrace_flag))
> > +		/*
> > +		 * If there is already a trigger_all_cpu_backtrace() in progress
> > +		 * (backtrace_flag == 1), don't output double cpu dump infos.
> > +		 */
> > +		goto out_restore_irq;
> >  
> >  	cpumask_copy(to_cpumask(backtrace_mask), cpu_online_mask);
> >  
> > @@ -42,6 +60,12 @@ void arch_trigger_all_cpu_backtrace(void)
> >  			break;
> >  		mdelay(1);
> >  	}
> > +
> > +	clear_bit(0, &backtrace_flag);
> > +	smp_mb__after_clear_bit();
> > +
> > +out_restore_irq:
> > +	local_irq_restore(flags);
> >  }
> >  
> >  static int __kprobes
> > --
> > 1.7.2.3
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > the body of a message to majordomo@xxxxxxxxxxxxxxx
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at  http://www.tux.org/lkml/

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/