Re: [PATCH] ARM: CPU hotplug: Delegate complete() to surviving CPU

[Baruch Siach]

Hi Paul,

On Tue, Dec 12, 2017 at 09:20:59AM -0800, Paul E. McKenney wrote:
> The ARM implementation of arch_cpu_idle_dead() invokes complete(), but
> does so after RCU has stopped watching the outgoing CPU, which results
> in lockdep complaints because complete() invokes functions containing RCU
> readers.  This patch therefore uses Thomas Gleixner's trick of delegating
> the complete() call to a surviving CPU via smp_call_function_single().
> 
> Reported-by: Peng Fan <van.freenix@xxxxxxxxx>
> Reported-by: Russell King - ARM Linux <linux@xxxxxxxxxxxxxxx>
> Signed-off-by: Paul E. McKenney <paulmck@xxxxxxxxxxxxxxxxxx>
> Tested-by: Tested-by: Fabio Estevam <fabio.estevam@xxxxxxx>

Fabio reported only once, though he might have tested twice.

baruch

> Cc: Russell King <linux@xxxxxxxxxxxxxxx>
> Cc: Ingo Molnar <mingo@xxxxxxxxxx>
> Cc: "Peter Zijlstra (Intel)" <peterz@xxxxxxxxxxxxx>
> Cc: Michal Hocko <mhocko@xxxxxxxx>
> Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
> Cc: <linux-arm-kernel@xxxxxxxxxxxxxxxxxxx>
> 
> diff --git a/arch/arm/kernel/smp.c b/arch/arm/kernel/smp.c
> index b4fbf00ee4ad..75f85e20aafa 100644
> --- a/arch/arm/kernel/smp.c
> +++ b/arch/arm/kernel/smp.c
> @@ -267,6 +267,14 @@ void __cpu_die(unsigned int cpu)
>  }
>  
>  /*
> + * Invoke complete() on behalf of the outgoing CPU.
> + */
> +static void arch_cpu_idle_dead_complete(void *arg)
> +{
> +	complete(&cpu_died);
> +}
> +
> +/*
>   * Called from the idle thread for the CPU which has been shutdown.
>   *
>   * Note that we disable IRQs here, but do not re-enable them
> @@ -293,9 +301,11 @@ void arch_cpu_idle_dead(void)
>  	/*
>  	 * Tell __cpu_die() that this CPU is now safe to dispose of.  Once
>  	 * this returns, power and/or clocks can be removed at any point
> -	 * from this CPU and its cache by platform_cpu_kill().
> +	 * from this CPU and its cache by platform_cpu_kill().  We cannot
> +	 * call complete() this late, so we delegate it to an online CPU.
>  	 */
> -	complete(&cpu_died);
> +	smp_call_function_single(cpumask_first(cpu_online_mask),
> +				 arch_cpu_idle_dead_complete, NULL, 0);
>  
>  	/*
>  	 * Ensure that the cache lines associated with that completion are

-- 
     http://baruch.siach.name/blog/                  ~. .~   Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
   - baruch@xxxxxxxxxx - tel: +972.52.368.4656, http://www.tkos.co.il -