Re: [PATCH 12/13] powerpc/perf: Thread imc cpuhotplug support

[Gautham R Shenoy]

Hi Maddy, Anju,

On Thu, Mar 16, 2017 at 01:05:06PM +0530, Madhavan Srinivasan wrote:
> From: Anju T Sudhakar <anju@xxxxxxxxxxxxxxxxxx>
> 
> This patch adds support for thread IMC on cpuhotplug.
> 
> When a cpu goes offline, the LDBAR for that cpu is disabled, and when it comes
> back online the previous ldbar value is written back to the LDBAR for that cpu.
> 
> To register the hotplug functions for thread_imc, a new state
> CPUHP_AP_PERF_POWERPC_THREADIMC_ONLINE is added to the list of existing
> states.
> 
> Cc: Gautham R. Shenoy <ego@xxxxxxxxxxxxxxxxxx>
> Cc: Balbir Singh <bsingharora@xxxxxxxxx>
> Cc: Benjamin Herrenschmidt <benh@xxxxxxxxxxxxxxxxxxx>
> Cc: Paul Mackerras <paulus@xxxxxxxxx>
> Cc: Anton Blanchard <anton@xxxxxxxxx>
> Cc: Sukadev Bhattiprolu <sukadev@xxxxxxxxxxxxxxxxxx>
> Cc: Michael Neuling <mikey@xxxxxxxxxxx>
> Cc: Stewart Smith <stewart@xxxxxxxxxxxxxxxxxx>
> Cc: Daniel Axtens <dja@xxxxxxxxxx>
> Cc: Stephane Eranian <eranian@xxxxxxxxxx>
> Signed-off-by: Anju T Sudhakar <anju@xxxxxxxxxxxxxxxxxx>
> Signed-off-by: Madhavan Srinivasan <maddy@xxxxxxxxxxxxxxxxxx>
> ---
>  arch/powerpc/perf/imc-pmu.c | 33 ++++++++++++++++++++++++++++-----
>  include/linux/cpuhotplug.h  |  1 +
>  2 files changed, 29 insertions(+), 5 deletions(-)
> 
> diff --git a/arch/powerpc/perf/imc-pmu.c b/arch/powerpc/perf/imc-pmu.c
> index 6802960db51c..2ff39fe2a5ce 100644
> --- a/arch/powerpc/perf/imc-pmu.c
> +++ b/arch/powerpc/perf/imc-pmu.c
> @@ -687,6 +687,16 @@ static void cleanup_all_thread_imc_memory(void)
>  	on_each_cpu(cleanup_thread_imc_memory, NULL, 1);
>  }
> 
> +static void thread_imc_update_ldbar(unsigned int cpu_id)
> +{
> +	u64 ldbar_addr, ldbar_value;
> +
> +	ldbar_addr = (u64)virt_to_phys((void *)per_cpu_add[cpu_id]);
> +	ldbar_value = (ldbar_addr & (u64)THREAD_IMC_LDBAR_MASK) |
> +			(u64)THREAD_IMC_ENABLE;
> +	mtspr(SPRN_LDBAR, ldbar_value);
> +}
> +
>  /*
>   * Allocates a page of memory for each of the online cpus, and, writes the
>   * physical base address of that page to the LDBAR for that cpu. This starts
> @@ -694,20 +704,33 @@ static void cleanup_all_thread_imc_memory(void)
>   */
>  static void thread_imc_mem_alloc(void *dummy)
>  {
> -	u64 ldbar_addr, ldbar_value;
>  	int cpu_id = smp_processor_id();
> 
>  	per_cpu_add[cpu_id] = (u64)__get_free_pages(GFP_KERNEL | __GFP_ZERO,
>  						    0);
> -	ldbar_addr = (u64)virt_to_phys((void *)per_cpu_add[cpu_id]);
> -	ldbar_value = (ldbar_addr & (u64)THREAD_IMC_LDBAR_MASK) |
> -		(u64)THREAD_IMC_ENABLE;
> -	mtspr(SPRN_LDBAR, ldbar_value);
> +	thread_imc_update_ldbar(cpu_id);
> +}
> +
> +static int ppc_thread_imc_cpu_online(unsigned int cpu)
> +{
> +	thread_imc_update_ldbar(cpu);
> +	return 0;
> +
> +}
> +
> +static int ppc_thread_imc_cpu_offline(unsigned int cpu)
> +{
> +	mtspr(SPRN_LDBAR, 0);
> +	return 0;
>  }

This patch looks ok to me.

So it appears that in case of a full-core deep stop entry/exit you
will need to save/restore LDBAR as well. But I will take it up for the
next set of stop cleanups.

For this patch,
Reviewed-by: Gautham R. Shenoy <ego@xxxxxxxxxxxxxxxxxx>

> 
>  void thread_imc_cpu_init(void)
>  {
>  	on_each_cpu(thread_imc_mem_alloc, NULL, 1);
> +	cpuhp_setup_state(CPUHP_AP_PERF_POWERPC_THREADIMC_ONLINE,
> +			  "POWER_THREAD_IMC_ONLINE",
> +			   ppc_thread_imc_cpu_online,
> +			   ppc_thread_imc_cpu_offline);
>  }
> 
>  static void thread_imc_ldbar_disable(void *dummy)
> diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
> index abde85d9511a..724df46b2c3c 100644
> --- a/include/linux/cpuhotplug.h
> +++ b/include/linux/cpuhotplug.h
> @@ -138,6 +138,7 @@ enum cpuhp_state {
>  	CPUHP_AP_PERF_ARM_L2X0_ONLINE,
>  	CPUHP_AP_PERF_POWERPC_NEST_ONLINE,
>  	CPUHP_AP_PERF_POWERPC_COREIMC_ONLINE,
> +	CPUHP_AP_PERF_POWERPC_THREADIMC_ONLINE,
>  	CPUHP_AP_PERF_ARM_QCOM_L2_ONLINE,
>  	CPUHP_AP_WORKQUEUE_ONLINE,
>  	CPUHP_AP_RCUTREE_ONLINE,
> -- 
> 2.7.4
> 
--
Thanks and Regards
gautham.