Re: [PATCH V4 05/12] trace/hwlat: Support hotplug operations

[Steven Rostedt]

On Tue, 15 Jun 2021 11:28:44 +0200
Daniel Bristot de Oliveira <bristot@xxxxxxxxxx> wrote:

> Enable and disable hwlat thread during cpu hotplug online
> and offline operations, respectivelly.
> 
> Cc: Jonathan Corbet <corbet@xxxxxxx>
> Cc: Steven Rostedt <rostedt@xxxxxxxxxxx>
> Cc: Ingo Molnar <mingo@xxxxxxxxxx>
> Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
> Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
> Cc: Alexandre Chartre <alexandre.chartre@xxxxxxxxxx>
> Cc: Clark Willaims <williams@xxxxxxxxxx>
> Cc: John Kacur <jkacur@xxxxxxxxxx>
> Cc: Juri Lelli <juri.lelli@xxxxxxxxxx>
> Cc: Borislav Petkov <bp@xxxxxxxxx>
> Cc: "H. Peter Anvin" <hpa@xxxxxxxxx>
> Cc: x86@xxxxxxxxxx
> Cc: linux-doc@xxxxxxxxxxxxxxx
> Cc: linux-kernel@xxxxxxxxxxxxxxx
> Signed-off-by: Daniel Bristot de Oliveira <bristot@xxxxxxxxxx>
> ---
>  kernel/trace/trace.c       |  6 ++++++
>  kernel/trace/trace_hwlat.c | 34 ++++++++++++++++++++++++++++++++++
>  2 files changed, 40 insertions(+)
> 
> diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
> index 9299057feb56..c094865e2f71 100644
> --- a/kernel/trace/trace.c
> +++ b/kernel/trace/trace.c
> @@ -5064,7 +5064,13 @@ int tracing_set_cpumask(struct trace_array *tr,
>  	arch_spin_unlock(&tr->max_lock);
>  	local_irq_enable();
>  
> +	/*
> +	 * tracing_cpumask is read by tracers that support CPU
> +	 * hotplug.
> +	 */
> +	get_online_cpus();
>  	cpumask_copy(tr->tracing_cpumask, tracing_cpumask_new);
> +	put_online_cpus();
>  
>  	return 0;
>  }
> diff --git a/kernel/trace/trace_hwlat.c b/kernel/trace/trace_hwlat.c
> index 6c6918e86087..9fcfd588c4f6 100644
> --- a/kernel/trace/trace_hwlat.c
> +++ b/kernel/trace/trace_hwlat.c
> @@ -490,6 +490,35 @@ static int start_cpu_kthread(unsigned int cpu)
>  	return 0;
>  }
>  
> +/*
> + * hwlat_cpu_init - CPU hotplug online callback function
> + */
> +static int hwlat_cpu_init(unsigned int cpu)
> +{
> +	struct trace_array *tr = hwlat_trace;
> +

You need to take the trace_types_lock here, between testing the
hwlat_busy and starting the threads. Otherwise, between the two, the
hwlat tracer could be turned off while a CPU is coming on line, and
then you just started a per cpu thread, while the hwlat tracer is not
enabled.

> +	if (!hwlat_busy)
> +		return 0;
> +
> +	if (!cpumask_test_cpu(cpu, tr->tracing_cpumask))
> +		return 0;
> +
> +	return start_cpu_kthread(cpu);
> +}
> +
> +/*
> + * hwlat_cpu_die - CPU hotplug offline callback function
> + */
> +static int hwlat_cpu_die(unsigned int cpu)
> +{

Same here.

-- Steve


> +	if (!hwlat_busy)
> +		return 0;
> +
> +	stop_cpu_kthread(cpu);
> +
> +	return 0;
> +}
> +
>  /*
>   * start_per_cpu_kthreads - Kick off the hardware latency sampling/detector kthreads
>   *
> @@ -903,6 +932,11 @@ __init static int init_hwlat_tracer(void)
>  	if (ret)
>  		return ret;
>  
> +	ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "trace/hwlat:online",
> +				hwlat_cpu_init, hwlat_cpu_die);
> +	if (ret < 0)
> +		pr_warn(BANNER "Error to init cpu hotplug support\n");
> +
>  	init_tracefs();
>  
>  	return 0;