Re: [PATCH V2] sched: topology: make cache topology separate from cpu topology

[Peter Zijlstra]

On Thu, Mar 10, 2022 at 07:25:33PM -0800, Qing Wang wrote:
> From: Wang Qing <wangqing@xxxxxxxx>
> 
> Some architectures(e.g. ARM64), caches are implemented like below:
> SD(Level 1):          ************ DIE ************
> SD(Level 0):          **** MC ****    **** MC *****
> cluster:              **cluster 0**   **cluster 1**
> cores:                0   1   2   3   4   5   6   7
> cache(Level 1):       C   C   C   C   C   C   C   C
> cache(Level 2):  	  **C**   **C**   **C**   **C**
> cache(Level 3):       *******shared Level 3********
> sd_llc_id(current):   0   0   0   0   4   4   4   4
> sd_llc_id(should be): 0   0   2   2   4   4   6   6
> 
> Caches and cpus have different topology, this causes cpus_share_cache()
> return the wrong value in sd, which will affect the CPU load balance.

Then fix your SD_flags already.

> diff --git a/include/linux/arch_topology.h b/include/linux/arch_topology.h
> index cce6136b..3048fa6
> --- a/include/linux/arch_topology.h
> +++ b/include/linux/arch_topology.h
> @@ -82,6 +82,8 @@ extern struct cpu_topology cpu_topology[NR_CPUS];
>  #define topology_cluster_cpumask(cpu)	(&cpu_topology[cpu].cluster_sibling)
>  #define topology_llc_cpumask(cpu)	(&cpu_topology[cpu].llc_sibling)
>  void init_cpu_topology(void);
> +void init_cpu_cache_topology(void);
> +void fix_cpu_llc(int cpu, int *first_cpu, int *cpu_num);
>  void store_cpu_topology(unsigned int cpuid);
>  const struct cpumask *cpu_coregroup_mask(int cpu);
>  const struct cpumask *cpu_clustergroup_mask(int cpu);
> diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
> index d201a70..d894ced
> --- a/kernel/sched/topology.c
> +++ b/kernel/sched/topology.c
> @@ -661,6 +661,9 @@ static void update_top_cache_domain(int cpu)
>  	if (sd) {
>  		id = cpumask_first(sched_domain_span(sd));
>  		size = cpumask_weight(sched_domain_span(sd));
> +#ifdef CONFIG_GENERIC_ARCH_TOPOLOGY
> +		fix_cpu_llc(cpu, &id, &size);
> +#endif
>  		sds = sd->shared;
>  	}

NAK on that.