Re: [PATCH V4 1/2] powerpc/numa: Update CPU topology when VPHN enabled

[Nathan Fontenot]

On 05/26/2017 04:29 PM, Michael Bringmann wrote:
> 
> powerpc/numa: Correct the currently broken capability to set the
> topology for shared CPUs in LPARs.  At boot time for shared CPU
> lpars, the topology for each shared CPU is set to node zero, however,
> this is now updated correctly using the Virtual Processor Home Node
> (VPHN) capabilities information provided by the pHyp. The VPHN handling
> in Linux is disabled, if PRRN handling is present.

I'm still not sure this is what we want. Looking at the topology updating
code, we only enable VPHN if PRRN is not present.

My understanding of the current situation is that the node for partitions
with shared cpus are not set. The reason for this is that the device tree
presented to a partition using shared cpus at boot puts all cpus in node
zero and then uses the VPHN capability to inform the partition which node
each cpu really belongs to.

Additionally, I think this is how DLPAR of shared cpu partitions work.
After the cpu is DLPAR added we should get a VPHN notification to inform
us of the true node that the cpu belongs to.

When the PRRN capability was introduced it was thought to be a follow-on
to the VPHN capability and so the code to start topology updating only
enables VPHN if PRRN is not present. I think what we need to do is always
enable VPHN for shared cpu partitions.

-Nathan

> 
> Signed-off-by: Michael Bringmann <mwb@xxxxxxxxxxxxxxxxxx>
> ---
> Changes in V4:
>  -- Fix conditional compile bug.
> ---
>  arch/powerpc/mm/numa.c                 |   19 ++++++++++++++++++-
>  arch/powerpc/platforms/pseries/dlpar.c |    2 ++
>  2 files changed, 20 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c
> index 371792e..afcee3f 100644
> --- a/arch/powerpc/mm/numa.c
> +++ b/arch/powerpc/mm/numa.c
> @@ -29,6 +29,7 @@
>  #include <linux/seq_file.h>
>  #include <linux/uaccess.h>
>  #include <linux/slab.h>
> +#include <linux/sched.h>
>  #include <asm/cputhreads.h>
>  #include <asm/sparsemem.h>
>  #include <asm/prom.h>
> @@ -1153,6 +1154,8 @@ struct topology_update_data {
>  static int vphn_enabled;
>  static int prrn_enabled;
>  static void reset_topology_timer(void);
> +static int topology_inited;
> +static int topology_update_needed;
> 
>  /*
>   * Store the current values of the associativity change counters in the
> @@ -1321,8 +1324,11 @@ int arch_update_cpu_topology(void)
>  	struct device *dev;
>  	int weight, new_nid, i = 0;
> 
> -	if (!prrn_enabled && !vphn_enabled)
> +	if (!prrn_enabled && !vphn_enabled) {
> +		if (!topology_inited)
> +			topology_update_needed = 1;
>  		return 0;
> +	}
> 
>  	weight = cpumask_weight(&cpu_associativity_changes_mask);
>  	if (!weight)
> @@ -1361,6 +1367,8 @@ int arch_update_cpu_topology(void)
>  			cpumask_andnot(&cpu_associativity_changes_mask,
>  					&cpu_associativity_changes_mask,
>  					cpu_sibling_mask(cpu));
> +			pr_info("Assoc chg gives same node %d for cpu%d\n",
> +					new_nid, cpu);
>  			cpu = cpu_last_thread_sibling(cpu);
>  			continue;
>  		}
> @@ -1377,6 +1385,9 @@ int arch_update_cpu_topology(void)
>  		cpu = cpu_last_thread_sibling(cpu);
>  	}
> 
> +	if (i)
> +		updates[i-1].next = NULL;
> +
>  	pr_debug("Topology update for the following CPUs:\n");
>  	if (cpumask_weight(&updated_cpus)) {
>  		for (ud = &updates[0]; ud; ud = ud->next) {
> @@ -1423,6 +1434,7 @@ int arch_update_cpu_topology(void)
> 
>  out:
>  	kfree(updates);
> +	topology_update_needed = 0;
>  	return changed;
>  }
> 
> @@ -1600,6 +1612,11 @@ static int topology_update_init(void)
>  	if (!proc_create("powerpc/topology_updates", 0644, NULL, &topology_ops))
>  		return -ENOMEM;
> 
> +	topology_inited = 1;
> +	if (topology_update_needed)
> +		bitmap_fill(cpumask_bits(&cpu_associativity_changes_mask),
> +					nr_cpumask_bits);
> +
>  	return 0;
>  }
>  device_initcall(topology_update_init);
> diff --git a/arch/powerpc/platforms/pseries/dlpar.c b/arch/powerpc/platforms/pseries/dlpar.c
> index bda18d8..5106263 100644
> --- a/arch/powerpc/platforms/pseries/dlpar.c
> +++ b/arch/powerpc/platforms/pseries/dlpar.c
> @@ -592,6 +592,8 @@ static ssize_t dlpar_show(struct class *class, struct class_attribute *attr,
> 
>  static int __init pseries_dlpar_init(void)
>  {
> +	arch_update_cpu_topology();
> +
>  	pseries_hp_wq = alloc_workqueue("pseries hotplug workqueue",
>  					WQ_UNBOUND, 1);
>  	return sysfs_create_file(kernel_kobj, &class_attr_dlpar.attr);
>