Re: [PATCH v2] arch/sparc: support NR_CPUS = 4096

[David Miller]

From: Jane Chu <jane.chu@xxxxxxxxxx>
Date: Mon,  5 Jun 2017 16:48:31 -0600

> Linux SPARC64 limits NR_CPUS to 4064 because init_cpu_send_mondo_info()
> only allocates a single page for NR_CPUS mondo entries. Thus we cannot
> use all 4096 CPUs on some SPARC platforms.
> 
> To fix, allocate (2^order) pages where order is set according to the size
> of cpu_list for possible cpus. Since cpu_list_pa and cpu_mondo_block_pa
> are not used in asm code, there are no imm13 offsets from the base PA
> that will break because they can only reach one page.
> 
> Orabug: 25505750
> 
> Signed-off-by: Jane Chu <jane.chu@xxxxxxxxxx>
> 
> Reviewed-by: Bob Picco <bob.picco@xxxxxxxxxx>
> Reviewed-by: Atish Patra <atish.patra@xxxxxxxxxx>
> ---
>  arch/sparc/Kconfig         |    4 ++--
>  arch/sparc/kernel/irq_64.c |   16 ++++++++++++----
>  2 files changed, 14 insertions(+), 6 deletions(-)
> 
> diff --git a/arch/sparc/Kconfig b/arch/sparc/Kconfig
> index 58243b0..4399be7 100644
> --- a/arch/sparc/Kconfig
> +++ b/arch/sparc/Kconfig
> @@ -192,9 +192,9 @@ config NR_CPUS
>  	int "Maximum number of CPUs"
>  	depends on SMP
>  	range 2 32 if SPARC32
> -	range 2 1024 if SPARC64
> +	range 2 4096 if SPARC64
>  	default 32 if SPARC32
> -	default 64 if SPARC64
> +	default 4096 if SPARC64
>  
>  source kernel/Kconfig.hz
>  
> diff --git a/arch/sparc/kernel/irq_64.c b/arch/sparc/kernel/irq_64.c
> index 4d0248a..cc8f6c3 100644
> --- a/arch/sparc/kernel/irq_64.c
> +++ b/arch/sparc/kernel/irq_64.c
> @@ -1034,17 +1034,25 @@ static void __init init_cpu_send_mondo_info(struct trap_per_cpu *tb)
>  {
>  #ifdef CONFIG_SMP
>  	unsigned long page;
> +	void *mondo;
>  
> -	BUILD_BUG_ON((NR_CPUS * sizeof(u16)) > (PAGE_SIZE - 64));
> +	BUILD_BUG_ON((NR_CPUS * sizeof(u16)) > PAGE_SIZE);
> +
> +	/* Make sure mondo block is 64byte aligned */
> +	mondo = kzalloc(64, GFP_KERNEL);
> +	if (!mondo) {
> +		prom_printf("SUN4V: Error, cannot allocate mondo block.\n");
> +		prom_halt();
> +	}
> +	tb->cpu_mondo_block_pa = __pa(mondo);

Hmmm, you said that this has to be 64 byte aligned right?  We might have
to do something in order to insure that, as kmalloc() only guarantees
ARCH_KMALLOC_MINALIGN which I think is 8 on sparc.

I suppose this would work:

	mondo = kzalloc(64 + 63, GFP_KERNEL);

and then 64-byte align that pointer.