Re: [PATCH] x86: Allow NR_CPUS=1024

[Josh Boyer]

On Mon, Nov 04, 2013 at 07:53:43AM +0100, Ingo Molnar wrote:
> 
> * Josh Boyer <jwboyer@xxxxxxxxxx> wrote:
> 
> > On Sun, Nov 03, 2013 at 11:21:32AM +0100, Ingo Molnar wrote:
> > > 
> > > * Ingo Molnar <mingo@xxxxxxxxxx> wrote:
> > > 
> > > > 
> > > > * Josh Boyer <jwboyer@xxxxxxxxxx> wrote:
> > > > 
> > > > > The current range for SMP configs is 2 - 512, or a full 4096 in the case 
> > > > > of MAXSMP.  There are machines that have 1024 CPUs in them today and 
> > > > > configuring a kernel for that means you are forced to set MAXSMP.  This 
> > > > > adds additional unnecessary overhead.  While that overhead might be 
> > > > > considered tiny for large machines, it isn't necessarily so if you are 
> > > > > building a kernel that runs across a wide variety of machines.  We 
> > > > > increase the range to 1024 to help with this.
> > > > >
> > > > > Signed-off-by: Josh Boyer <jwboyer@xxxxxxxxxxxxxxxxx>
> > > > > ---
> > > > >  arch/x86/Kconfig | 2 +-
> > > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > > > 
> > > > > diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
> > > > > index f67e839..d726b2d 100644
> > > > > --- a/arch/x86/Kconfig
> > > > > +++ b/arch/x86/Kconfig
> > > > > @@ -825,7 +825,7 @@ config MAXSMP
> > > > >  config NR_CPUS
> > > > >  	int "Maximum number of CPUs" if SMP && !MAXSMP
> > > > >  	range 2 8 if SMP && X86_32 && !X86_BIGSMP
> > > > > -	range 2 512 if SMP && !MAXSMP
> > > > > +	range 2 1024 if SMP && !MAXSMP
> > > > >  	default "1" if !SMP
> > > > >  	default "4096" if MAXSMP
> > > > >  	default "32" if SMP && (X86_NUMAQ || X86_SUMMIT || X86_BIGSMP || X86_ES7000)
> > > > 
> > > > Any reason not to allow it to go up to 4096? The original concern was 
> > > > that CPUS=4096 wasn't working very well and you had to select MAXSMP 
> > > > deliberately and keep all the pieces.
> > 
> > No real reason to not allow all the way to 4096, no.  I just started
> > small as I wanted 1024 specifically, and this is the simplest way to
> > achieve that.
> > 
> > > The other reason was CONFIG_CPUMASK_OFFSTACK: with 4096 CPUs a cpumask is 
> > > 512 bytes, too large to be kept on the kernel stack.
> > > 
> > > MAXSMP forces CONFIG_CPUMASK_OFFSTACK so there's no such concern there.
> > > 
> > > With 1024 CPUs a single cpumask is 128 bytes - rather significant as well. 
> > > With 512 CPUs it's 64 bytes - borderline.
> > > 
> > > So I think a better solution would be to allow an increase above 512 CPUs 
> > > only if CONFIG_CPUMASK_OFFSTACK is also enabled.
> > 
> > OK, that makes sense.  So in this scenario, we could probably either:
> > 
> > a) do away with MAXSMP entirely and just depend on 
> > CONFIG_CPUMASK_OFFSTACK.
> > 
> > b) make MAXSMP something even higher than 4096.  Like 5120 or 6144, etc.
> > 
> > Which would you prefer?  Either is easy enough to code up, I just need 
> > to know which I should shoot for.
> 
> Why touch MAXSMP at all? It's really just a shortcut for 'configure the 
> kernel silly large', via a single option, nothing else. You are not forced 
> to use it and it should not affect configurability of NR_CPUS.
>
> What we _really_ want here is to fix NR_CPUS setting: to extend its range 
> and to enforce that NR_CPUS cannot be set larger than 512 without setting 
> CONFIG_CPUMASK_OFFSTACK.

OK.  I was just thinking that if we've come to the conclusion that 4096
CPUs isn't silly large anymore, we should make MAXSMP be something we
consider silly large.  Anyway, I'm fine with leaving MAXSMP as it is.
I'll work on the rest today.

josh
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/