Re: Bulk CPU Hotplug (Was Re: [PATCH] Do not force shutdown/rebootto boot cpu.)

[Robin Holt]

On Mon, Apr 15, 2013 at 12:16:44PM +0200, Ingo Molnar wrote:
> 
> * Robin Holt <holt@xxxxxxx> wrote:
> 
> > From 1767003c943325e52ac78cac6fdbaf2ab638888d Mon Sep 17 00:00:00 2001
> > From: Robin Holt <holt@xxxxxxx>
> > Date: Wed, 3 Apr 2013 13:52:00 -0500
> > Subject: [PATCH] Migrate shutdown/reboot to boot cpu.
> > 
> > We recently noticed that reboot of a 1024 cpu machine takes approx 16
> > minutes of just stopping the cpus.  The slowdown was tracked to commit
> > f96972f.
> > 
> > The current implementation does all the work of hot removing the cpus
> > before halting the system.  We are switching to just migrating to the
> > boot cpu and then calling continuing with shutdown/reboot.
> > 
> > This also has the effect of not breaking x86's command line parameter for
> > specifying the reboot cpu.  Note, this code was shamelessly copied from
> > arch/x86/kernel/reboot.c with bits removed pertaining to the reboot_cpu
> > command line parameter.
> > 
> > Signed-off-by: Robin Holt <holt@xxxxxxx>
> > To: Shawn Guo <shawn.guo@xxxxxxxxxx>
> > To: Ingo Molnar <mingo@xxxxxxxxxx>
> > To: Russ Anderson <rja@xxxxxxx>
> > Cc: Andrew Morton <akpm@xxxxxxxxxxxxxxxxxxxx>
> > Cc: "H. Peter Anvin" <hpa@xxxxxxxxx>
> > Cc: Lai Jiangshan <laijs@xxxxxxxxxxxxxx>
> > Cc: Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx>
> > Cc: Linux Kernel Mailing List <linux-kernel@xxxxxxxxxxxxxxx>
> > Cc: Michel Lespinasse <walken@xxxxxxxxxx>
> > Cc: Oleg Nesterov <oleg@xxxxxxxxxx>
> > Cc: "Paul E. McKenney" <paulmck@xxxxxxxxxxxxxxxxxx>
> > Cc: Paul Mackerras <paulus@xxxxxxxxx>
> > Cc: Peter Zijlstra <peterz@xxxxxxxxxxxxx>
> > Cc: Robin Holt <holt@xxxxxxx>
> > Cc: "rusty@xxxxxxxxxxxxxxx" <rusty@xxxxxxxxxxxxxxx>
> > Cc: Tejun Heo <tj@xxxxxxxxxx>
> > Cc: the arch/x86 maintainers <x86@xxxxxxxxxx>
> > Cc: Thomas Gleixner <tglx@xxxxxxxxxxxxx>
> > Cc: <stable@xxxxxxxxxxxxxxx>
> > ---
> >  kernel/sys.c | 17 +++++++++++++++--
> >  1 file changed, 15 insertions(+), 2 deletions(-)
> > 
> > diff --git a/kernel/sys.c b/kernel/sys.c
> > index 0da73cf..4d1047d 100644
> > --- a/kernel/sys.c
> > +++ b/kernel/sys.c
> > @@ -357,6 +357,19 @@ int unregister_reboot_notifier(struct notifier_block *nb)
> >  }
> >  EXPORT_SYMBOL(unregister_reboot_notifier);
> >  
> > +void migrate_to_boot_cpu(void)
> > +{
> > +	/* The boot cpu is always logical cpu 0 */
> > +	int reboot_cpu_id = 0;
> > +
> > +	/* Make certain the cpu I'm about to reboot on is online */
> > +	if (!cpu_online(reboot_cpu_id))
> > +		reboot_cpu_id = smp_processor_id();
> > +
> > +	/* Make certain I only run on the appropriate processor */
> > +	set_cpus_allowed_ptr(current, cpumask_of(reboot_cpu_id));
> > +}
> 
> I guess you suspect what I'm going to suggest next? :-)
> 
> While I think something like this commit would be acceptable as a minimal 
> regression fix, it would be really lovely to add a second patch as well, which 
> removes the same code from arch/x86/kernel/reboot.c:native_machine_shutdown() and 
> merged it with the kernel/sys.c version? That way all platforms gained a 
> reboot_cpu command line, and we'd have less code duplication as well. Win-win.
> 
> ( While at it, it might make sense to move the reboot/shutdown related bits from 
>   kernel/sys.c to a new kernel/shutdown.c or kernel/reboot.c file or so. )

That all sounds good.  I will work on that this morning.

Robin
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/