Re: [PATCH] cpu-hotplug: Register update_sched_domains() notifierwith higher prio

From: Andrew Morton
Date: Mon Mar 10 2008 - 18:27:21 EST


On Mon, 10 Mar 2008 18:43:55 +0530 Gautham R Shenoy <ego@xxxxxxxxxx> wrote:

> cpu-hotplug: Register update_sched_domains() notifier with higher prio
> From: Gautham R Shenoy <ego@xxxxxxxxxx>
>
> The current -rt wake_up logic uses the rq->rd->online_map which is
> updated on cpu-hotplug events by update_sched_domains(). Currently
> update_sched_domains() is registered with a priority 0. It is
> preferable that update_sched_domains() be the first notifier called on
> a CPU_DEAD or CPU_ONLINE events inorder to ensure that the
> rq->rd->online_map is in sync with the cpu_online_map.
>
> This way on a CPU_DOWN_PREPARE,
> we would destroy the sched_domains and reattach all the rq's to the
> def_root_domain. This will be followed by a CPU_DOWN_PREPARE in
> migration_call() which will clear the bit of the cpu going offline
> from the corresponding rq.
> This will ensure that no task will be woken up on the cpu which is
> going to be offlined between CPU_DOWN_PREPARE and CPU_DEAD.
>
> Ditto for the CPU_ONLINE path.
>
> Tested on a 4-way Intel Xeon Box with cpu-hotplug tests running in parallel
> with kernbench.
>
> Signed-off-by: Gautham R Shenoy <ego@xxxxxxxxxx>
> ---
>
> kernel/sched.c | 12 ++++++++++--
> 1 files changed, 10 insertions(+), 2 deletions(-)
>
> diff --git a/kernel/sched.c b/kernel/sched.c
> index 69ecb1a..374c46f 100644
> --- a/kernel/sched.c
> +++ b/kernel/sched.c
> @@ -7079,8 +7079,16 @@ void __init sched_init_smp(void)
> if (cpus_empty(non_isolated_cpus))
> cpu_set(smp_processor_id(), non_isolated_cpus);
> put_online_cpus();
> - /* XXX: Theoretical race here - CPU may be hotplugged now */
> - hotcpu_notifier(update_sched_domains, 0);
> + /*
> + * XXX: Theoretical race here - CPU may be hotplugged now
> + *
> + * We register the notifier with priority 11, which means that
> + * update_sched_domains() will be called just before migration_call().
> + *
> + * This is necessary to ensure that the rt wake up logic works fine
> + * and the rq->rd->online_map remains in sync with the cpu_online_map.
> + */
> + hotcpu_notifier(update_sched_domains, 11);
>
> /* Move init over to a non-isolated CPU */
> if (set_cpus_allowed(current, non_isolated_cpus) < 0)

This fixes my poweroff-doesn't-work post-2.6.24 regression.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/