Deadlock between cpu_hotplug_begin and cpu_add_remove_lock

From: Paul Mackerras
Date: Wed Jan 22 2014 - 00:53:36 EST

This arises out of a report from a tester that offlining a CPU never
finished on a system they were testing. This was on a POWER8 running
a 3.10.x kernel, but the issue is still present in mainline AFAICS.

What I found when I looked at the system was this:

* There was a ppc64_cpu process stuck inside cpu_hotplug_begin(),
called from _cpu_down(), from cpu_down(). This process was holding
the cpu_add_remove_lock mutex, since cpu_down() calls
cpu_maps_update_begin() before calling _cpu_down(). It was stuck
there because cpu_hotplug.refcount == 1.

* There was a mdadm process trying to acquire the cpu_add_remove_lock
mutex inside register_cpu_notifier(), called from
raid5_alloc_percpu() in drivers/md/raid5.c. That process had
previously called get_online_cpus, which is why cpu_hotplug.refcount
was 1.

Result: deadlock.

Thus it seems that the following code is not safe:


There are a few different places that do that sort of thing; besides
drivers/md/raid5.c, there are instances in arch/x86/kernel/cpu,
arch/x86/oprofile, drivers/cpufreq/acpi-cpufreq.c,
drivers/oprofile/nmi_timer_int.c and kernel/trace/ring_buffer.c.

My question is this: is it reasonable to call register_cpu_notifier
inside a get/put_online_cpus block? If so, the deadlock needs to be
fixed; if not, the callers need to be fixed, and the restriction
should be documented.

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at