Re: smp_call_function_single lockups

From: Rafael David Tinoco
Date: Thu Feb 19 2015 - 12:30:48 EST

I could only find an advisory (regarding sr-iov and irq remaps) from
HP to RHEL6.2 users stating that Gen8 firmware does not enable it by

The interrupt remapping capability depends on x2apic enabled in the
BIOS and HP ProLiant Gen8 systems do not enable x2apic; therefore, the
following workaround is required for device assignment: Edit the
/etc/grub.conf and add intremap=no_x2apic_optout option to the kernel
command line options.

Probably for backwards compatibility... not sure if there is an option
to enable/disable it in firmware (like DL390 seems to have).
I don't think so... but I was told by HP team that I should use x2apic
for >= Gen8.

On Thu, Feb 19, 2015 at 2:59 PM, Linus Torvalds
<torvalds@xxxxxxxxxxxxxxxxxxxx> wrote:
> On Thu, Feb 19, 2015 at 8:32 AM, Rafael David Tinoco <inaddy@xxxxxxxxxx> wrote:
>> Feb 19 08:21:28 derain kernel: [ 3.637682] Switched APIC routing to
>> cluster x2apic.
> Ok. That "cluster x2apic" mode is just about the nastiest mode when it
> comes to sending a single ipi. We do that insane dance where we
> - turn single cpu number into cpumask
> - copy the cpumask to a percpu temporary storage
> - walk each cpu in the cpumask
> - for each cpu, look up the cluster siblings
> - for each cluster sibling that is also in the cpumask, look up the
> logical apic mask and add it to the actual ipi destination mask
> - send an ipi to that final mask.
> which is just insane. It's complicated, it's fragile, and it's unnecessary.
> If we had a simple "send_IPI()" function, we could do this all with
> something much saner, and it would look sopmething like
> static void x2apic_send_IPI(int cpu, int vector)
> {
> u32 dest = per_cpu(x86_cpu_to_logical_apicid, cpu);
> x2apic_wrmsr_fence();
> __x2apic_send_IPI_dest(dest, vector, APIC_DEST_LOGICAL);
> }
> and then 'void native_send_call_func_single_ipi()' would just look like
> void native_send_call_func_single_ipi(int cpu)
> {
> }
> but I might have missed something (and we might want to have a wrapper
> that says "if the apic doesn't have a 'send_IPI' function, use
> "send_IPI_mask(cpumask_of(cpu, vector) instead"
> The fact that you need that no_x2apic_optout (which in turn means that
> your ACPI tables seem to say "don't use x2apic") also makes me worry.
> Are there known errata for the x2apic?
> Linus
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at
Please read the FAQ at