Re: futex funkiness -- massive lockups

[Peter Zijlstra]

On Tue, Mar 04, 2014 at 05:43:04PM -0800, Davidlohr Bueso wrote:
> Hi,
> 
> A large amount of lockups are seen on a 480 core system doing some sort
> of database-like workload. All except one are soft lockups. This is a
> SLES11 system with most of the recent futex changes backported,
> including commits 63b1a816, b0c29f79, 99b60ce6, a52b89eb, 0d00c7b2,
> 5cdec2d8 and f12d5bfc.
> 
> [212071.494920]  [<ffffffff810589e5>] load_balance+0xa5/0x470
> [212071.494920]  [<ffffffff81058f13>] rebalance_domains+0x163/0x220
> [212071.494920]  [<ffffffff81059144>] run_rebalance_domains+0x44/0x60
> [212071.494920]  [<ffffffff810677df>] __do_softirq+0x11f/0x260
> [212071.494920]  [<ffffffff81469fdc>] call_softirq+0x1c/0x30
> [212071.494920]  [<ffffffff81004435>] do_softirq+0x65/0xa0
> [212071.494920]  [<ffffffff810675a5>] irq_exit+0xc5/0xe0
> [212071.494920]  [<ffffffff81026588>] smp_apic_timer_interrupt+0x68/0xa0
> [212071.494920]  [<ffffffff81469773>] apic_timer_interrupt+0x13/0x20
> [212071.494920]  [<ffffffff81460ec5>] _raw_spin_lock+0x15/0x20
> [212071.494920]  [<ffffffff8109626a>] futex_wake+0xba/0x180
> [212071.494920]  [<ffffffff810987c4>] do_futex+0x94/0x1c0
> [212071.494920]  [<ffffffff81098972>] sys_futex+0x82/0x170
> [212071.494920]  [<ffffffff81468d92>] system_call_fastpath+0x16/0x1b
> 

Like Linus said; that looks like its stuck in the load balancer. Now 480
is certainly more CPUs that usual. However, SGI ran with lots more and I
don't recall them seeing soft lockups from this.

OTOH I do know the softirq runs for more than a softirq should; even on
moderate systems.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/