Re: [PATCH 1/1] sched: prevent divide by zero error incpu_avg_load_per_task

[Ingo Molnar]

* Steven Rostedt <rostedt@xxxxxxxxxxx> wrote:

> From: Steven Rostedt <rostedt@xxxxxxxxxxx>
> 
> Impact: fix to divide by zero
> 
> While testing the branch profiler, I hit this crash:
> 
> divide error: 0000 [#1] PREEMPT SMP
> [...]
> Call Trace:
>  <IRQ> <0> [<ffffffff8024fd43>] find_busiest_group+0x3e5/0xcaa
>  [<ffffffff8025da75>] rebalance_domains+0x2da/0xa21

> The code for cpu_avg_load_per_task has:
> 
> 	if (rq->nr_running)
> 		rq->avg_load_per_task = rq->load.weight / rq->nr_running;
> 
> The runqueue lock is not held here, and there is nothing that 
> prevents the rq->nr_running from going to zero after it passes the 
> if condition.
> 
> The branch profiler simply made the race window bigger.
> 
> This patch saves off the rq->nr_running to a local variable and uses 
> that for both the condition and the division.

good catch! Applied to tip/sched/urgent, thanks Steve!

the rebalancer scans remote runqueues without holding the runqueue 
lock for performance reasons, so nr_running indeed has to be loaded 
into a local variable here.

I think it could hit anywhere upstream as well, even without the 
branch tracer: depends on the register pressure and GCC's choices for 
reloading that register. If say FRAME_POINTER is enabled and we are 
running with some more agressive optimization or just an older/suckier 
GCC that does a spurious reload, then this might happen too.

it's not a classic race, it's an SMP bug that will either trigger or 
not trigger at all, depending on compiler behavior.

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/