Re: [PATCH RFC 1/2] sched: Minimize the idle cpu selection race window.

[Mike Galbraith]

On Tue, 2017-10-31 at 09:20 +0100, Peter Zijlstra wrote:
> On Tue, Oct 31, 2017 at 12:27:41AM -0500, Atish Patra wrote:
> > Currently, multiple tasks can wakeup on same cpu from
> > select_idle_sibiling() path in case they wakeup simulatenously
> > and last ran on the same llc. This happens because an idle cpu
> > is not updated until idle task is scheduled out. Any task waking
> > during that period may potentially select that cpu for a wakeup
> > candidate.
> > 
> > Introduce a per cpu variable that is set as soon as a cpu is
> > selected for wakeup for any task. This prevents from other tasks
> > to select the same cpu again. Note: This does not close the race
> > window but minimizes it to accessing the per-cpu variable. If two
> > wakee tasks access the per cpu variable at the same time, they may
> > select the same cpu again. But it minimizes the race window
> > considerably.
> 
> The very most important question; does it actually help? What
> benchmarks, give what numbers?

I played with something ~similar (cmpxchg() idle cpu reservation) a
while back in the context of schbench, and it did help that, but for
generic fast mover benchmarks, the added overhead had the expected
effect, it shaved throughput a wee bit (rob Peter, pay Paul, repeat).

I still have the patch lying about in my rubbish heap, but didn't
bother to save any of the test results.

	-Mike