Re: sched: tweak select_idle_sibling to look for idle threads

[Mike Galbraith]

On Tue, 2016-04-12 at 16:07 -0400, Chris Mason wrote:

> I think that if we're worried about the cost of the idle scan for this
> workload, find_idlest_group() is either going to hurt much more, or not
> search enough CPUs to find the idle one.

find_idlest_group()?  No no no, that's not what I mean at all.

wake_wide() identifies loads that really want to spread out, thus turns
off affine wakeups.  We still call select_idle_sibling(), only
difference being that target is the original cpu, not the waking cpu. 
 Given making that wide connection bidirectional helped FB's load, it
seems reasonable that passing wide information to select_idle_sibling()
would have a good chance of hitting the candidate that stands to gain
from a full socket scan, while also keeping that cache scrambling scan
far away from the rest. 

> But I'm happy to try patches or other ideas, I have a fixed version of
> the bitmap one going through production benchmarks now.

Making that wide/full search cheap is still good, because wake_wide()
also identifies interrupt sources that are waking many, so cheap wide
search should increase utilization there as well.  The thought was to
just make the wide thing have a tad wider effect on what it already
does affect.. and hope that doesn't demolish anything.

	-Mike