Re: [PATCH 03/16] sched/fair: Disregard idle task wakee_flips in wake_wide

[Peter Zijlstra]

On Wed, Jun 01, 2016 at 09:57:23PM +0200, Peter Zijlstra wrote:
> On Mon, May 23, 2016 at 01:00:10PM +0100, Morten Rasmussen wrote:
> > On Mon, May 23, 2016 at 01:12:07PM +0200, Mike Galbraith wrote:
> > > On Mon, 2016-05-23 at 11:58 +0100, Morten Rasmussen wrote:
> > > > wake_wide() is based on task wakee_flips of the waker and the wakee to
> > > > decide whether an affine wakeup is desirable. On lightly loaded systems
> > > > the waker is frequently the idle task (pid=0) which can accumulate a lot
> > > > of wakee_flips in that scenario. It makes little sense to prevent affine
> > > > wakeups on an idle cpu due to the idle task wakee_flips, so it makes
> > > > more sense to ignore them in wake_wide().
> > > 
> > > You sure?  What's the difference between a task flipping enough to
> > > warrant spreading the load, and an interrupt source doing the same? 
> > >  I've both witnessed firsthand, and received user confirmation of this
> > > very thing improving utilization.
> > 
> > Right, I didn't consider the interrupt source scenario, my fault.
> > 
> > The problem then seems to be distinguishing truly idle and busy doing
> > interrupts. The issue that I observe is that wake_wide() likes pushing
> > tasks around in lightly scenarios which isn't desirable for power
> > management. Selecting the same cpu again may potentially let others
> > reach deeper C-state.
> > 
> > With that in mind I will if I can do better. Suggestions are welcome :-)
> 
> Seeing how we always so select_idle_siblings() after affine_sd, the only
> wake_affine movement that matters is cross-llc.
> 
> So intra-llc wakeups can avoid the movement, no?

Won't help I think; the interrupt that got us in this situation will
already have wrecked your idle time/state to begin with. You really want
to help interrupt routing.