Re: [patch] sched/fair: Move se->vruntime normalization state into struct sched_entity

[Paul E. McKenney]

On Tue, May 24, 2016 at 10:04:17AM -0700, Paul E. McKenney wrote:
> On Mon, May 23, 2016 at 2:19 AM +0200, Peter Zijlstra wrote:
> > On Sun, May 22, 2016 at 09:00:01AM +0200, Mike Galbraith wrote:
> > > On Sat, 2016-05-21 at 21:00 +0200, Mike Galbraith wrote:
> > > > On Sat, 2016-05-21 at 16:04 +0200, Mike Galbraith wrote:
> > > >
> > > > > Wakees that were not migrated/normalized eat an unwanted min_vruntime,
> > > > > and likely take a size XXL latency hit.  Big box running master bled
> > > > > profusely under heavy load until I turned TTWU_QUEUE off.
> > >
> > > May as well make it official and against master.today.  Fly or die
> > > little patchlet.
> > >
> > > sched/fair: Move se->vruntime normalization state into struct sched_entity
> > 
> > Does this work?
> 
> This gets rid of the additional lost wakeups introduced during the
> merge window, thank you!
> 
> The pre-existing low-probability lost wakeups still persist, sad to say
> Can't have everything, I guess.

However, their behavior has changed.  Previously, the TREE03 rcutorture
scenario had the most stalls.  Now TREE07 appears to be more likely.
The corresponding Kconfig fragments are shown below, in case it gives
someone ideas about where the problem might be.

My current guess, based on insufficient data, is that TREE07 has about
an 80-90% chance of seeing a lost wakeup during a two-hour run.

							Thanx, Paul

------------------------------------------------------------------------
TREE03
------------------------------------------------------------------------
CONFIG_SMP=y
CONFIG_NR_CPUS=16
CONFIG_PREEMPT_NONE=n
CONFIG_PREEMPT_VOLUNTARY=n
CONFIG_PREEMPT=y
#CHECK#CONFIG_PREEMPT_RCU=y
CONFIG_HZ_PERIODIC=y
CONFIG_NO_HZ_IDLE=n
CONFIG_NO_HZ_FULL=n
CONFIG_RCU_TRACE=y
CONFIG_HOTPLUG_CPU=y
CONFIG_RCU_FANOUT=2
CONFIG_RCU_FANOUT_LEAF=2
CONFIG_RCU_NOCB_CPU=n
CONFIG_DEBUG_LOCK_ALLOC=n
CONFIG_RCU_BOOST=y
CONFIG_RCU_KTHREAD_PRIO=2
CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
CONFIG_RCU_EXPERT=y

------------------------------------------------------------------------
TREE07
------------------------------------------------------------------------
CONFIG_SMP=y
CONFIG_NR_CPUS=16
CONFIG_CPUMASK_OFFSTACK=y
CONFIG_PREEMPT_NONE=y
CONFIG_PREEMPT_VOLUNTARY=n
CONFIG_PREEMPT=n
#CHECK#CONFIG_TREE_RCU=y
CONFIG_HZ_PERIODIC=n
CONFIG_NO_HZ_IDLE=n
CONFIG_NO_HZ_FULL=y
CONFIG_NO_HZ_FULL_ALL=n
CONFIG_NO_HZ_FULL_SYSIDLE=y
CONFIG_RCU_FAST_NO_HZ=n
CONFIG_RCU_TRACE=y
CONFIG_HOTPLUG_CPU=y
CONFIG_RCU_FANOUT=2
CONFIG_RCU_FANOUT_LEAF=2
CONFIG_RCU_NOCB_CPU=n
CONFIG_DEBUG_LOCK_ALLOC=n
CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
CONFIG_RCU_EXPERT=y