[regression] cross core scheduling frequency drop bisected to 0c313cb20732

[Mike Galbraith]

Greetings,

While measuring current NO_HZ cost to light tasks jabbering cross core
at high frequency (~7% max), I noticed that master lost an improvement
for same acquired in 4.5, so bisected it.

4.5.0
homer:~ # taskset 0xc pipe-test 1
2.367681 usecs/loop -- avg 2.367681 844.7 KHz
2.372502 usecs/loop -- avg 2.368163 844.5 KHz
2.342506 usecs/loop -- avg 2.365597 845.5 KHz
2.383029 usecs/loop -- avg 2.367341 844.8 KHz
2.321859 usecs/loop -- avg 2.362792 846.5 KHz   1.00

master
homer:~ # taskset 0xc pipe-test 1
2.797656 usecs/loop -- avg 2.797656 714.9 KHz
2.804518 usecs/loop -- avg 2.798342 714.7 KHz
2.804206 usecs/loop -- avg 2.798929 714.6 KHz
2.802887 usecs/loop -- avg 2.799324 714.5 KHz
2.801577 usecs/loop -- avg 2.799550 714.4 KHz   0.84

master 0c313cb20732 reverted
homer:~ # !taskset
homer:~ # taskset 0xc pipe-test 1
2.277494 usecs/loop -- avg 2.277494 878.2 KHz
2.320979 usecs/loop -- avg 2.281843 876.5 KHz
2.272750 usecs/loop -- avg 2.280933 876.8 KHz
2.272209 usecs/loop -- avg 2.280061 877.2 KHz
2.277279 usecs/loop -- avg 2.279783 877.3 KHz   1.03

0c313cb207326f759a58f486214288411b25d4cf is the first bad commit
commit 0c313cb207326f759a58f486214288411b25d4cf
Author: Rafael J. Wysocki <rafael.j.wysocki@xxxxxxxxx>
Date:   Sun Mar 20 01:33:35 2016 +0100

    cpuidle: menu: Fall back to polling if next timer event is near
    
    Commit a9ceb78bc75c (cpuidle,menu: use interactivity_req to disable
    polling) changed the behavior of the fallback state selection part
    of menu_select() so it looks at interactivity_req instead of
    data->next_timer_us when it makes its decision.  That effectively
    caused polling to be used more often as fallback idle which led to
    significant increases of energy consumption in some cases.
    
    Commit e132b9b3bc7f (cpuidle: menu: use high confidence factors
    only when considering polling) changed that logic again to be more
    predictable, but that didn't help with the increased energy
    consumption problem.
    
    For this reason, go back to making decisions on which state to fall
    back to based on data->next_timer_us which is the time we know for
    sure something will happen rather than a prediction (which may be
    inaccurate and turns out to be so often enough to be problematic).
    However, take the target residency of the first proper idle state
    (C1) into account, so that state is not used as the fallback one
    if its target residency is greater than data->next_timer_us.
    
    Fixes: a9ceb78bc75c (cpuidle,menu: use interactivity_req to disable polling)
    Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@xxxxxxxxx>
    Reported-and-tested-by: Doug Smythies <dsmythies@xxxxxxxxx>