Re: [lkp-developer] [sched/fair] 4e5160766f: +149% ftq.noise.50% regression

[Vincent Guittot]

Hi Ying,

On 12 December 2016 at 06:43, kernel test robot
<ying.huang@xxxxxxxxxxxxxxx> wrote:
> Greeting,
>
> FYI, we noticed a 149% regression of ftq.noise.50% due to commit:
>
>
> commit: 4e5160766fcc9f41bbd38bac11f92dce993644aa ("sched/fair: Propagate asynchrous detach")
> https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master
>
> in testcase: ftq
> on test machine: 8 threads Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz with 8G memory
> with following parameters:
>
>         nr_task: 100%
>         samples: 6000ss
>         test: cache
>         freq: 20
>         cpufreq_governor: powersave

Why using powersave ? Are you testing  every governors ?

>
> test-description: The FTQ benchmarks measure hardware and software interference or 'noise' on a node from the applications perspective.
> test-url: https://github.com/rminnich/ftq

It's a bit difficult to understand exactly what is measured and what
is ftq.noise.50% because this result is not part of the bench which
seems to only record a log of data in a file and ftq.noise.50% seems
to be lkp specific

I have tried to reproduce the lkp test on a debian jessie then a
ubuntu server 16.10 but lkp doesn't seems to install cleanly as there
are some errors:

sudo bin/lkp run     job.yaml
IPMI BMC is not supported on this machine, skip bmc-watchdog setup!
2016-12-12 13:58:39 ./ftq_cache -f 20 -n 6000 -t 8 -a 524288
Start 5088418680237 end 5438443372098 elapsed 350024691861
cyclestart 14236344834332 cycleend 15214154208877 elapsed 977809374545
Avg Cycles(ticks) per ns. is 2.793544; nspercycle is 0.357968
Pre-computed ticks per ns: 2.793541
Sample frequency is 20.000000
ticks per ns 2.79354
chown: utilisateur incorrect: Âlkp.lkpÂ
chown: utilisateur incorrect: Âlkp.lkpÂ
wait for background monitors: 9405 9407 oom-killer nfs-hang
curl: (6) Could not resolve host: ftq.time


>
> In addition to that, the commit also has significant impact on the following tests:
>
> +------------------+--------------------------------------------------------------------------------+
> | testcase: change | unixbench: unixbench.score 2.7% improvement                                    |
> | test machine     | 4 threads Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz with 4G memory               |
> | test parameters  | cpufreq_governor=performance                                                   |
> |                  | nr_task=100%                                                                   |
> |                  | runtime=300s                                                                   |
> |                  | test=execl                                                                     |
> +------------------+--------------------------------------------------------------------------------+
>
>
> Details are as below:
> -------------------------------------------------------------------------------------------------->
>
>
> To reproduce:
>
>         git clone git://git.kernel.org/pub/scm/linux/kernel/git/wfg/lkp-tests.git
>         cd lkp-tests
>         bin/lkp install job.yaml  # job file is attached in this email
>         bin/lkp run     job.yaml
>
> testcase/path_params/tbox_group/run: ftq/100%-6000ss-cache-20-powersave/lkp-hsw-d01
>
> 09a43ace1f986b00  4e5160766fcc9f41bbd38bac11
> ----------------  --------------------------
>          %stddev      change         %stddev
>              \          |                \
>        305 Â 30%       260%       1100 Â 14%  ftq.noise.75%
>       1386 Â 19%       149%       3457 Â  7%  ftq.noise.50%
>       2148 Â 11%        98%       4257 Â  4%  ftq.noise.25%
>    3963589                     3898578        ftq.time.involuntary_context_switches
>
>
>
>                                    ftq.noise.50_
>
>   4000 ++------------O------------------------------------------------------+
>        |                                                           O      O |
>   3500 ++     O             O                        O    O O O             O
>        | O  O      O   O      O O  O O O    O O    O   O         O          |
>        O        O                         O                          O O    |
>   3000 ++                                       O                           |
>        |                 O                                                  |
>   2500 ++                                                                   |
>        |                                                                    |
>   2000 ++                                                                   |
>        |    *                  .*                                           |
>        |   + :     *   *      *  +                                          |
>   1500 ++ +  :    + + + +    :    + .*                                      |
>        |.*    *. +   *   *.. :     *  +                                     |
>   1000 *+-------*-----------*----------*------------------------------------+
>
>         [*] bisect-good sample
>         [O] bisect-bad  sample
>
>
> Disclaimer:
> Results have been estimated based on internal Intel analysis and are provided
> for informational purposes only. Any difference in system hardware or software
> design or configuration may affect actual performance.
>
>
> Thanks,
> Ying Huang