Re: [lkp-robot] [f2fs] 4ac912427c: -33.7% aim7.jobs-per-min regression

From: Thorsten Leemhuis
Date: Mon Mar 13 2017 - 12:40:05 EST

@Chao Yu/@Jaegeuk Kim: I'm considering to add this to the regressions
report for 4.11; or is there a reason why it shouldn't be considered a
regression? Ciao, Thorsten

On 08.03.2017 02:21, kernel test robot wrote:
>
> Greeting,
>
> We noticed a -33.7 regression of aim7.jobs-per-min due to commit:
>
> commit: 4ac912427c4214d8031d9ad6fbc3bc75e71512df ("f2fs: introduce free nid bitmap")
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
>
> in testcase: aim7
> on test machine: 40 threads Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz with 384G memory
> with following parameters:
>
> disk: 1BRD_48G
> fs: f2fs
> test: disk_wrt
> load: 3000
> cpufreq_governor: performance
>
> test-description: AIM7 is a traditional UNIX system level benchmark suite which is used to test and measure the performance of multiuser system.
> test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/
>
>
>
> Details are as below:
> -------------------------------------------------------------------------------------------------->
>
>
> To reproduce:
>
> git clone git://git.kernel.org/pub/scm/linux/kernel/git/wfg/lkp-tests.git
> cd lkp-tests
> bin/lkp install job.yaml # job file is attached in this email
> bin/lkp run job.yaml
>
> testcase/path_params/tbox_group/run: aim7/1BRD_48G-f2fs-disk_wrt-3000-performance/lkp-ivb-ep01
>
> ced2c7ea8e99b467 4ac912427c4214d8031d9ad6fb
> ---------------- --------------------------
> %stddev change %stddev
> \ | \
> 117419 Â 1% -33.7% 77863 Â 0% aim7.jobs-per-min
> 153.78 Â 1% +50.6% 231.63 Â 0% aim7.time.elapsed_time
> 153.78 Â 1% +50.6% 231.63 Â 0% aim7.time.elapsed_time.max
> 805644 Â 3% +11.3% 896604 Â 0% aim7.time.involuntary_context_switches
> 5408 Â 1% +15.4% 6240 Â 0% aim7.time.system_time
> 5066069 Â 0% +10.5% 5600256 Â 9% meminfo.DirectMap2M
> 135538 Â 8% -41.9% 78738 Â 8% meminfo.Dirty
> 980.67 Â 16% -67.8% 315.50 Â 12% meminfo.Writeback
> 71322 Â 10% -44.0% 39953 Â 1% numa-meminfo.node0.Dirty
> 11158 Â 18% -27.1% 8132 Â 0% numa-meminfo.node0.Mapped
> 56776 Â 6% -32.5% 38309 Â 0% numa-meminfo.node1.Dirty
> 9684 Â 22% +30.9% 12676 Â 0% numa-meminfo.node1.Mapped
> 6069 Â 57% -78.1% 1328 Â 18% softirqs.NET_RX
> 619333 Â 4% +8.0% 669152 Â 3% softirqs.RCU
> 128030 Â 2% +33.3% 170724 Â 0% softirqs.SCHED
> 2331994 Â 1% +15.3% 2688290 Â 0% softirqs.TIMER
> 7701 Â 1% -35.7% 4948 Â 3% vmstat.io.bo
> 64.67 Â 2% -39.7% 39.00 Â 2% vmstat.procs.b
> 333.33 Â 7% -48.5% 171.50 Â 2% vmstat.procs.r
> 17530 Â 1% -23.4% 13425 Â 1% vmstat.system.cs
> 47642 Â 1% -5.3% 45100 Â 1% vmstat.system.in
> 33522 Â 4% -43.1% 19068 Â 0% proc-vmstat.nr_dirty
> 236.00 Â 14% -66.1% 80.00 Â 3% proc-vmstat.nr_writeback
> 33907 Â 4% -43.3% 19222 Â 0% proc-vmstat.nr_zone_write_pending
> 28194 Â 10% +10.4% 31131 Â 6% proc-vmstat.pgactivate
> 746402 Â 2% +24.6% 929960 Â 3% proc-vmstat.pgfault
> 153.78 Â 1% +50.6% 231.63 Â 0% time.elapsed_time
> 153.78 Â 1% +50.6% 231.63 Â 0% time.elapsed_time.max
> 805644 Â 3% +11.3% 896604 Â 0% time.involuntary_context_switches
> 3524 Â 0% -23.4% 2701 Â 0% time.percent_of_cpu_this_job_got
> 5408 Â 1% +15.4% 6240 Â 0% time.system_time
> 12.19 Â 1% +36.7% 16.66 Â 0% time.user_time
> 48260939 Â 3% +12.1% 54110616 Â 2% cpuidle.C1-IVT.time
> 33149237 Â 5% +52.6% 50597349 Â 1% cpuidle.C1E-IVT.time
> 89642 Â 4% +52.8% 136976 Â 0% cpuidle.C1E-IVT.usage
> 13534795 Â 6% +276.3% 50934566 Â 55% cpuidle.C3-IVT.time
> 42893 Â 6% +138.8% 102439 Â 30% cpuidle.C3-IVT.usage
> 6.431e+08 Â 2% +390.1% 3.152e+09 Â 10% cpuidle.C6-IVT.time
> 802009 Â 2% +375.3% 3811880 Â 10% cpuidle.C6-IVT.usage
> 1535987 Â 4% +156.3% 3936830 Â 4% cpuidle.POLL.time
> 88.14 Â 0% -24.9% 66.17 Â 3% turbostat.%Busy
> 2659 Â 0% -44.7% 1471 Â 3% turbostat.Avg_MHz
> 3016 Â 0% -26.3% 2224 Â 0% turbostat.Bzy_MHz
> 5.20 Â 5% +127.0% 11.80 Â 2% turbostat.CPU%c1
>
>
>
> perf-stat.page-faults
>
> 1e+06 ++-----------------------------------------------------------------+
> 900000 O+O O O O O O O O |
> | O O O O O O O O O O |
> 800000 ++ .*.*. .*. .*. .*.. |
> 700000 ++*.*.*.*.*..*.* *.*.*.*.*.*..*.* * *.*.*.*.* *.*.*.* *
> | : : |
> 600000 ++: : |
> 500000 ++ : :|
> 400000 ++ : :|
> |: : :|
> 300000 ++ : :|
> 200000 ++ : :|
> | : |
> 100000 ++ : |
> 0 *+------------------------------------O-O------------------------*-+
>
>
> perf-stat.minor-faults
>
> 1e+06 ++-----------------------------------------------------------------+
> 900000 O+O O O O O O O O |
> | O O O O O O O O O O |
> 800000 ++ .*.*. .*. .*. .*.. |
> 700000 ++*.*.*.*.*..*.* *.*.*.*.*.*..*.* * *.*.*.*.* *.*.*.* *
> | : : |
> 600000 ++: : |
> 500000 ++ : :|
> 400000 ++ : :|
> |: : :|
> 300000 ++ : :|
> 200000 ++ : :|
> | : |
> 100000 ++ : |
> 0 *+------------------------------------O-O------------------------*-+
>
>
> aim7.jobs-per-min
>
> 140000 ++-----------------------------------------------------------------+
> | *. .*.. .*.*.*.*.*.*.*..*.*.*.*.*.*. |
> 120000 ++: *.*.* *.*.* *.*.*.*.*..*.*.*. *
> | : * |
> 100000 ++: : |
> |: : |
> 80000 O+O O O O O O O O O O O O O O O O O O : :|
> |: : :|
> 60000 ++ : :|
> |: : :|
> 40000 ++ : :|
> | : :|
> 20000 ++ : |
> | : |
> 0 *+------------------------------------O-O------------------------*-+
>
>
>
> [*] bisect-good sample
> [O] bisect-bad sample
>
>
> Disclaimer:
> Results have been estimated based on internal Intel analysis and are provided
> for informational purposes only. Any difference in system hardware or software
> design or configuration may affect actual performance.
>
>
> Thanks,
> Xiaolong
>