Re: [lkp] [kernel/fs] d57d611505: +54.7% turbostat.%Busy
From: Stephane Gasparini
Date: Tue Feb 23 2016 - 03:55:44 EST
The increase of this metric is expected as the I/O wait (I/O busy) were not accounted on some write operation before this patch and are accounted now.
â
Steph
> On Feb 23, 2016, at 9:26 AM, kernel test robot <ying.huang@xxxxxxxxxxxxxxx> wrote:
>
> FYI, we noticed the below changes on
>
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
> commit d57d611505d911c6f9f81cd9bd6dbd293d66dd9f ("kernel/fs: fix I/O wait not accounted for RW O_DSYNC")
>
>
> =========================================================================================
> compiler/cpufreq_governor/disk/filesize/fs/iterations/kconfig/nr_directories/nr_files_per_directory/nr_threads/rootfs/sync_method/tbox_group/test_size/testcase:
> gcc-4.9/performance/1HDD/9B/f2fs/1x/x86_64-rhel/16d/256fpd/32t/debian-x86_64-2015-02-07.cgz/fsyncBeforeClose/lkp-ne04/400M/fsmark
>
> commit:
> 09954bad448791ef01202351d437abdd9497a804
> d57d611505d911c6f9f81cd9bd6dbd293d66dd9f
>
> 09954bad448791ef d57d611505d911c6f9f81cd9bd
> ---------------- --------------------------
> %stddev %change %stddev
> \ | \
> 14760 Â 1% +6.3% 15695 Â 1% fsmark.time.involuntary_context_switches
> 46.00 Â 0% -8.2% 42.25 Â 1% fsmark.time.percent_of_cpu_this_job_got
> 26151 Â116% -71.3% 7498 Â 23% latency_stats.sum.call_rwsem_down_read_failed.f2fs_mkdir.[f2fs].vfs_mkdir.SyS_mkdir.entry_SYSCALL_64_fastpath
> 7418 Â 23% -27.7% 5361 Â 3% numa-vmstat.node0.nr_slab_unreclaimable
> 1.47 Â 1% -9.9% 1.33 Â 0% time.user_time
> 1447 Â 4% -29.0% 1028 Â 7% uptime.idle
> 7.75 Â 19% +206.5% 23.75 Â 1% vmstat.procs.b
> 29675 Â 23% -27.7% 21447 Â 3% numa-meminfo.node0.SUnreclaim
> 83099 Â 8% -11.5% 73568 Â 1% numa-meminfo.node0.Slab
> 11.91 Â 1% +54.7% 18.43 Â 1% turbostat.%Busy
> 358.00 Â 1% +60.0% 572.75 Â 1% turbostat.Avg_MHz
> 21.54 Â 0% -17.4% 17.79 Â 1% turbostat.CPU%c1
> 24.78 Â 3% +41.1% 34.98 Â 1% turbostat.CPU%c3
> 41.76 Â 1% -31.0% 28.81 Â 1% turbostat.CPU%c6
> 8.53 Â 5% -25.6% 6.34 Â 10% turbostat.Pkg%pc3
> 66312232 Â 0% -68.5% 20910669 Â 4% cpuidle.C1-NHM.time
> 46386272 Â 2% -20.5% 36884085 Â 5% cpuidle.C1E-NHM.time
> 2.79e+08 Â 2% +51.5% 4.226e+08 Â 0% cpuidle.C3-NHM.time
> 254315 Â 2% +25.1% 318250 Â 2% cpuidle.C3-NHM.usage
> 7.585e+08 Â 1% -22.2% 5.901e+08 Â 1% cpuidle.C6-NHM.time
> 415414 Â 1% -38.1% 257115 Â 1% cpuidle.C6-NHM.usage
> 1.135e+08 Â 1% +77.8% 2.017e+08 Â 2% cpuidle.POLL.time
> 102777 Â 1% +36.0% 139811 Â 1% cpuidle.POLL.usage
> 2011 Â 25% -27.1% 1467 Â 2% sched_debug.cfs_rq:/.exec_clock.2
> 1820 Â 3% -12.2% 1597 Â 7% sched_debug.cfs_rq:/.exec_clock.4
> 4297 Â 6% +78.1% 7653 Â 54% sched_debug.cfs_rq:/.min_vruntime.12
> 4803 Â 18% -16.1% 4028 Â 21% sched_debug.cfs_rq:/.min_vruntime.8
> 3763 Â 4% -17.0% 3124 Â 7% sched_debug.cfs_rq:/.min_vruntime.9
> 3408 Â 4% -14.9% 2900 Â 2% sched_debug.cfs_rq:/.min_vruntime.min
> 4.75 Â 54% -73.7% 1.25 Â173% sched_debug.cfs_rq:/.nr_spread_over.2
> 1.99 Â 16% +25.5% 2.50 Â 9% sched_debug.cfs_rq:/.nr_spread_over.stddev
> -4362 Â-25% -92.7% -318.24 Â-1304% sched_debug.cfs_rq:/.spread0.12
> 200.25 Â 47% -48.8% 102.50 Â 22% sched_debug.cfs_rq:/.util_avg.7
> 187.75 Â 64% -77.2% 42.75 Â 56% sched_debug.cfs_rq:/.util_avg.8
> 1.41 Â 31% +37.0% 1.93 Â 12% sched_debug.cpu.clock.stddev
> 1.41 Â 31% +37.0% 1.93 Â 12% sched_debug.cpu.clock_task.stddev
> 1.63 Â 67% -53.9% 0.75 Â 38% sched_debug.cpu.cpu_load[4].stddev
> 33822 Â 16% -17.2% 28018 Â 4% sched_debug.cpu.nr_switches.13
> 36824 Â 3% +12.9% 41569 Â 4% sched_debug.cpu.nr_switches.14
> 33117 Â 7% -11.1% 29438 Â 5% sched_debug.cpu.nr_switches.15
> 57634 Â 3% +8.8% 62716 Â 3% sched_debug.cpu.nr_switches.4
> 1723 Â 7% -11.3% 1527 Â 5% sched_debug.cpu.nr_uninterruptible.10
> 1378 Â 6% -8.7% 1258 Â 5% sched_debug.cpu.nr_uninterruptible.15
> 1232 Â 10% +25.3% 1543 Â 3% sched_debug.cpu.nr_uninterruptible.8
> 34660 Â 13% -19.1% 28034 Â 4% sched_debug.cpu.sched_count.13
> 37545 Â 2% +18.9% 44629 Â 7% sched_debug.cpu.sched_count.14
> 34003 Â 9% -13.4% 29456 Â 5% sched_debug.cpu.sched_count.15
> 62715 Â 6% +9.5% 68642 Â 7% sched_debug.cpu.sched_count.2
> 14091 Â 19% -20.4% 11212 Â 5% sched_debug.cpu.sched_goidle.13
> 14175 Â 3% +15.4% 16354 Â 6% sched_debug.cpu.sched_goidle.14
> 13465 Â 9% -12.5% 11782 Â 6% sched_debug.cpu.sched_goidle.15
> 24646 Â 3% +9.4% 26968 Â 4% sched_debug.cpu.sched_goidle.4
> 15258 Â 6% -22.8% 11773 Â 4% sched_debug.cpu.ttwu_count.11
> 15765 Â 5% +48.1% 23355 Â 15% sched_debug.cpu.ttwu_count.12
> 16869 Â 3% -26.1% 12461 Â 4% sched_debug.cpu.ttwu_count.13
> 3142 Â 5% +13.0% 3551 Â 8% sched_debug.cpu.ttwu_local.13
>
> lkp-ne04: Nehalem-EP
> Memory: 12G
>
> uptime.idle
>
> 1600 ++-------------------------------------------------------------------+
> | .*. *. |
> 1500 *+**.** * **.*.**.* *.**.**.*.**.* *.**.*.**.**.*.**.**.*.* **.*
> 1400 ++ : : : : :+ |
> | :: :: * |
> 1300 ++ * * |
> | |
> 1200 ++ |
> | |
> 1100 O+OO OO O O OO O OO OO O O OO O |
> 1000 ++ |
> | |
> 900 ++ O O |
> | |
> 800 ++-------------------------------------------------------------------+
>
>
> cpuidle.POLL.time
>
> 2.2e+08 ++---------------O--O----O----------------------------------------+
> O O O O O |
> 2e+08 ++O O O O O O O O |
> | O OO O O |
> | |
> 1.8e+08 ++ |
> | |
> 1.6e+08 ++ |
> | |
> 1.4e+08 ++ |
> | |
> | |
> 1.2e+08 ++ .* .* *. .*.**. .**. *.**. *.* *. *.**.**.**.|
> *.**.** * *. : **. * ** **.* *.* *.* * *
> 1e+08 ++------------*-----*---------------------------------------------+
>
>
> cpuidle.C1-NHM.time
>
> 8e+07 ++------------------------------------------------------------------+
> | |
> 7e+07 ++ .* *. .* .* *. *. .* *.* |
> | *. *.**.*. .**.*. * *.* * *. *.* *.* * *. * *. .* *.**.*
> 6e+07 *+* * ** * * * * |
> | |
> 5e+07 ++ |
> | |
> 4e+07 ++ |
> | |
> 3e+07 ++ |
> | O O |
> 2e+07 O+OO OO OO O OO O OO OO OO O O |
> | |
> 1e+07 ++------------------------------------------------------------------+
>
>
> cpuidle.C3-NHM.time
>
> 4.4e+08 ++------------O---------------------------------------------------+
> | OO O O O O O O |
> 4.2e+08 O+ O O O O O O |
> 4e+08 ++ O O O O O |
> | |
> 3.8e+08 ++ |
> 3.6e+08 ++ |
> | |
> 3.4e+08 ++ |
> 3.2e+08 ++ |
> | * |
> 3e+08 ++ +: ** *.* .* |
> 2.8e+08 *+ *. *. *. * :.* + + .* .**. .* :.* .*.* * *.**.* .* .*
> | * * * * * * * * ** * * *.* * *.** |
> 2.6e+08 ++----------------------------------------------------------------+
>
>
> cpuidle.C6-NHM.time
>
> 8e+08 ++----------*-----------------------------------------------------+
> | * .* .* + .* |
> |.* + * * ** *. *.*. *. *.**.**.**. *. .**.**.**. *.**. *.* |
> 7.5e+08 *+ * * * * * * * * *.*
> | |
> | |
> 7e+08 ++ |
> | |
> 6.5e+08 ++ |
> | |
> O |
> 6e+08 ++ O OO O O O O O O |
> | O OO O O O O O O |
> | O O |
> 5.5e+08 ++----------------------------------------------------------------+
>
>
> cpuidle.C6-NHM.usage
>
> 440000 ++-----------------------------------------------------------------+
> 420000 ++ *. *. *. .*. * .* .* .*.* .* |
> |.* .**.**. : *.* * **.** * + *.**.*.** * :.** : .** *.*
> 400000 *+ * * * * *.** |
> 380000 ++ |
> | |
> 360000 ++ |
> 340000 ++ |
> 320000 ++ |
> | |
> 300000 ++ |
> 280000 ++ |
> | O O O O O |
> 260000 O+ O OO OO O O OO O O OO O O |
> 240000 ++-----------------------------------------------------------------+
>
>
> turbostat.Avg_MHz
>
> 650 ++--------------------------------------------------------------------+
> | |
> 600 O+ OO O O O O |
> | O O OO O O OO OO O |
> 550 ++ O O O |
> | |
> 500 ++ |
> | |
> 450 ++ |
> | |
> 400 ++ |
> | *.*.* .**. .**.*. .**. .* .**. *.|
> 350 *+**.*.**.**.*. *.*.* .* *.* **.* ** *.**.* * *.* *
> | * * |
> 300 ++--------------------------------------------------------------------+
>
>
> turbostat._Busy
>
> 20 ++---------------------------------------------------------------------+
> 19 O+ O O O O |
> | O O OO O O O O O OO |
> 18 ++OO O O O |
> 17 ++ |
> | |
> 16 ++ |
> 15 ++ |
> 14 ++ |
> | |
> 13 ++ |
> 12 ++ .* .**.*.* .*.**. *.*.**. .**.* .*.**.*.**.*.**.|
> *.**.*.**.* *. .**.* .* * *.* * * *
> 11 ++ * * |
> 10 ++---------------------------------------------------------------------+
>
>
> turbostat.CPU_c1
>
> 23 ++---------------------------------------------------------------------+
> | * * *. |
> 22 ++ * +: : *. .* *. : *. * * *. |
> | *. *. .* : * : : * :+ * .*.: * + + *. *.*.* .*.* *.**.*
> 21 *+* *.* * : : *.* * * * * * * |
> | *.* |
> 20 ++ |
> | |
> 19 ++ |
> | |
> 18 ++ O OO O O O O O |
> | OO O O O O O O O |
> 17 O+ O O O |
> | |
> 16 ++---------------------------------------------------------------------+
>
>
> turbostat.CPU_c3
>
> 38 ++---------------------------------------------------------------------+
> | O |
> 36 ++OO O O O O |
> 34 ++ O O O O O OO O |
> O O O |
> 32 ++ O O O |
> | |
> 30 ++ |
> | |
> 28 ++ * |
> 26 ++ : : *.* *.* * .*
> *. *. .* .*. : : *. : : *. .**. + : *. *.*. :+ .* .*. *. * |
> 24 ++* * * ** * * : : * *.** :+ * **.* * * * *.* |
> | * * |
> 22 ++---------------------------------------------------------------------+
>
>
> turbostat.CPU_c6
>
> 44 ++---*-------*---------------------------------------------------------+
> |.* + + *.*.* + .**. *. *. .** * .*. *.* *. .*.* |
> 42 *+ * * * **.*.* + * * + .**. + * * :.*.* *.** *.|
> 40 ++ * * * * *
> | |
> 38 ++ |
> 36 ++ |
> | |
> 34 ++ |
> 32 ++ |
> O O O |
> 30 ++ O O O OO O O O |
> 28 ++OO O O O O O O |
> | O O |
> 26 ++---------------------------------------------------------------------+
>
>
> vmstat.procs.b
>
> 30 ++---------------------------------------------------------------------+
> | |
> | O |
> 25 O+O O O O OO O O O O O |
> | O O O O O O O O |
> | |
> 20 ++ |
> | |
> 15 ++ |
> | |
> | |
> 10 ++* *.* .* .* *.*. *. .* .**.*. * |
> |+ :.*. .* .* *.* * + *. .* * * :.*. * **. .**. + :.*
> * * **.* :.* * * * * * * * |
> 5 ++-----------*---------------------------------------------------------+
>
> [*] bisect-good sample
> [O] bisect-bad sample
>
> To reproduce:
>
> git clone git://git.kernel.org/pub/scm/linux/kernel/git/wfg/lkp-tests.git
> cd lkp-tests
> bin/lkp install job.yaml # job file is attached in this email
> bin/lkp run job.yaml
>
>
> Disclaimer:
> Results have been estimated based on internal Intel analysis and are provided
> for informational purposes only. Any difference in system hardware or software
> design or configuration may affect actual performance.
>
>
> Thanks,
> Ying Huang
> <job.yaml><reproduce.sh>