performance changes on 4400755e: 200.0% fsmark.files_per_sec, -18.1% fsmark.files_per_sec, and few more
From: Yuanahn Liu
Date: Wed Mar 18 2015 - 00:59:27 EST
Hi,
FYI, we noticed performance changes on `fsmark.files_per_sec' by 4400755e356f9a2b0b7ceaa02f57b1c7546c3765:
> commit 4400755e356f9a2b0b7ceaa02f57b1c7546c3765
> Author: NeilBrown <neilb@xxxxxxx>
> AuthorDate: Thu Feb 26 12:47:56 2015 +1100
> Commit: NeilBrown <neilb@xxxxxxx>
> CommitDate: Wed Mar 4 13:40:19 2015 +1100
>
> md/raid5: allow the stripe_cache to grow and shrink.
26089f4902595a2f64c512066af07af6e82eb096 4400755e356f9a2b0b7ceaa02f57b1c7546c3765
---------------------------------------- ----------------------------------------
run time(m) metric_value ±stddev run time(m) metric_value ±stddev change testbox/benchmark/sub-testcase
--- ------ ---------------------------- --- ------ ---------------------------- -------- ------------------------------
3 18.6 6.400 ±0.0% 5 9.2 19.200 ±0.0% 200.0% ivb44/fsmark/1x-64t-3HDD-RAID5-ext4-4M-40G-fsyncBeforeClose
3 24.7 6.400 ±0.0% 3 13.7 12.800 ±0.0% 100.0% ivb44/fsmark/1x-64t-3HDD-RAID5-f2fs-4M-40G-fsyncBeforeClose
3 17.5 28.267 ±9.6% 3 12.3 42.833 ±6.5% 51.5% ivb44/fsmark/1x-64t-3HDD-RAID5-f2fs-4M-120G-NoSync
3 16.7 30.700 ±1.5% 3 12.6 40.733 ±2.4% 32.7% ivb44/fsmark/1x-1t-3HDD-RAID5-f2fs-4M-120G-NoSync
3 29.0 5.867 ±0.8% 5 23.6 7.240 ±0.7% 23.4% ivb44/fsmark/1x-1t-3HDD-RAID5-btrfs-4M-40G-fsyncBeforeClose
3 28.5 6.000 ±0.0% 3 23.2 7.367 ±0.6% 22.8% ivb44/fsmark/1x-1t-3HDD-RAID5-f2fs-4M-40G-fsyncBeforeClose
5 11.7 14.600 ±0.0% 5 9.7 17.500 ±0.4% 19.9% ivb44/fsmark/1x-1t-3HDD-RAID5-xfs-4M-40G-fsyncBeforeClose
3 22.4 25.600 ±0.0% 5 17.9 30.120 ±4.1% 17.7% ivb44/fsmark/1x-64t-3HDD-RAID5-xfs-4M-120G-NoSync
5 10.8 47.320 ±0.6% 5 9.3 54.820 ±0.2% 15.8% ivb44/fsmark/1x-1t-3HDD-RAID5-xfs-4M-120G-NoSync
1 0.5 252.400 ±0.0% 1 0.5 263.300 ±0.0% 4.3% ivb44/fsmark/1x-1t-4BRD_12G-RAID5-ext4-4M-30G-NoSync
3 0.5 273.100 ±4.3% 3 0.6 223.567 ±6.5% -18.1% ivb44/fsmark/1x-64t-4BRD_12G-RAID5-btrfs-4M-30G-NoSync
3 8.1 63.133 ±0.5% 3 9.2 55.633 ±0.2% -11.9% ivb44/fsmark/1x-1t-3HDD-RAID5-btrfs-4M-120G-NoSync
3 8.2 64.000 ±0.0% 3 9.2 57.600 ±0.0% -10.0% ivb44/fsmark/1x-64t-3HDD-RAID5-btrfs-4M-120G-NoSync
NOTE: here are some more info about those test parameters for you to
understand the testcase better:
1x: where 'x' means iterations or loop, corresponding to the 'L' option of fsmark
1t, 64t: where 't' means thread
4M: means the single file size, corresponding to the '-s' option of fsmark
40G, 30G, 120G: means the total test size
4BRD_12G: BRD is the ramdisk, where '4' means 4 ramdisk, and where '12G' means
the size of one ramdisk. So, it would be 48G in total. And we made a
raid on those ramdisk.
As you can see from above data, interestingly, all performance
regressions come from btrfs testing. That's why Chris is also
in the cc list, with which just FYI.
FYI, here I listed more detailed changes for the maximal postive and negtive changes.
more detailed changes about ivb44/fsmark/1x-64t-3HDD-RAID5-ext4-4M-40G-fsyncBeforeClose
---------
26089f4902595a2f 4400755e356f9a2b0b7ceaa02f
---------------- --------------------------
%stddev %change %stddev
\ | \
6.40 ± 0% +200.0% 19.20 ± 0% fsmark.files_per_sec
1.015e+08 ± 1% -73.6% 26767355 ± 3% fsmark.time.voluntary_context_switches
13793 ± 1% -73.9% 3603 ± 5% fsmark.time.system_time
78473 ± 6% -64.3% 28016 ± 7% fsmark.time.involuntary_context_switches
15789555 ± 9% -54.7% 7159485 ± 13% fsmark.app_overhead
1115 ± 0% -50.3% 554 ± 1% fsmark.time.elapsed_time.max
1115 ± 0% -50.3% 554 ± 1% fsmark.time.elapsed_time
1235 ± 2% -47.5% 649 ± 3% fsmark.time.percent_of_cpu_this_job_got
456465 ± 1% -26.7% 334594 ± 4% fsmark.time.minor_page_faults
275 ± 0% +1257.7% 3733 ± 2% slabinfo.raid5-md0.num_objs
275 ± 0% +1257.7% 3733 ± 2% slabinfo.raid5-md0.active_objs
11 ± 0% +1250.9% 148 ± 2% slabinfo.raid5-md0.active_slabs
11 ± 0% +1250.9% 148 ± 2% slabinfo.raid5-md0.num_slabs
2407 ± 4% +293.4% 9471 ± 26% numa-meminfo.node0.Writeback
600 ± 4% +294.9% 2372 ± 26% numa-vmstat.node0.nr_writeback
1114505 ± 0% -77.4% 251696 ± 2% softirqs.TASKLET
1808027 ± 1% -77.7% 402378 ± 4% softirqs.RCU
12158665 ± 1% -77.1% 2786069 ± 4% cpuidle.C3-IVT.usage
1119433 ± 0% -77.3% 254192 ± 2% softirqs.BLOCK
37824202 ± 1% -75.1% 9405078 ± 4% cpuidle.C6-IVT.usage
1.015e+08 ± 1% -73.6% 26767355 ± 3% time.voluntary_context_switches
13793 ± 1% -73.9% 3603 ± 5% time.system_time
5971084 ± 1% -73.6% 1574912 ± 5% softirqs.SCHED
10539492 ± 3% -72.0% 2956258 ± 6% cpuidle.C1E-IVT.usage
2 ± 0% +230.0% 6 ± 12% vmstat.procs.b
14064 ± 1% -71.2% 4049 ± 6% softirqs.HRTIMER
7388306 ± 1% -71.2% 2129929 ± 4% softirqs.TIMER
3.496e+09 ± 1% -70.3% 1.04e+09 ± 1% cpuidle.C3-IVT.time
0.88 ± 6% +224.9% 2.87 ± 11% turbostat.Pkg%pc6
19969464 ± 2% -66.2% 6750675 ± 5% cpuidle.C1-IVT.usage
78473 ± 6% -64.3% 28016 ± 7% time.involuntary_context_switches
4.23 ± 5% +181.4% 11.90 ± 3% turbostat.Pkg%pc2
2.551e+09 ± 1% -61.4% 9.837e+08 ± 3% cpuidle.C1E-IVT.time
8084 ± 3% +142.6% 19608 ± 3% meminfo.Writeback
2026 ± 4% +141.6% 4895 ± 4% proc-vmstat.nr_writeback
165 ± 4% -56.9% 71 ± 14% numa-vmstat.node1.nr_inactive_anon
7.748e+09 ± 3% -50.3% 3.852e+09 ± 3% cpuidle.C1-IVT.time
175 ± 5% -53.2% 82 ± 13% numa-vmstat.node1.nr_shmem
1115 ± 0% -50.3% 554 ± 1% time.elapsed_time.max
1115 ± 0% -50.3% 554 ± 1% time.elapsed_time
1147 ± 0% -49.0% 585 ± 1% uptime.boot
2260889 ± 0% -48.8% 1157272 ± 1% proc-vmstat.pgfree
16805 ± 2% -35.9% 10776 ± 23% numa-vmstat.node1.nr_dirty
1235 ± 2% -47.5% 649 ± 3% time.percent_of_cpu_this_job_got
67245 ± 2% -35.9% 43122 ± 23% numa-meminfo.node1.Dirty
39041 ± 0% -45.7% 21212 ± 2% uptime.idle
13 ± 9% -49.0% 6 ± 11% vmstat.procs.r
3072 ± 10% -40.3% 1833 ± 9% cpuidle.POLL.usage
3045115 ± 0% -46.1% 1642053 ± 1% proc-vmstat.pgfault
202 ± 1% -45.2% 110 ± 0% proc-vmstat.nr_inactive_anon
4583079 ± 2% -31.4% 3143602 ± 16% numa-vmstat.node1.numa_hit
28.03 ± 0% +69.1% 47.39 ± 1% turbostat.CPU%c6
223 ± 1% -41.1% 131 ± 1% proc-vmstat.nr_shmem
4518820 ± 3% -30.8% 3128304 ± 16% numa-vmstat.node1.numa_local
3363496 ± 3% -27.4% 2441619 ± 20% numa-vmstat.node1.nr_dirtied
3345346 ± 3% -27.4% 2428396 ± 20% numa-vmstat.node1.nr_written
0.18 ± 18% +105.6% 0.37 ± 36% turbostat.Pkg%pc3
3427913 ± 3% -27.3% 2492563 ± 20% numa-vmstat.node1.nr_inactive_file
13712431 ± 3% -27.3% 9971152 ± 20% numa-meminfo.node1.Inactive
13711768 ± 3% -27.3% 9970866 ± 20% numa-meminfo.node1.Inactive(file)
3444598 ± 3% -27.2% 2508920 ± 20% numa-vmstat.node1.nr_file_pages
13778510 ± 3% -27.2% 10036287 ± 20% numa-meminfo.node1.FilePages
8819175 ± 1% -28.3% 6320188 ± 19% numa-numastat.node1.numa_hit
8819051 ± 1% -28.3% 6320152 ± 19% numa-numastat.node1.local_node
14350918 ± 3% -26.8% 10504070 ± 19% numa-meminfo.node1.MemUsed
100892 ± 3% -26.0% 74623 ± 19% numa-vmstat.node1.nr_slab_reclaimable
403571 ± 3% -26.0% 298513 ± 19% numa-meminfo.node1.SReclaimable
3525 ± 13% +36.6% 4817 ± 14% slabinfo.blkdev_requests.active_objs
3552 ± 13% +36.3% 4841 ± 14% slabinfo.blkdev_requests.num_objs
30779 ± 4% -34.7% 20084 ± 12% proc-vmstat.pgmigrate_success
30779 ± 4% -34.7% 20084 ± 12% proc-vmstat.numa_pages_migrated
447400 ± 2% -23.2% 343701 ± 16% numa-meminfo.node1.Slab
2.532e+10 ± 0% -33.1% 1.694e+10 ± 1% cpuidle.C6-IVT.time
3081 ± 9% +28.0% 3945 ± 12% slabinfo.mnt_cache.num_objs
3026 ± 9% +28.8% 3898 ± 12% slabinfo.mnt_cache.active_objs
5822 ± 4% +77.8% 10350 ± 25% numa-meminfo.node1.Writeback
1454 ± 4% +77.3% 2579 ± 25% numa-vmstat.node1.nr_writeback
424984 ± 1% -26.5% 312255 ± 3% proc-vmstat.numa_pte_updates
368001 ± 1% -26.8% 269440 ± 3% proc-vmstat.numa_hint_faults
456465 ± 1% -26.7% 334594 ± 4% time.minor_page_faults
3.86 ± 3% -24.4% 2.92 ± 2% turbostat.CPU%c3
4661151 ± 2% +20.6% 5622999 ± 9% numa-vmstat.node1.nr_free_pages
18644452 ± 2% +20.6% 22491300 ± 9% numa-meminfo.node1.MemFree
876 ± 2% +28.2% 1124 ± 5% slabinfo.kmalloc-4096.num_objs
858 ± 3% +24.0% 1064 ± 5% slabinfo.kmalloc-4096.active_objs
17767832 ± 8% -25.4% 13249545 ± 17% cpuidle.POLL.time
285093 ± 1% -23.1% 219372 ± 5% proc-vmstat.numa_hint_faults_local
105423 ± 2% -16.1% 88498 ± 0% meminfo.Dirty
26365 ± 1% -16.0% 22152 ± 1% proc-vmstat.nr_dirty
41.04 ± 1% -14.1% 35.26 ± 1% turbostat.CPU%c1
9385 ± 4% -14.3% 8043 ± 6% slabinfo.kmalloc-192.active_objs
9574 ± 3% -13.9% 8241 ± 6% slabinfo.kmalloc-192.num_objs
2411 ± 3% +17.0% 2820 ± 4% slabinfo.kmalloc-2048.active_objs
12595574 ± 0% -10.0% 11338368 ± 1% proc-vmstat.pgalloc_normal
5262 ± 1% +13.3% 5962 ± 1% slabinfo.kmalloc-1024.num_objs
5262 ± 1% +12.7% 5932 ± 1% slabinfo.kmalloc-1024.active_objs
2538 ± 3% +13.7% 2885 ± 4% slabinfo.kmalloc-2048.num_objs
5299546 ± 0% -9.9% 4776351 ± 0% slabinfo.buffer_head.active_objs
5299546 ± 0% -9.9% 4776351 ± 0% slabinfo.buffer_head.num_objs
135885 ± 0% -9.9% 122470 ± 0% slabinfo.buffer_head.num_slabs
135885 ± 0% -9.9% 122470 ± 0% slabinfo.buffer_head.active_slabs
28.04 ± 2% +715.6% 228.69 ± 3% iostat.sdb.avgrq-sz
28.05 ± 2% +708.1% 226.72 ± 2% iostat.sdc.avgrq-sz
2245 ± 3% -81.6% 413 ± 1% iostat.sda.w/s
5.33 ± 1% +1008.2% 59.07 ± 1% iostat.sda.w_await
5.85 ± 1% +1126.4% 71.69 ± 4% iostat.sda.r_await
5.36 ± 1% +978.6% 57.79 ± 3% iostat.sdc.w_await
1263 ± 4% -85.8% 179 ± 6% iostat.sdc.r/s
2257 ± 3% -81.6% 414 ± 2% iostat.sdb.w/s
1264 ± 4% -85.8% 179 ± 6% iostat.sdb.r/s
5.55 ± 0% +1024.2% 62.37 ± 4% iostat.sdb.await
5.89 ± 1% +1125.9% 72.16 ± 6% iostat.sdb.r_await
5.36 ± 0% +1014.3% 59.75 ± 3% iostat.sdb.w_await
5.57 ± 1% +987.9% 60.55 ± 3% iostat.sdc.await
5.51 ± 0% +1017.3% 61.58 ± 1% iostat.sda.await
1264 ± 4% -85.8% 179 ± 6% iostat.sda.r/s
28.09 ± 2% +714.2% 228.73 ± 2% iostat.sda.avgrq-sz
5.95 ± 2% +1091.0% 70.82 ± 6% iostat.sdc.r_await
2252 ± 3% -81.5% 417 ± 2% iostat.sdc.w/s
4032 ± 2% +151.6% 10143 ± 1% iostat.sdb.wrqm/s
4043 ± 2% +151.0% 10150 ± 1% iostat.sda.wrqm/s
4035 ± 2% +151.2% 10138 ± 1% iostat.sdc.wrqm/s
26252 ± 1% -54.0% 12077 ± 4% vmstat.system.in
37813 ± 0% +101.0% 75998 ± 1% vmstat.io.bo
37789 ± 0% +101.0% 75945 ± 1% iostat.md0.wkB/s
205 ± 0% +96.1% 402 ± 1% iostat.md0.w/s
164286 ± 1% -46.2% 88345 ± 2% vmstat.system.cs
27.07 ± 2% -46.7% 14.42 ± 3% turbostat.%Busy
810 ± 2% -46.7% 431 ± 3% turbostat.Avg_MHz
15.56 ± 2% +71.7% 26.71 ± 1% iostat.sda.avgqu-sz
15.65 ± 2% +69.1% 26.46 ± 2% iostat.sdc.avgqu-sz
15.67 ± 2% +72.7% 27.06 ± 2% iostat.sdb.avgqu-sz
25151 ± 0% +68.3% 42328 ± 1% iostat.sda.wkB/s
25153 ± 0% +68.2% 42305 ± 1% iostat.sdb.wkB/s
25149 ± 0% +68.2% 42292 ± 1% iostat.sdc.wkB/s
97.45 ± 0% -21.1% 76.90 ± 0% turbostat.CorWatt
12517 ± 0% -20.2% 9994 ± 1% iostat.sdc.rkB/s
12517 ± 0% -20.0% 10007 ± 1% iostat.sda.rkB/s
12512 ± 0% -19.9% 10018 ± 1% iostat.sdb.rkB/s
1863 ± 3% +24.7% 2325 ± 1% iostat.sdb.rrqm/s
1865 ± 3% +24.3% 2319 ± 1% iostat.sdc.rrqm/s
1864 ± 3% +24.6% 2322 ± 1% iostat.sda.rrqm/s
128 ± 0% -16.4% 107 ± 0% turbostat.PkgWatt
150569 ± 0% -8.7% 137525 ± 0% iostat.md0.avgqu-sz
4.29 ± 0% -5.1% 4.07 ± 0% turbostat.RAMWatt
more detailed changes about ivb44/fsmark/1x-64t-4BRD_12G-RAID5-btrfs-4M-30G-NoSync
---------
26089f4902595a2f 4400755e356f9a2b0b7ceaa02f
---------------- --------------------------
%stddev %change %stddev
\ | \
273 ± 4% -18.1% 223 ± 6% fsmark.files_per_sec
29.24 ± 1% +27.2% 37.20 ± 8% fsmark.time.elapsed_time.max
29.24 ± 1% +27.2% 37.20 ± 8% fsmark.time.elapsed_time
399 ± 4% -20.0% 319 ± 3% fsmark.time.percent_of_cpu_this_job_got
129891 ± 20% -28.9% 92334 ± 15% fsmark.time.voluntary_context_switches
266 ± 0% +413.4% 1365 ± 5% slabinfo.raid5-md0.num_objs
266 ± 0% +413.4% 1365 ± 5% slabinfo.raid5-md0.active_objs
0.23 ± 27% +98.6% 0.46 ± 35% turbostat.CPU%c3
56612063 ± 9% +36.7% 77369763 ± 20% cpuidle.C1-IVT.time
5579498 ± 14% -36.0% 3571516 ± 6% cpuidle.C1E-IVT.time
4668 ± 38% +64.7% 7690 ± 19% numa-vmstat.node0.nr_unevictable
18674 ± 38% +64.7% 30762 ± 19% numa-meminfo.node0.Unevictable
9298 ± 37% +64.4% 15286 ± 19% proc-vmstat.nr_unevictable
4629 ± 37% +64.1% 7596 ± 19% numa-vmstat.node1.nr_unevictable
18535 ± 37% +63.9% 30385 ± 19% numa-meminfo.node1.Unevictable
4270894 ± 19% +65.6% 7070923 ± 21% cpuidle.C3-IVT.time
38457 ± 37% +59.0% 61148 ± 19% meminfo.Unevictable
3748226 ± 17% +26.6% 4743674 ± 16% numa-vmstat.node0.numa_local
4495283 ± 13% -24.8% 3382315 ± 17% numa-vmstat.node0.nr_free_pages
3818432 ± 16% +26.5% 4830938 ± 16% numa-vmstat.node0.numa_hit
17966826 ± 13% -24.7% 13537228 ± 17% numa-meminfo.node0.MemFree
14901309 ± 15% +29.7% 19330906 ± 12% numa-meminfo.node0.MemUsed
26 ± 21% -32.9% 17 ± 14% cpuidle.POLL.usage
1.183e+09 ± 1% +29.6% 1.533e+09 ± 8% cpuidle.C6-IVT.time
29.24 ± 1% +27.2% 37.20 ± 8% time.elapsed_time
29.24 ± 1% +27.2% 37.20 ± 8% time.elapsed_time.max
399 ± 4% -20.0% 319 ± 3% time.percent_of_cpu_this_job_got
850 ± 4% -8.6% 777 ± 5% slabinfo.blkdev_requests.num_objs
850 ± 4% -8.6% 777 ± 5% slabinfo.blkdev_requests.active_objs
14986 ± 9% +17.1% 17548 ± 8% numa-vmstat.node0.nr_slab_reclaimable
11943 ± 5% -12.6% 10441 ± 2% slabinfo.kmalloc-192.num_objs
59986 ± 9% +17.0% 70186 ± 8% numa-meminfo.node0.SReclaimable
3703 ± 6% +10.2% 4082 ± 7% slabinfo.btrfs_delayed_data_ref.num_objs
133551 ± 6% +16.1% 154995 ± 1% proc-vmstat.pgfault
129891 ± 20% -28.9% 92334 ± 15% time.voluntary_context_switches
11823 ± 4% -12.0% 10409 ± 3% slabinfo.kmalloc-192.active_objs
3703 ± 6% +9.7% 4061 ± 7% slabinfo.btrfs_delayed_data_ref.active_objs
19761 ± 2% -11.2% 17542 ± 6% slabinfo.anon_vma.active_objs
19761 ± 2% -11.2% 17544 ± 6% slabinfo.anon_vma.num_objs
13002 ± 3% +14.9% 14944 ± 5% slabinfo.kmalloc-256.num_objs
12695 ± 3% +13.8% 14446 ± 7% slabinfo.kmalloc-256.active_objs
1190 ± 1% -11.8% 1050 ± 3% slabinfo.mnt_cache.num_objs
1190 ± 1% -11.8% 1050 ± 3% slabinfo.mnt_cache.active_objs
136862 ± 1% -13.8% 117938 ± 7% cpuidle.C6-IVT.usage
1692630 ± 3% +12.3% 1900854 ± 0% numa-vmstat.node0.nr_written
1056 ± 2% +8.8% 1149 ± 3% slabinfo.mm_struct.active_objs
1056 ± 2% +8.8% 1149 ± 3% slabinfo.mm_struct.num_objs
24029 ± 11% -30.6% 16673 ± 8% vmstat.system.cs
8859 ± 2% -15.0% 7530 ± 8% vmstat.system.in
905630 ± 2% -16.8% 753097 ± 4% iostat.md0.wkB/s
906433 ± 2% -16.9% 753482 ± 4% vmstat.io.bo
3591 ± 2% -16.9% 2982 ± 4% iostat.md0.w/s
13.22 ± 5% -16.3% 11.07 ± 1% turbostat.%Busy
402 ± 4% -15.9% 338 ± 1% turbostat.Avg_MHz
54236 ± 3% +10.4% 59889 ± 4% iostat.md0.avgqu-sz
7.67 ± 1% +4.5% 8.01 ± 1% turbostat.RAMWatt
--yliu
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/