Re: [lkp] [x86/platform/UV] 71854cb812: will-it-scale.per_thread_ops -2.3% regression

[Ye Xiaolong]

On 10/25, Thomas Gleixner wrote:
>On Tue, 25 Oct 2016, kernel test robot wrote:
>> FYI, we noticed a -2.3% regression of will-it-scale.per_thread_ops due to commit:
>> 
>> commit 71854cb812ec23bfe5f63d52217e6b9e6cb901f5 ("x86/platform/UV: Fix support for EFI_OLD_MEMMAP after BIOS callback updates")
>> https://github.com/0day-ci/linux Alex-Thorlton/x86-platform-UV-Fix-support-for-EFI_OLD_MEMMAP-after-BIOS-callback-updates/20161020-095215
>> 
>> in testcase: will-it-scale
>> on test machine: 12 threads Intel(R) Core(TM) i7 CPU X 980 @ 3.33GHz with 6G memory
>
>This is completely bogus. That patch does not even affect anything outside
>of the SGI UV platform. And on your i7 system uv_bios_call() is definitely
>not invoked.

Yes, this is weird, the per_thread_ops change is small and should be run
to run variation, the actual significant change is will-it-scale.time.user_time
-27% decrease, but the patch seems not relevant, we can't interpret it. :(

We've tried to queue the jobs (4 times) for 71854cb812ec23bfe5f63d52217e6b9e6cb901f5 and v4.9-rc1
with new kconfig (added CONFIG_DEBUG_INFO_REDUCED), and result shows
user_time change is quite stable.


=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/tbox_group/test/testcase:
  gcc-6/performance/x86_64-rhel-7.2+CONFIG_DEBUG_INFO_REDUCED/debian-x86_64-2016-08-31.cgz/wsm/read2/will-it-scale

commit:
  v4.9-rc1
  71854cb812ec23bfe5f63d52217e6b9e6cb901f5

        v4.9-rc1 71854cb812ec23bfe5f63d5221
---------------- -------------------------- 
         %stddev     %change         %stddev
             \          |                \
   1670068 ±  0%      -3.8%    1606650 ±  1%  will-it-scale.per_thread_ops
      9749 ±  2%   +1328.0%     139222 ±105%  will-it-scale.time.involuntary_context_switches
    981.29 ±  0%      +2.2%       1002 ±  0%  will-it-scale.time.system_time
     81.78 ±  0%     -26.9%      59.74 ±  0%  will-it-scale.time.user_time
     32894 ±  0%      -3.1%      31863 ±  2%  vmstat.system.cs
      9749 ±  2%   +1328.0%     139222 ±105%  time.involuntary_context_switches
    380917 ±  2%     -10.2%     341970 ±  3%  sched_debug.cpu.avg_idle.avg
     89166 ± 33%     -73.4%      23731 ± 29%  sched_debug.cpu.avg_idle.min
     16.38 ± 10%     -32.3%      11.08 ± 18%  sched_debug.cpu.nr_uninterruptible.max
      0.29 ±  1%     +32.6%       0.38 ±  1%  perf-stat.branch-miss-rate%
 2.897e+09 ±  1%     +33.5%  3.867e+09 ±  2%  perf-stat.branch-misses
  10084878 ±  0%      -3.2%    9761852 ±  2%  perf-stat.context-switches
      0.00 ±  7%      -9.3%       0.00 ±  1%  perf-stat.dTLB-store-miss-rate%
  33489012 ±  7%      -9.2%   30416429 ±  1%  perf-stat.dTLB-store-misses

Thanks,
Xiaolong
>
>I appreciate your effort, but posting such obviously bogus results does not
>make people more confident in your testing efforts.
>
>Thanks,
>
>	tglx