Re: [linux-next:master] [drivers/char/mem] 1b057bd800: stress-ng.splice.ops_per_sec -99.8% regression

[Oliver Sang]

hi, Greg Kroah-Hartman,

On Tue, Oct 17, 2023 at 06:56:56PM +0200, Greg Kroah-Hartman wrote:
> On Tue, Oct 17, 2023 at 11:06:42PM +0800, kernel test robot wrote:
> > 
> > 
> > Hello,
> > 
> > kernel test robot noticed a -99.8% regression of stress-ng.splice.ops_per_sec on:
> > 
> > 
> > commit: 1b057bd800c3ea0c926191d7950cd2365eddc9bb ("drivers/char/mem: implement splice() for /dev/zero, /dev/full")
> > https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
> > 
> > testcase: stress-ng
> > test machine: 64 threads 2 sockets Intel(R) Xeon(R) Gold 6346 CPU @ 3.10GHz (Ice Lake) with 256G memory
> > parameters:
> > 
> > 	nr_threads: 100%
> > 	testtime: 60s
> > 	class: pipe
> > 	test: splice
> > 	cpufreq_governor: performance
> > 
> > 
> > In addition to that, the commit also has significant impact on the following tests:
> > 
> > +------------------+-------------------------------------------------------------------------------------------------+
> > | testcase: change | stress-ng: stress-ng.splice.ops_per_sec 38.9% improvement                                       |
> 
> So everything now goes faster, right?  -99.8% regression means 99.8%
> faster?

let me clarify.

our auto bisect captured this commit as 'first bad commit' in two tests.

Test 1:

it found a (very big) regression comparing to parent commit.

19e3e6cdfdc73400 1b057bd800c3ea0c926191d7950 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
  12433266           -99.8%      22893 ±  3%  stress-ng.splice.ops_per_sec

the detail data for parent in multi-runs:
  "stress-ng.splice.ops_per_sec": [
    12444442.19,
    12599010.87,
    12416009.38,
    12494132.89,
    12286766.76,
    12359235.82
  ],

for 1b057bd800:
  "stress-ng.splice.ops_per_sec": [
    24055.57,
    23235.46,
    22142.13,
    23782.13,
    21732.13,
    22415.46
  ],

so this is much slower.

the config for this Test 1 is:
testcase: stress-ng
test machine: 64 threads 2 sockets Intel(R) Xeon(R) Gold 6346 CPU @ 3.10GHz (Ice Lake) with 256G memory
parameters:
	nr_threads: 100%
	testtime: 60s
	class: pipe
	test: splice
	cpufreq_governor: performance


Test 2:

this is still a stress-ng test, but the config is different with Test 1
(the bare metal machine config, and stress-ng parameters):

testcase: stress-ng
test machine: 36 threads 1 sockets Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz (Cascade Lake) with 128G memory
parameters:
	nr_threads=1
	testtime=60s
	class=os
	test=splice
	disk=1HDD
	fs=ext4
	cpufreq_governor=performance

Test 2 shows a big improvement:

19e3e6cdfdc73400 1b057bd800c3ea0c926191d7950 
---------------- --------------------------- 
         %stddev     %change         %stddev
             \          |                \  
    171798           +38.9%     238710 ±  4%  stress-ng.splice.ops_per_sec

the detail data:
for parent:
  "stress-ng.splice.ops_per_sec": [
    173056.44,
    172030.08,
    171401.68,
    171694.23,
    171001.19,
    171606.93
  ],

for 1b057bd800:
  "stress-ng.splice.ops_per_sec": [
    244347.89,
    259085.63,
    231423.88,
    232897.93,
    226714.77,
    237792.34
  ],


there are monitoring data such like perf data in original report. FYI

> 
> thanks,
> 
> greg k-h