Re: [linus:master] [mm] cacded5e42: aim9.brk_test.ops_per_sec -5.0% regression

[Oliver Sang]

hi, Lorenzo,

On Tue, Oct 08, 2024 at 09:44:24AM +0100, Lorenzo Stoakes wrote:
> On Tue, Oct 08, 2024 at 04:31:59PM +0800, Oliver Sang wrote:
> > hi, Lorenzo Stoakes,
> >
> > sorry for late, we are in holidays last week.
> >
> > On Mon, Sep 30, 2024 at 09:21:52AM +0100, Lorenzo Stoakes wrote:
> > > On Mon, Sep 30, 2024 at 10:21:27AM GMT, kernel test robot wrote:
> > > >
> > > >
> > > > Hello,
> > > >
> > > > kernel test robot noticed a -5.0% regression of aim9.brk_test.ops_per_sec on:
> > > >
> > > >
> > > > commit: cacded5e42b9609b07b22d80c10f0076d439f7d1 ("mm: avoid using vma_merge() for new VMAs")
> > > > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
> > > >
> > > > testcase: aim9
> > > > test machine: 48 threads 2 sockets Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz (Ivy Bridge-EP) with 64G memory
> > >
> > > Hm, quite an old microarchitecture no?
> > >
> > > Would it be possible to try this on a range of uarch's, especially more
> > > recent noes, with some repeated runs to rule out statistical noise? Much
> > > appreciated!
> >
> > we run this test on below platforms, and observed similar regression.
> > one thing I want to mention is for performance tests, we run one commit at least
> > 6 times. for this aim9 test, the data is quite stable, so there is no %stddev
> > value in our table. we won't show this value if it's <2%
> 
> Thanks, though I do suggest going forward it's worth adding the number even
> if it's <2% or highlighting that, I found that quite misleading.
> 
> Also might I suggest reporting the most recent uarch first? As this seeming
> to be ivy bridge only delayed my responding to this 

we have 80+ testsuite but a reletively small machine pool (due to resource
constraint), the recent uarch machines are used mostly for more popular
testsuites or those easy for us to catch regression per our experience.

unfortunately, the aim9 is only allot to Ivy Bridge as regular tests now.
the data on other platforms I shared with you in last thread are from manual
runs. sorry if this causes any inconvenience.

> (not to sound
> ungrateful for the report, which is very useful, but it'd be great if you
> guys could test in -next, as this was there for weeks with no apparent
> issues).

we don't test a single tree, instead, we merged a lot of trees together to
so-called hourly kernel and test upon it. mainline is stable and is our merge
base for lots of hourly kernels, so it has big chance to be tested and bisect
successfully. -next could also be the merge base some time, but since it's
rebased frequently, hard for us to finish test and bisect in time, some time
we even cannot use it as merge base since various issues. it's really a pity
that we miss issues on -next ...

> 
> I will look into this now, if I provide patches would you be able to test
> them using the same boxes? It'd be much appreciated!

sure! that's our pleasure!

> 
> Thanks, Lorenzo
>