Re: Kernel 4.7rc3 - Performance drop 30-40% for SPECjbb2005 and SPECjvm2008 benchmarks against 4.6 kernel

[Jirka Hladky]

Hi Peter,

please find the reproducer script attached. My command to reproduce the bug is:

./run-specjvm.sh --benchmarkThreads 32 --iterations 1 --iterationTime
180 --warmuptime 90 xml.transform xml.validation

I run just xml benchmarks to speed up the runtime.

Please check
 https://bugzilla.kernel.org/show_bug.cgi?id=120481#c9
for some details how to run the benchmark.

The benchmark needs Window manager to be installed to create graphs.
However, you can run the script from ssh terminal. I don't know
exactly why is that but I know that Python's matplot library has the
same requirements.

last known good commit: e7904a28f5331c21d17af638cb477c83662e3cb6
first known bad commit: 6ecdd74962f246dfe8750b7bea481a1c0816315d

Last two commits to be checked:

 git log --pretty=oneline
e7904a28f5331c21d17af638cb477c83662e3cb6..6ecdd74962f246dfe8750b7bea481a1c0816315d
6ecdd74962f246dfe8750b7bea481a1c0816315d sched/fair: Generalize the
load/util averages resolution definition
2159197d66770ec01f75c93fb11dc66df81fd45b sched/core: Enable increased
load resolution on 64-bit kernels

I use following command to review the results produced by reproduce.sh script.

find ./ -name "*log" | xargs grep -H Score | grep xml.validation |
grep "[0-9]\{4\}[.][0-9]\{2\} ops/m"

Jirka

On Wed, Jun 22, 2016 at 9:16 AM, Peter Zijlstra <peterz@xxxxxxxxxxxxx> wrote:
> On Fri, Jun 17, 2016 at 01:04:23AM +0200, Jirka Hladky wrote:
>> > > we see performance drop 30-40% for SPECjbb2005 and SPECjvm2008
>> > Blergh, of course I don't have those.. :/
>>
>> SPECjvm2008 is publicly available.
>> https://www.spec.org/download.html
>
> Urgh, I _so_ hate java.
>
> Why does it have to pop up windows split between my screens, total fail.
>
> In any case, I run it like:
>
>    java -jar SPECjvm2008.jar --benchmarkThreads 40
>
> because I have 40 cpus (2 sockets * 10 cores/socket * 2 threads/core).
>
> It seems to produce numbers, but then ends with a splat:
>
> Error while creating report: Assistive Technology not found: org.GNOME.Accessibility.AtkWrapper
> java.awt.AWTError: Assistive Technology not found: org.GNOME.Accessibility.AtkWrapper
>         at java.awt.Toolkit.loadAssistiveTechnologies(Toolkit.java:807)
>         at java.awt.Toolkit.getDefaultToolkit(Toolkit.java:886)
>         at sun.swing.SwingUtilities2.getSystemMnemonicKeyMask(SwingUtilities2.java:2020)
>         at javax.swing.plaf.basic.BasicLookAndFeel.initComponentDefaults(BasicLookAndFeel.java:1158)
>         at javax.swing.plaf.metal.MetalLookAndFeel.initComponentDefaults(MetalLookAndFeel.java:431)
>         at javax.swing.plaf.basic.BasicLookAndFeel.getDefaults(BasicLookAndFeel.java:148)
>         at javax.swing.plaf.metal.MetalLookAndFeel.getDefaults(MetalLookAndFeel.java:1577)
>         at javax.swing.UIManager.setLookAndFeel(UIManager.java:539)
>         at javax.swing.UIManager.setLookAndFeel(UIManager.java:579)
>         at javax.swing.UIManager.initializeDefaultLAF(UIManager.java:1349)
>         at javax.swing.UIManager.initialize(UIManager.java:1459)
>         at javax.swing.UIManager.maybeInitialize(UIManager.java:1426)
>         at javax.swing.UIManager.getDefaults(UIManager.java:659)
>         at javax.swing.UIManager.getColor(UIManager.java:701)
>         at org.jfree.chart.JFreeChart.<clinit>(JFreeChart.java:246)
>         at org.jfree.chart.ChartFactory.createXYLineChart(ChartFactory.java:1478)
>         at spec.reporter.BenchmarkChart.<init>(BenchmarkChart.java:47)
>         at spec.reporter.ReportGenerator.handleBenchmarkResult(ReportGenerator.java:141)
>         at spec.reporter.ReportGenerator.handleBenchmarksResults(ReportGenerator.java:105)
>         at spec.reporter.ReportGenerator.<init>(ReportGenerator.java:87)
>         at spec.reporter.ReportGenerator.main2(ReportGenerator.java:750)
>         at spec.reporter.Reporter.main2(Reporter.java:51)
>         at spec.harness.Launch.createReport(Launch.java:307)
>         at spec.harness.Launch.runBenchmarkSuite(Launch.java:250)
>         at spec.harness.Launch.main(Launch.java:452)
>
> WTF a benchmark needs that crap is beyond me, but whatever, I have
> numbers.
>
> I'll try and reproduce.

Attachment: reproduce.sh
Description: Bourne shell script