Benchmark comparison - 1.3.12

Carlo Emilio Prelz (fluido@telepac.pt)
Wed, 26 Jul 1995 21:15:19 +0200 (MET DST)


Hi. Here are the benchmark comparison for the definitive version
1.3.12. I am comparing it only to V1.3.11 WITH Linus's patches (as
well as with 1.2.10 as usual): differences are very small.

IPX is still not working.

The WWW pages are now up and running. Try them at:

http://neon.ingenia.com/cgi-bin/bm-request

Lots of thanks to Mike Shaver!

You can request a page that is pratcically identical to what I post
here (you can select which benchmarks to compare), and a page in which
you compare many releases on one single benchmark. You can also choose
geometrical averages (they are automatically included in the log
files - no big sweat).

For the moment, only my log files are accessible. If I find the
correct inspiration in the next weekend, I am planning to add the
possibility of having more directories holding log files: each
directory will correspond to one machine and will have a small
description file for the machine.

These directories should have to reside on Mike's machine for now. I
have thought of receiving URL's of log files or log directories, but
I'd have to link (some of) the server libs for getting & opening
remote files (I think). Maybe this is easy, I don't know.

Alternatively, other sites could host the software, along with their
log files. No comparisons will be possible between machines, that
way.

My software is available for anybody who will ask for it. At some
moment I may think to make a nice package of it & upload it to
sunsite. Maybe. If you have ideas on how to make it better, they are
welcome.

I have seen a few messages that ask for more rigor and statistical
precision in extracting benchmark data. Well, I think the actual
numbers already reach the scope that I had in mind - that is, give to
the kernel developers and everyone else an idea about the effects of
what they change. My conscience is clean :-)

I may decide to add the calculation for the standard deviation, but
that requires touching a series of small shell scripts (one for each
benchmark type) in the original sources. Anyway, I think that you may
be interested once in the deviation data. Posting it regularly seems a
bit useless, and I can't see where to put the numbers (comparing
standard deviations between releases seems a bit pointless to me...)

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

From:
* Linux pimpinel.fluido.org 1.3.12pre #23 Wed Jul 19 21:55:21 MET DST 1995 i486
* Wed Jul 19 22:00:21 MET DST 1995

To:
* Linux pimpinel.fluido.org 1.3.12 #24 Tue Jul 25 21:46:19 MET DST 1995 i486
* Tue Jul 25 22:31:42 MET DST 1995

*******
Results
*******

Shell scripts (4 concurrent) || 21.2 -> 22.9 +8.02%
Pipe-based Context Switching Test || 9969.6 -> 10478.0 +5.10%
File Copy (10 seconds) || 1196.0 -> 1246.0 +4.18%
Process Creation Test || 126.3 -> 131.0 +3.72%
Execl Throughput Test || 62.8 -> 64.5 +2.71%
Shell scripts (1 concurrent) || 83.6 -> 85.8 +2.63%
Shell scripts (2 concurrent) || 44.1 -> 45.0 +2.04%
File Copy (30 seconds) || 1099.0 -> 1115.0 +1.46%
System Call Overhead Test || 30865.7 -> 31204.1 +1.10%
Dc: sqrt(2) to 99 decimal places || 5677.4 -> 5701.2 +0.42%
Dhrystone 2 using register variables || 49969.4 -> 49983.8 +0.03%
C Compiler Test || 45.5 -> 45.5 +0.00%
Shell scripts (8 concurrent) || 11.0 -> 11.0 +0.00%
Arithmetic Test (type = short) || 7503.2 -> 7502.9 -0.00%
Arithmetic Test (type = float) || 5062.4 -> 5062.1 -0.01%
Arithmetic Test (type = int) || 8363.3 -> 8361.8 -0.02%
Arithmetic Test (type = double) || 5062.7 -> 5061.7 -0.02%
Arithmetic Test (type = register) || 8363.9 -> 8361.9 -0.02%
Arithmetic Test (type = long) || 8363.7 -> 8361.3 -0.03%
Arithmetic Test (type = arithoh) || 127078.2 -> 127040.5 -0.03%
Recursion Test--Tower of Hanoi || 726.4 -> 722.6 -0.52%
File Write (10 seconds) || 9240.0 -> 9190.0 -0.54%
File Write (30 seconds) || 9136.0 -> 9030.0 -1.16%
Dhrystone 2 without register variables || 50300.9 -> 49677.6 -1.24%
Pipe Throughput Test || 18485.1 -> 18066.7 -2.26%
File Read (30 seconds) || 97056.0 -> 93339.0 -3.83%
File Read (10 seconds) || 92613.0 -> 88909.0 -4.00%

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

From:
* Linux pimpinel.fluido.org 1.2.10 #59 Tue Jun 13 09:46:17 MET DST 1995 i486
* Tue Jun 27 10:18:13 MET DST 1995

To:
* Linux pimpinel.fluido.org 1.3.12 #24 Tue Jul 25 21:46:19 MET DST 1995 i486
* Tue Jul 25 22:31:42 MET DST 1995

*******
Results
*******

Pipe-based Context Switching Test || 3446.9 -> 10478.0 +203.98%
Execl Throughput Test || 51.6 -> 64.5 +25.00%
File Write (10 seconds) || 8100.0 -> 9190.0 +13.46%
Shell scripts (8 concurrent) || 10.0 -> 11.0 +10.00%
Shell scripts (2 concurrent) || 41.0 -> 45.0 +9.76%
Shell scripts (1 concurrent) || 78.3 -> 85.8 +9.58%
Process Creation Test || 119.9 -> 131.0 +9.26%
Shell scripts (4 concurrent) || 21.0 -> 22.9 +9.05%
C Compiler Test || 42.2 -> 45.5 +7.82%
Dhrystone 2 without register variables || 46284.2 -> 49677.6 +7.33%
System Call Overhead Test || 29238.2 -> 31204.1 +6.72%
File Copy (30 seconds) || 1054.0 -> 1115.0 +5.79%
File Copy (10 seconds) || 1180.0 -> 1246.0 +5.59%
Recursion Test--Tower of Hanoi || 709.2 -> 722.6 +1.89%
File Write (30 seconds) || 8911.0 -> 9030.0 +1.34%
Dhrystone 2 using register variables || 49802.4 -> 49983.8 +0.36%
Arithmetic Test (type = float) || 5055.6 -> 5062.1 +0.13%
Arithmetic Test (type = register) || 8353.2 -> 8361.9 +0.10%
Arithmetic Test (type = int) || 8353.4 -> 8361.8 +0.10%
Arithmetic Test (type = arithoh) || 126914.4 -> 127040.5 +0.10%
Arithmetic Test (type = double) || 5056.8 -> 5061.7 +0.10%
Arithmetic Test (type = long) || 8353.5 -> 8361.3 +0.09%
Arithmetic Test (type = short) || 7496.0 -> 7502.9 +0.09%
File Read (30 seconds) || 93460.0 -> 93339.0 -0.13%
Pipe Throughput Test || 18512.0 -> 18066.7 -2.41%
Dc: sqrt(2) to 99 decimal places || 5906.2 -> 5701.2 -3.47%
File Read (10 seconds) || 92743.0 -> 88909.0 -4.13%

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

Enjoy!
Carlo

--
  *                                          ...Ma appena fuori tutto e' gomma,
* K * Carlo E. Prelz - fluido@telepac.pt      tutto e' cicca impiastricciata...
  *                                     (Marco Zappa-Niente cicca nella scuola)