Benchmarks - 1.3.35

Carlo Emilio Prelz (fluido@marktest.pt)
Tue, 17 Oct 1995 22:19:14 +0100 (MET)


I'm back with my benchmark reports for Linux releases. Here follow the
comparison of Byte Unix benchmark results between the latest release
and both the previous release and a reference 1.2 release. You can
obtain benchmark comparisons between releases 1.2.10, 1.3.3-1.3.20 and
1.3.30-1.3.35 on the kernel comparison web page,

http://neon.ingenia.com/cgi-bin/bm

kindly hosted by Mike Shaver. Select my machine, Pimpinel. Anybody who
ran Byte benchmarks on their machines can send me the generated log
files (you find them after the benchmark run under .../results, a file
named log). If you haven't already done so, remember to patch the
fstime.c source as follows:

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--
--- fstime.old
+++ fstime.c
@@ -146,7 +146,7 @@
alarm(seconds);
while(!sigalarm)
{
- if (read(f, buf, BUFF_SIZE) < 0) /* read while checking for an error */
+ if (read(f, buf, BUFF_SIZE) != BUFF_SIZE) /* read while checking for an error */
switch(errno)
{
case 0:
@@ -189,7 +189,7 @@
alarm(seconds);
while(!sigalarm)
{
- if (read(f, buf, BUFF_SIZE) < 0)
+ if (read(f, buf, BUFF_SIZE) != BUFF_SIZE)
switch(errno)
{
case 0:
@@ -226,6 +226,7 @@
{
extern int sigalarm;
sigalarm = 1;
+alarm(0);
return(0);
}
--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

The current software on neon allows the subdivision of log files into
separate directories, each with a small index file that specifies
characteristics for the included benchmark results. So, if you want
YOUR log files to be comparable, you can send them to me by email, at
fluido@marktest.pt. With your first logfile you must include a
section similar to the following:

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

NAME#Pimpinel
EMAIL#fluido@marktest.pt
CPU#Intel 486
MHZ#66
MB#20
HDC#ISA
GCC#2.6.3
AOUT#Yes

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

The above fields are the ones that I implemented in the software, but
you can think of further ones, I will add them. Just specify the field
name and a short descriptive phrase that will appear on the web
page. The existing fields are self-evident, aren't they?

NAME and EMAIL are required, the rest is facultative. I think these
info are important because cross-machine comparison is sort-of useless
if you have no hardware references.

For the moment, I will be processing the above by hand, but I am
thinking of a way of automatically updating the archives by email.

I decided not to go on experimenting new kernels on our internet
server. Better to wait for the next stable release. I am SOOO
wise... Anyway, I will go on testing new releases here on my home
machine. I have two things to comment about the behavior of the latest
release here. They both deal with PPP.

1) I had left on 1.3.35 on the server machine for one night, and the
new pppd server would not conclude the handshaking phase correctly. I
am using the new pppd here as a client, to connect via modem to the
server. I have no messages from the server side, but this is what I
get from the client's log file:

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

... ... ...
Oct 17 07:01:57 pimpinel chat[1323]: send (??????)
Oct 17 07:01:57 pimpinel pppd[1317]: Connected...
Oct 17 07:01:58 pimpinel pppd[1317]: Using interface ppp0
Oct 17 07:01:58 pimpinel pppd[1317]: Connect: ppp0 <--> /dev/cua2
Oct 17 07:01:58 pimpinel pppd[1317]: set_xaccm: 00000000 00000000 00000000 60000000
Oct 17 07:01:58 pimpinel pppd[1317]: send_config: mtu = 1500
Oct 17 07:01:58 pimpinel pppd[1317]: send_config: asyncmap = ffffffff
Oct 17 07:01:58 pimpinel pppd[1317]: send_config: flags = 0
Oct 17 07:01:58 pimpinel pppd[1317]: recv_config: mru = 1500
Oct 17 07:01:58 pimpinel pppd[1317]: recv_config: asyncmap = 0
Oct 17 07:01:58 pimpinel pppd[1317]: recv_config: flags = 10
Oct 17 07:01:58 pimpinel pppd[1317]: sent [LCP ConfReq id=0x1 <mru 1500> <asyncmap 0x0> <magic 0x248d8511> <pcomp> <accomp>]
Oct 17 07:01:58 pimpinel pppd[1317]: fsm_sdata(LCP): Sent code 1, id 1.
Oct 17 07:01:58 pimpinel pppd[1317]: Timeout 2194:10920 in 3 seconds.
Oct 17 07:01:58 pimpinel pppd[1317]: Setting itimer for 3 seconds in timeout.
Oct 17 07:01:58 pimpinel pppd[1317]: LCP: sending Configure-Request, id 1
Oct 17 07:02:01 pimpinel pppd[1317]: Alarm
Oct 17 07:02:01 pimpinel pppd[1317]: sent [LCP ConfReq id=0x1 <mru 1500> <asyncmap 0x0> <magic 0x248d8511> <pcomp> <accomp>]
Oct 17 07:02:01 pimpinel pppd[1317]: fsm_sdata(LCP): Sent code 1, id 1.
Oct 17 07:02:01 pimpinel pppd[1317]: Timeout 2194:10920 in 3 seconds.
Oct 17 07:02:01 pimpinel pppd[1317]: Setting itimer for 3 seconds in timeout.
Oct 17 07:02:01 pimpinel pppd[1317]: LCP: sending Configure-Request, id 1
Oct 17 07:02:02 pimpinel pppd[1317]: Hangup (SIGHUP)
Oct 17 07:02:02 pimpinel pppd[1317]: Untimeout 2194:10920.
Oct 17 07:02:02 pimpinel pppd[1317]: Setting itimer for 0 seconds in untimeout.
Oct 17 07:02:02 pimpinel pppd[1317]: ioctl(PPPIOCGDEBUG): I/O error
Oct 17 07:02:02 pimpinel pppd[1317]: ioctl(TIOCSETD): I/O error
Oct 17 07:02:02 pimpinel pppd[1317]: ioctl(TIOCNXCL): I/O error
Oct 17 07:02:02 pimpinel pppd[1317]: tcsetattr: I/O error
Oct 17 07:02:02 pimpinel pppd[1317]: Exit.

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

With the same setup I called successfully another PPP server. After
reverting to 1.2.13 and the old pppd, I was able to connect again.

2) As soon as the new PPP software was introduced, I started having
problems in making outgoing calls with chat on the same line that is
monitored by mgetty for incoming calls. With the old software, mgetty
was leaving the line alone for the time that it was locked by
chat/pppd. Now, chat starts talking with the line, but mgetty is
immediately restarted and tries to reinitialize the line. This seems
not to be related with either mgetty (unchanged) or chat itself (I
tried with both the old and the new version OF CHAT, and the problem
was evident only after 1.3.32) Maybe the problem is with line locking
in pppd? Or has something been changed in the kernel to upset serial
line locking? For the moment, I have to disable mgetty from inittab
when using pppd.

I think this would be of interest to a (probably existing) ppp mailing
list. Is there anybody who is on that list and can forward this
message there (or better, have these problems been found and solved
already there)? Thanks.

After all this, here are the comparisons! I am also including a
comparison with 1.3.20, the last release that I tested before my trip
away from Lisbon.

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

From:
* Linux pimpinel.fluido.org 1.3.34 #50 Fri Oct 13 22:50:54 MET 1995 i486
* Fri Oct 13 23:37:35 MET 1995

To:
* Linux pimpinel.fluido.org 1.3.35 #51 Tue Oct 17 04:35:07 MET 1995 i486
* Tue Oct 17 08:01:52 MET 1995

*******
Results
*******

Process Creation Test || 344.5 -> 359.2 +4.27%
Pipe Throughput Test || 17006.7 -> 17685.8 +3.99%
File Read (30 seconds) || 4386.0 -> 4515.0 +2.94%
System Call Overhead Test || 29641.4 -> 30465.6 +2.78%
File Read (10 seconds) || 4398.0 -> 4505.0 +2.43%
Dhrystone 2 using register variables || 49803.4 -> 49814.9 +0.02%
Arithmetic Test (type = short) || 7509.4 -> 7509.4 +0.00%
File Write (10 seconds) || 1400.0 -> 1400.0 +0.00%
Shell scripts (4 concurrent) || 25.0 -> 25.0 +0.00%
Shell scripts (8 concurrent) || 12.0 -> 12.0 +0.00%
Arithmetic Test (type = float) || 5066.7 -> 5066.3 -0.01%
Arithmetic Test (type = arithoh) || 127164.2 -> 127151.3 -0.01%
Arithmetic Test (type = double) || 5066.8 -> 5066.2 -0.01%
Recursion Test--Tower of Hanoi || 727.1 -> 727.0 -0.01%
Arithmetic Test (type = register) || 8370.1 -> 8368.9 -0.01%
Arithmetic Test (type = int) || 8370.0 -> 8368.8 -0.01%
Arithmetic Test (type = long) || 8368.6 -> 8366.4 -0.03%
Shell scripts (1 concurrent) || 93.0 -> 92.9 -0.11%
Shell scripts (2 concurrent) || 49.1 -> 49.0 -0.20%
Pipe-based Context Switching Test || 9520.3 -> 9484.5 -0.38%
File Copy (30 seconds) || 969.0 -> 965.0 -0.41%
C Compiler Test || 47.4 -> 47.1 -0.63%
Execl Throughput Test || 65.6 -> 64.8 -1.22%
File Copy (10 seconds) || 1020.0 -> 998.0 -2.16%
Dhrystone 2 without register variables || 49733.2 -> 48645.6 -2.19%
Dc: sqrt(2) to 99 decimal places || 11460.0 -> 11123.8 -2.93%
File Write (30 seconds) || 1200.0 -> 1146.0 -4.50%

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

From:
* Linux pimpinel.fluido.org 1.3.20 #40 Fri Aug 18 21:39:56 MET DST 1995 i486
* Fri Aug 18 22:15:40 MET DST 1995

To:
* Linux pimpinel.fluido.org 1.3.35 #51 Tue Oct 17 04:35:07 MET 1995 i486
* Tue Oct 17 08:01:52 MET 1995

*******
Results
*******

Process Creation Test || 119.3 -> 359.2 +201.09%
Dc: sqrt(2) to 99 decimal places || 5565.3 -> 11123.8 +99.88%
Shell scripts (1 concurrent) || 83.9 -> 92.9 +10.73%
Shell scripts (2 concurrent) || 44.9 -> 49.0 +9.13%
Shell scripts (8 concurrent) || 11.0 -> 12.0 +9.09%
Shell scripts (4 concurrent) || 23.0 -> 25.0 +8.70%
Pipe-based Context Switching Test || 8769.8 -> 9484.5 +8.15%
File Read (30 seconds) || 4262.0 -> 4515.0 +5.94%
Pipe Throughput Test || 16976.0 -> 17685.8 +4.18%
File Read (10 seconds) || 4342.0 -> 4505.0 +3.75%
Execl Throughput Test || 62.6 -> 64.8 +3.51%
System Call Overhead Test || 29677.7 -> 30465.6 +2.65%
C Compiler Test || 46.2 -> 47.1 +1.95%
Recursion Test--Tower of Hanoi || 713.8 -> 727.0 +1.85%
Dhrystone 2 using register variables || 49562.7 -> 49814.9 +0.51%
Arithmetic Test (type = short) || 7504.3 -> 7509.4 +0.07%
Arithmetic Test (type = float) || 5063.2 -> 5066.3 +0.06%
Arithmetic Test (type = int) || 8364.1 -> 8368.8 +0.06%
Arithmetic Test (type = arithoh) || 127081.3 -> 127151.3 +0.06%
Arithmetic Test (type = double) || 5063.7 -> 5066.2 +0.05%
Arithmetic Test (type = long) || 8362.4 -> 8366.4 +0.05%
Arithmetic Test (type = register) || 8365.6 -> 8368.9 +0.04%
File Write (10 seconds) || 1400.0 -> 1400.0 +0.00%
Dhrystone 2 without register variables || 49819.5 -> 48645.6 -2.36%
File Copy (30 seconds) || 1001.0 -> 965.0 -3.60%
File Write (30 seconds) || 1203.0 -> 1146.0 -4.74%
File Copy (10 seconds) || 1053.0 -> 998.0 -5.22%

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

From:
* Linux pimpinel.fluido.org 1.2.10 #59 Tue Jun 13 09:46:17 MET DST 1995 i486
* Tue Jun 27 10:18:13 MET DST 1995

To:
* Linux pimpinel.fluido.org 1.3.35 #51 Tue Oct 17 04:35:07 MET 1995 i486
* Tue Oct 17 08:01:52 MET 1995

*******
Results
*******

Process Creation Test || 119.9 -> 359.2 +199.58%
Pipe-based Context Switching Test || 3446.9 -> 9484.5 +175.16%
Dc: sqrt(2) to 99 decimal places || 5906.2 -> 11123.8 +88.34%
Execl Throughput Test || 51.6 -> 64.8 +25.58%
Shell scripts (8 concurrent) || 10.0 -> 12.0 +20.00%
Shell scripts (2 concurrent) || 41.0 -> 49.0 +19.51%
Shell scripts (4 concurrent) || 21.0 -> 25.0 +19.05%
Shell scripts (1 concurrent) || 78.3 -> 92.9 +18.65%
C Compiler Test || 42.2 -> 47.1 +11.61%
Dhrystone 2 without register variables || 46284.2 -> 48645.6 +5.10%
System Call Overhead Test || 29238.2 -> 30465.6 +4.20%
Recursion Test--Tower of Hanoi || 709.2 -> 727.0 +2.51%
Arithmetic Test (type = float) || 5055.6 -> 5066.3 +0.21%
Arithmetic Test (type = register) || 8353.2 -> 8368.9 +0.19%
Arithmetic Test (type = arithoh) || 126914.4 -> 127151.3 +0.19%
Arithmetic Test (type = double) || 5056.8 -> 5066.2 +0.19%
Arithmetic Test (type = int) || 8353.4 -> 8368.8 +0.18%
Arithmetic Test (type = short) || 7496.0 -> 7509.4 +0.18%
Arithmetic Test (type = long) || 8353.5 -> 8366.4 +0.15%
Dhrystone 2 using register variables || 49802.4 -> 49814.9 +0.03%
Pipe Throughput Test || 18512.0 -> 17685.8 -4.46%
File Write (10 seconds) || 8100.0 -> 1400.0 -82.72%
File Write (30 seconds) || 8911.0 -> 1146.0 -87.14%

--8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<----8<--

Enjoy!
Carlo

--
  *                                          ...Ma appena fuori tutto e' gomma,
* K * Carlo E. Prelz - fluido@marktest.pt     tutto e' cicca impiastricciata...
  *                                     (Marco Zappa-Niente cicca nella scuola)