cache killer memory death test - 2.0 vs 2.2 vs arca - programs inside

Harvey J. Stein (hjstein@bfr.co.il)
18 Apr 1999 17:34:28 +0300


I tested 2.0.36, 2.2.5 & 2.2.5arca12 with the program appended to the
end of this message. The program builds a large gdbm file. It's a
test mockup of a program that I run daily on a large number of
machines. The input file for a good test is large (>=1.5mb), so I
generate it randomly using an awk script (included here). Since awk
starts with the same random number seed each time, if you follow the
instructions you should get the same input file as I'm using.

I ran two tests. The first is a "noninteractive" test & the second is
an "interactive" test.

Test 1:

Reboot, log in & run dbase program on a 50,000 record file. Delete
dbase file, sync & run again. Delete dbase file, sync & run again.
Don't do anything while dbase is running. All three runs reported.
"worst" is worst elapsed time for writing group of 1,000
records. "2nd" is next to worst time.

Test 2:
Reboot, log in, startx & run dbase on a 50,000 record file while doing
other stuff. This means running netscape, emacs, 1 or more xterms &
pppd, except with 2.0.36 for which the ppp modules wouldn't load.
Comments refers to interactive feel while test is running. I didn't
run it 3x because it was too agonizing.

Test 1 Test 2
user sys elapsed worst 2nd user sys elapsed worst 2nd comments
2.0.36 27.69 7.70 2:37.27 9.73 6.88 28.97 10.21 17:30.68 76.37 75.11 poor
28.32 7.33 0:43.87 3.36 2.32
27.52 7.40 0:43.08 3.42 2.22

2.2.5 28.13 6.80 2:50.52 8.00 7.19 29.44 8.70 13:21.34 61.29 64.95 fine
28.64 6.61 1:47.39 7.91 6.77
28.55 6.40 1:48.49 7.88 7.11

2.2.5arca12 27.99 6.77 1:24.68 14.93 9.38 30.13 6.91 4:13.60 14.08 13.09 very poor
28.98 6.69 1:19.67 14.12 9.56
28.18 6.77 1:18.99 13.89 9.39

Comments:

2.0.36 has poor interactive performance during the test. Bringing up
windows, reading files, editing within emacs all have noticable
pauses. It's annoying, but it's usable. It took over 3 minutes for
netscape to come up. Presumably the first noninteractive test was
much slower than the other two because the system was swapping
everything out (but this should be verified). But this varies - when
the application running on my workstation in the office (64mb PII300,
120k records) with kernel 2.0.36, I typically leave my desk & wait for
it to finish.

2.2.5 was quite usable during the test. It took about 1.5 minutes for
the netscape window to come up. After they were running things were
ok. local editing, bringing up slashdot in netscape, etc, all were
somewhat sluggish but usable. Pauses similar to 2.0.36, but of
shorter duration. Why were test 1's successive runs much slower than
for 2.0.36? Maybe 2.2.5 is more aggressive about keeping programs in
ram?

arca12 is fast in getting the data to disk, but very poor in
interactive feel - much worse than 2.0.36. Maybe it just needs
tuning? If so, it should probably ship with different parameters. The
mouse froze once or twice for ~5 seconds. Moving windows around &
switching viewports was sometimes ok & sometimes took 5-10 seconds. I
started netscape just after starting the dbase test program & netscape
didn't come up until the test was over! Note that the worst & 2nd
worst arca #s aren't good, but they're almost anomalies. The worst 4
(in the 1st non-interactive test) are 14.93, 9.38, 5.27 & 2.68. All
other group times are <=1.32 seconds. The poor interactive
performance isn't because of the red-black trees - I experienced the
same behavior with arca8 (a pre-rb-tree release).

BTW, for testing I use a 50,000 line file (= 50,000 gdbm records), but
in "real life" the input file has ~120,000 lines, and the system
behaves much worst in practice than the tests show. I cut the size
down for testing to speed things up, and because I'm testing at home
on a 32mb machine, whereas the application runs in the office on a
PII-300 with 64mb of ram & on 533mhz alphas with 128mb of ram. None
the less, the tests still indicate the behavior I've seen in general.
Feel free to try out the tests with a larger input file to see what
happens.

Methodology:

1. Build the program (make).
2. Build a 50,000 record input file for the tests:
./generate_input.awk -v nr=50000 >t4.in
3. Reboot & do test as in something like:
time ./dbase <t4.in >t4.linux225.out1 2>&1
rm dbase.gdbm
sync
time ./dbase <t4.in >t4.linux225.out2 2>&1
rm dbase.gdbm
sync
time ./dbase <t4.in >t4.linux225.out3 2>&1
rm dbase.gdbm
sync
or
startx
time ./dbase <t4.in >t4.linux225.interactive.out1 2>&1
4. If there are still tests to do, goto 3.

Hardware/software:

pentium 133, 32mb ram, redhat 5.2 installed:

hjstein@bacall:~/dbase$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 5
model : 2
model name : Pentium 75 - 200
stepping : 12
cpu MHz : 132.959741
fdiv_bug : no
hlt_bug : no
sep_bug : no
f00f_bug : yes
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr mce cx8
bogomips : 53.04
hjstein@bacall:~/dbase$ cat /proc/pci
PCI devices found:
Bus 0, device 0, function 0:
Host bridge: Intel 82439HX Triton II (rev 1).
Medium devsel. Master Capable. Latency=64.
Bus 0, device 7, function 0:
ISA bridge: Intel 82371SB PIIX3 ISA (rev 0).
Medium devsel. Fast back-to-back capable. Master Capable. No bursts.
Bus 0, device 7, function 1:
IDE interface: Intel 82371SB PIIX3 IDE (rev 0).
Medium devsel. Fast back-to-back capable. Master Capable. Latency=64.
I/O at 0xffa0 [0xffa1].
Bus 0, device 13, function 0:
VGA compatible controller: S3 Inc. Vision 968 (rev 0).
Medium devsel. IRQ 11.
Non-prefetchable 32 bit memory at 0xfa000000 [0xfa000000].

Programs:

tarred, gzipped & uuencoded Makefile, dbase.c & generate_input.awk.

begin 644 dbase.tar.gz
M'XL(`*.6&3<``^U9>U/;2!+/O]:GF"4%D<"`#1CJ<,@5&(>P@,W9<+E=0JED
M/6PMLN231@%VE^^^W3TS>OA!LG677-4MDY21>F;Z/;_NL9V!E;B;#GYNV*^^
MS:C7:KL[.^P58VQO5_RMRW=\W*XW&HSM[M6V=^M[C:T:D+9V=NNO6.T;Z5,:
M:<*MF+%7HU\2[OKA,^O<./D>"GW?\=H/[2!U7/;6C>,PVAB]TW)2\IAL\L>)
MF\PA^^-Y9'`FGZ)RQY]FRV,_'$[OYD[@#\JT-/2!7*8-G<$8*1HJYK@><\-T
MS'YCK99Y?&1VSZJ5S576B1B8$\5L=5.KB#1C[5[/%(MZ[>YENU-EL/`J]EV'
M\8A%$S<4'\Y@8_ZVH\/C;%_+"M]PN=[B%AV?1<):Y]U^NRS-#J+$%9_.8H&=
MZ_-S,H<=<NZ.)QRW^B$D(F=A&@0D>L'>X^M+VCJ[,W'M*'18#']BA]W[?,02
M:^RR._=Q`:_+=N^"F%VZ\=A/$C\*$^E?7S@-PIGY83Z/Z\Y9I_NQ@UZX#E'V
M,/1_!>.)#<E]8K9M.@.3**8=.6Y3TV"Y^\#=.&2@=,*9/8+#.KW.'"?#Y.;V
MIE&[;2(G[37DA1^ZH(`0?M;^Z;S=D3JQ>GUF0>?PHIVOV*[-+#CI=:\O.]<7
M^:+=F37]ZZ/996PGS]2"\B%X_*8D>ZU^"_9.FR9R!&CZY\AW#%BAG1P?79CO
M3\_;##+'8Z5QP#!GFL4UL?O9+*U3:T@;,4"DYP<N*74KUBR)DH"G;4FLQ9@Z
M;L)IZ<U6#9RMB:#X(2<>@R"R[Q((JY33J&\U-40$WR;;5YF'(I0&F%''1TQ)
MIL@I5P%$I#9D+:(,M\83]IN&/!5YG`C%)W%DF[BH2=.H"`B[2YJ03=E><)I4
M`GT(^<_-;$[/):QR0Q-2^/H[X@**TK2^H@,IDV5`&)[*/&-W$L6";9$CEA9!
MK19,675#1S"2\IPH'02NT-R<N+$ICFANDX/%1_A4[2WHLP'N,%-\8NLL%SEW
M18$GP#5[GF?R19Y)F:<;6!-$M!)/X<HR%QDDX>TILV$SJ`9/GF[V6V;K_,R\
M:IT90LH$*@?W]"5A[@%;_MO&EE?%#65"NC9#4LI)PJ=PJ4HX1<[=G-9"SHE2
M-V]*IWUKN,)8L$1*G)DMY0_8HR&$XAFG_(43H!+#]T`,G-T?Q(DQX!14/.D"
M*(P`$]7*$F!LM[>?P;`5ELL2>%Y4&\^/$PY6@_1*[/(4,'6Z)`H?/XFXP.F<
M6'P$$`>"=(/Y"1NF5FR%W!5%3#+A(U<4D.P0"TCQP-,('R:JHQ<@!IQ3D:B3
M(8:BH?7JN6^>]J[[/?8[/GW,GGHGO4M)RYYZW:L/D@9/B@'Y3-A4<N?!8G<R
MY4[;"L-(5G@)4NS3F^7DTYM]!I^4/!6"LZJP$J")8%NG-VJF$"CF^%KV$<K9
MF7IRH6ID*$F^5!"F$N6K+`,&,E'2$.W#%D0F2RD]])DFQIC1F8PEE5"^G!>Q
M%U4F-TROB;Q7B0\RT[$VMNY<$UH/G<K1*CQ]MH(4G(IG81A'Z03:._&6I`-%
MR)`3>6#G(K$DD3;G;):6'7/96:H6>!7Y"`6Q]7&H<!U@B8$TS3@45TQXC-"F
MIDK&`7'Z6#,JH:9HN_2\[C.1-5]E('E%";PI=C/4+TSY('\M*$C5&=64/H*5
M8$3F]]Q5BSQ$NVB-<,$!&=`LDLN^PVECK2YW@F2%!`F/8DJ4*FI195(P=2NG
MG7Z[=R7=G4!;:H^8#GL-6?IMPK(ZV]<JF.U90&3*LY65DHZ%@X"[9P[#J6B&
M9?O).MA.HV=61&<M6,GC4`ASN3G'R2>`>=!LGDY_7GH4"LD`,2B\6C#ISRCR
MGSI"Z`#*H$Y*%67=URKRM0(C-LFN%5A6[F.?N_B07:R>"P/>33!I*D]:91"[
MUETSSQ9*EAGY6]!B8,_-].7$8%X4RQ,QZ_*RS3.BX8J%]&FI-91:($([:Z4!
MGZ?*=7@71O>A0G[A=B^.QH7CLK$(D^6-:DJ%IR(HJ5(BD"F'I$'J!P[6$+HB
MK'I9&4&HF`L#@GD00=T(X,Z#5^X#5I-6EY&M@#I@9?YF6_;(-252U&LPFK+_
M4XVQ&,6F6761$IS,?$Z""V:[#F*`I6JB"$TZ[8]0M7\7T/+^L']E&-A%90[!
M[)1.@MU&J?$I\!1A<'DTX0*VB%_KL/6AW3_]&2[U*[E5X"_XC#P])QE":LV8
M$_L":^99T%XX,M*RDI5N*2NYX5+7:7\0^DXYYWZ$78NRQ4N@I_%TCVHBP_\)
M9/M*'NZ50KRK%$JA?KO[GOPE`[^VUA00K!)AF:*)2%/+8$;UZ<N!0PF3T%7-
M31)A9E5ED3K74^9F-Z1FF5U;]O#[K(`(^=5KI9@["YF<H(WPOHC)M&OGL)KC
M?;4$`7">?Z2#Y'X*_F-HRZZ)UF.8P-W;B[&ZYUKR&Q-F<7(A`P=O9$<640QN
M7X4X2HJ\=DN(D^VQU&]!X9<9,(/GA=-1ZF]D*SRWAY@]>Q55X?'T$8&PLXSK
M*&Y_=HK*_?PI]2W3OC2Q=,0%26+ES%9`\VS;C../TTG@VQ;4)2]*H7Y8^*7'
M)"W$8#JIR[)D#5@L8'$E*#I9U8(OVO8D8O6$N9A7A>SJ8#2?KQ)CRP_I,FK%
M0[LJFM!5>/Y\<ZM*A=P[54_PR][0P$Q73-G?L2"R.C'_7W_9_3)F!@5P<PBW
MP!C2VZ2LWK#N[_Z;,@#^]AJ-1;__P/-V7?S^L]UH;-=WD+*W_?+[SW<9KW_8
M3)-X<^"'FT,(.UOW-,U+0YMC-SX&&'BH/LHK&"+_`WO+X)T@0('5`Z$)UH@2
M^9'@).,56R%@`]0)P4TN(@I;I5D=[HS&["8L+'H0V0'@:V0G].E+E9`,A4CQ
MKM>,M4933N#7MTM+](9-/K+PL?-L$@/V]H!VB[>U-77-5#MU^ANXG+OQC>)O
M!9.1!3=<XU8@*,&J1$)<+[_@.&J?G':('VT8T`5XZ?"H==Q^?_+A],>S\XM.
M]_(?O?[5]3\__NNGGZV!#05B./)_N0O&833Y=YSP]//]P^.OI+Z2"DS@<\A'
MNF)+6I!Q_D&MR<"H;#&\Y49),_Q;[!+3`5S3,Q95YE<)G67[2S6;W*1E#608
MLW>BMT-F4."285+P.F9)HP;]0FP4FU/9<V0AE'-"71$(BH+@5](7A,3KZTWY
M+&HE=*WU&C2M#6Q<\1]57))$-F0RH9J:6(N%S?4U7??7Z\8RRX-7O"G1-FQI
M!<%]\#G>:_XRM4K@_X5UY^)7G-]&QO.__S/$_NSW?_B/O__O;NV\X/_W&.?=
MP^/STZ-V'T[D>H#W$JWU_OSPI(]Q`=*0K7>W-(W2!/HX\:M@]%<Y'2_C9;R,
.E_'_._X`FM+M7@`H````
`
end

-- 
Harvey J. Stein
BFM Financial Research
hjstein@bfr.co.il

- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/