You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Stefano Mazzocchi <st...@apache.org> on 2006/11/17 20:32:47 UTC
[performance] harmony 'release' results included
I've built a 'release' version of Harmony and I'm incorporating the new
results in the previous runs.
--------------------------------------------------------------------------
harmony-debug
-------------
Apache Harmony Launcher : (c) Copyright 1991, 2006 The Apache Software
Foundation or its licensors, as applicable.
java version "1.5.0"
pre-alpha : not complete or compatible
svn = r476006, (Nov 16 2006), Linux/em64t/gcc 4.0.3, debug build
harmony-release
---------------
Apache Harmony Launcher : (c) Copyright 1991, 2006 The Apache Software
Foundation or its licensors, as applicable.
java version "1.5.0"
pre-alpha : not complete or compatible
svn = r476234, (Nov 17 2006), Linux/em64t/gcc 4.0.3, release build
sun5
---
java version "1.5.0_09"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_09-b03)
Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_09-b03, mixed mode)
sun6
----
java version "1.6.0-rc"
Java(TM) SE Runtime Environment (build 1.6.0-rc-b104)
Java HotSpot(TM) 64-Bit Server VM (build 1.6.0-rc-b104, mixed mode)
ibm
---
java version "1.5.0"
Java(TM) 2 Runtime Environment, Standard Edition (build
pxa64dev-20061002a (SR3) )
IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux amd64-64
j9vmxa6423-20061001 (JIT enabled)
J9VM - 20060915_08260_LHdSMr
JIT - 20060908_1811_r8
GC - 20060906_AA)
JCL - 20061002
bea
---
java version "1.5.0_06"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
BEA JRockit(R) (build
R26.4.0-63-63688-1.5.0_06-20060626-2259-linux-x86_64, )
--------------------------------------------------------------------------
Test #1: java scimark2 (http://math.nist.gov/scimark2/)
command: java jnt.scimark2.commandline
Sun6
Composite Score: 364.5832265230057
FFT (1024): 220.8458713892794
SOR (100x100): 696.1542342357722
Monte Carlo : 149.37978088875656
Sparse matmult (N=1000, nz=5000): 326.37451873283845
LU (100x100): 430.1617273683819
BEA
Composite Score: 359.13480378697835
FFT (1024): 303.8746880751562
SOR (100x100): 454.25628897202307
Monte Carlo : 93.23913192138497
Sparse matmult (N=1000, nz=5000): 530.44112637391
LU (100x100): 413.8627835924175
Sun5
Composite Score: 332.84987587548574
FFT (1024): 216.5144595799027
SOR (100x100): 689.429322146947
Monte Carlo : 25.791262124978065
Sparse matmult (N=1000, nz=5000): 317.5193965699373
LU (100x100): 414.99493895566377
IBM
Composite Score: 259.8249218693683
FFT (1024): 296.8415012789055
SOR (100x100): 428.974881649179
Monte Carlo : 89.15159857584082
Sparse matmult (N=1000, nz=5000): 144.3524241203982
LU (100x100): 339.8042037225181
Harmony-release
Composite Score: 124.6867792398682
FFT (1024): 214.17495491343038
SOR (100x100): 243.36384871876157
Monte Carlo : 14.886614559623572
Sparse matmult (N=1000, nz=5000): 71.70240430885481
LU (100x100): 79.30607369867067
Harmony-debug
Composite Score: 113.65082278962575
FFT (1024): 203.76641991778123
SOR (100x100): 224.37761309236748
Monte Carlo : 9.063866256533116
Sparse matmult (N=1000, nz=5000): 65.4051866327227
LU (100x100): 65.6410280487242
--------------------------------------------------------------------------
Test 2: Dhrystones (http://www.c-creators.co.jp/okayan/DhrystoneApplet/)
command: java dhry 100000000
sun6: 8552856 dhrystones/sec
sun5: 6605892
bea: 5678914
harmony-release 1132348
harmony-debug 669734
ibm: 501562
--------------------------------------------------------------------------
Test 3: Sieve (part of http://www.sax.de/~adlibit/tya18.tgz)
command: java Sieve 30
sun6 8545 sieves/sec
sun5 8364
bea 6174
harmony-release 1848
harmony-debug 1836
ibm 225
--
Stefano.
Re: [performance] harmony 'release' results included
Posted by Stefano Mazzocchi <st...@apache.org>.
Robin Garner wrote:
> Stefano Mazzocchi wrote:
>> Robin Garner wrote:
>>> Stefano,
>>>
>>> Which iteration of the benchmark are you timing ?
>>
>> ehm, the first.
>>
>>> This can make a huge
>>> difference (our postion is argued in our OOPSLA paper here
>>> http://www.dacapobench.org/dacapo-oopsla-2006.pdf).
>>
>> I can't seem to be able to read this PDF file, acrobat complains and
>> macosx preview reads only parts of it. am I the only one having trouble
>> with this?
>
> You could try this link. sourceforge might be playing up :-|
>
> http://cs.anu.edu.au/people/Robin.Garner/dacapo-oopsla-2006.pdf
that worked. I'll read with great interest.
--
Stefano.
Re: [performance] harmony 'release' results included
Posted by Robin Garner <ro...@anu.edu.au>.
Stefano Mazzocchi wrote:
> Robin Garner wrote:
>> Stefano,
>>
>> Which iteration of the benchmark are you timing ?
>
> ehm, the first.
>
>> This can make a huge
>> difference (our postion is argued in our OOPSLA paper here
>> http://www.dacapobench.org/dacapo-oopsla-2006.pdf).
>
> I can't seem to be able to read this PDF file, acrobat complains and
> macosx preview reads only parts of it. am I the only one having trouble
> with this?
You could try this link. sourceforge might be playing up :-|
http://cs.anu.edu.au/people/Robin.Garner/dacapo-oopsla-2006.pdf
Section 5 is the most relevant to what I'm saying.
cheers
>
>> It would be really informative to see these results plotted for 1st, 2nd
>> and 3rd iteration, for a range of heap sizes.
>
> I'll see what I can do. I'll try to run dacapo as well.
>
>> cheers
>>
>> Stefano Mazzocchi wrote:
>>> I've built a 'release' version of Harmony and I'm incorporating the new
>>> results in the previous runs.
>>>
>>> --------------------------------------------------------------------------
>>>
>>>
>>> harmony-debug
>>> -------------
>>> Apache Harmony Launcher : (c) Copyright 1991, 2006 The Apache Software
>>> Foundation or its licensors, as applicable.
>>> java version "1.5.0"
>>> pre-alpha : not complete or compatible
>>> svn = r476006, (Nov 16 2006), Linux/em64t/gcc 4.0.3, debug build
>>>
>>> harmony-release
>>> ---------------
>>> Apache Harmony Launcher : (c) Copyright 1991, 2006 The Apache Software
>>> Foundation or its licensors, as applicable.
>>> java version "1.5.0"
>>> pre-alpha : not complete or compatible
>>> svn = r476234, (Nov 17 2006), Linux/em64t/gcc 4.0.3, release build
>>>
>>> sun5
>>> ---
>>> java version "1.5.0_09"
>>> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_09-b03)
>>> Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_09-b03, mixed mode)
>>>
>>> sun6
>>> ----
>>> java version "1.6.0-rc"
>>> Java(TM) SE Runtime Environment (build 1.6.0-rc-b104)
>>> Java HotSpot(TM) 64-Bit Server VM (build 1.6.0-rc-b104, mixed mode)
>>>
>>> ibm
>>> ---
>>> java version "1.5.0"
>>> Java(TM) 2 Runtime Environment, Standard Edition (build
>>> pxa64dev-20061002a (SR3) )
>>> IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux amd64-64
>>> j9vmxa6423-20061001 (JIT enabled)
>>> J9VM - 20060915_08260_LHdSMr
>>> JIT - 20060908_1811_r8
>>> GC - 20060906_AA)
>>> JCL - 20061002
>>>
>>> bea
>>> ---
>>> java version "1.5.0_06"
>>> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
>>> BEA JRockit(R) (build
>>> R26.4.0-63-63688-1.5.0_06-20060626-2259-linux-x86_64, )
>>>
>>>
>>> --------------------------------------------------------------------------
>>>
>>>
>>> Test #1: java scimark2 (http://math.nist.gov/scimark2/)
>>>
>>> command: java jnt.scimark2.commandline
>>>
>>> Sun6
>>> Composite Score: 364.5832265230057
>>> FFT (1024): 220.8458713892794
>>> SOR (100x100): 696.1542342357722
>>> Monte Carlo : 149.37978088875656
>>> Sparse matmult (N=1000, nz=5000): 326.37451873283845
>>> LU (100x100): 430.1617273683819
>>>
>>> BEA
>>> Composite Score: 359.13480378697835
>>> FFT (1024): 303.8746880751562
>>> SOR (100x100): 454.25628897202307
>>> Monte Carlo : 93.23913192138497
>>> Sparse matmult (N=1000, nz=5000): 530.44112637391
>>> LU (100x100): 413.8627835924175
>>>
>>> Sun5
>>> Composite Score: 332.84987587548574
>>> FFT (1024): 216.5144595799027
>>> SOR (100x100): 689.429322146947
>>> Monte Carlo : 25.791262124978065
>>> Sparse matmult (N=1000, nz=5000): 317.5193965699373
>>> LU (100x100): 414.99493895566377
>>>
>>> IBM
>>> Composite Score: 259.8249218693683
>>> FFT (1024): 296.8415012789055
>>> SOR (100x100): 428.974881649179
>>> Monte Carlo : 89.15159857584082
>>> Sparse matmult (N=1000, nz=5000): 144.3524241203982
>>> LU (100x100): 339.8042037225181
>>>
>>> Harmony-release
>>> Composite Score: 124.6867792398682
>>> FFT (1024): 214.17495491343038
>>> SOR (100x100): 243.36384871876157
>>> Monte Carlo : 14.886614559623572
>>> Sparse matmult (N=1000, nz=5000): 71.70240430885481
>>> LU (100x100): 79.30607369867067
>>>
>>> Harmony-debug
>>> Composite Score: 113.65082278962575
>>> FFT (1024): 203.76641991778123
>>> SOR (100x100): 224.37761309236748
>>> Monte Carlo : 9.063866256533116
>>> Sparse matmult (N=1000, nz=5000): 65.4051866327227
>>> LU (100x100): 65.6410280487242
>>>
>>> --------------------------------------------------------------------------
>>>
>>>
>>> Test 2: Dhrystones (http://www.c-creators.co.jp/okayan/DhrystoneApplet/)
>>>
>>> command: java dhry 100000000
>>>
>>> sun6: 8552856 dhrystones/sec
>>> sun5: 6605892
>>> bea: 5678914
>>> harmony-release 1132348
>>> harmony-debug 669734
>>> ibm: 501562
>>>
>>> --------------------------------------------------------------------------
>>>
>>>
>>> Test 3: Sieve (part of http://www.sax.de/~adlibit/tya18.tgz)
>>>
>>> command: java Sieve 30
>>>
>>> sun6 8545 sieves/sec
>>> sun5 8364
>>> bea 6174
>>> harmony-release 1848
>>> harmony-debug 1836
>>> ibm 225
>>>
>>>
>>
>
>
--
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/
Re: [performance] harmony 'release' results included
Posted by Stefano Mazzocchi <st...@apache.org>.
Robin Garner wrote:
> Stefano,
>
> Which iteration of the benchmark are you timing ?
ehm, the first.
> This can make a huge
> difference (our postion is argued in our OOPSLA paper here
> http://www.dacapobench.org/dacapo-oopsla-2006.pdf).
I can't seem to be able to read this PDF file, acrobat complains and
macosx preview reads only parts of it. am I the only one having trouble
with this?
> It would be really informative to see these results plotted for 1st, 2nd
> and 3rd iteration, for a range of heap sizes.
I'll see what I can do. I'll try to run dacapo as well.
> cheers
>
> Stefano Mazzocchi wrote:
>> I've built a 'release' version of Harmony and I'm incorporating the new
>> results in the previous runs.
>>
>> --------------------------------------------------------------------------
>>
>>
>> harmony-debug
>> -------------
>> Apache Harmony Launcher : (c) Copyright 1991, 2006 The Apache Software
>> Foundation or its licensors, as applicable.
>> java version "1.5.0"
>> pre-alpha : not complete or compatible
>> svn = r476006, (Nov 16 2006), Linux/em64t/gcc 4.0.3, debug build
>>
>> harmony-release
>> ---------------
>> Apache Harmony Launcher : (c) Copyright 1991, 2006 The Apache Software
>> Foundation or its licensors, as applicable.
>> java version "1.5.0"
>> pre-alpha : not complete or compatible
>> svn = r476234, (Nov 17 2006), Linux/em64t/gcc 4.0.3, release build
>>
>> sun5
>> ---
>> java version "1.5.0_09"
>> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_09-b03)
>> Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_09-b03, mixed mode)
>>
>> sun6
>> ----
>> java version "1.6.0-rc"
>> Java(TM) SE Runtime Environment (build 1.6.0-rc-b104)
>> Java HotSpot(TM) 64-Bit Server VM (build 1.6.0-rc-b104, mixed mode)
>>
>> ibm
>> ---
>> java version "1.5.0"
>> Java(TM) 2 Runtime Environment, Standard Edition (build
>> pxa64dev-20061002a (SR3) )
>> IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux amd64-64
>> j9vmxa6423-20061001 (JIT enabled)
>> J9VM - 20060915_08260_LHdSMr
>> JIT - 20060908_1811_r8
>> GC - 20060906_AA)
>> JCL - 20061002
>>
>> bea
>> ---
>> java version "1.5.0_06"
>> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
>> BEA JRockit(R) (build
>> R26.4.0-63-63688-1.5.0_06-20060626-2259-linux-x86_64, )
>>
>>
>> --------------------------------------------------------------------------
>>
>>
>> Test #1: java scimark2 (http://math.nist.gov/scimark2/)
>>
>> command: java jnt.scimark2.commandline
>>
>> Sun6
>> Composite Score: 364.5832265230057
>> FFT (1024): 220.8458713892794
>> SOR (100x100): 696.1542342357722
>> Monte Carlo : 149.37978088875656
>> Sparse matmult (N=1000, nz=5000): 326.37451873283845
>> LU (100x100): 430.1617273683819
>>
>> BEA
>> Composite Score: 359.13480378697835
>> FFT (1024): 303.8746880751562
>> SOR (100x100): 454.25628897202307
>> Monte Carlo : 93.23913192138497
>> Sparse matmult (N=1000, nz=5000): 530.44112637391
>> LU (100x100): 413.8627835924175
>>
>> Sun5
>> Composite Score: 332.84987587548574
>> FFT (1024): 216.5144595799027
>> SOR (100x100): 689.429322146947
>> Monte Carlo : 25.791262124978065
>> Sparse matmult (N=1000, nz=5000): 317.5193965699373
>> LU (100x100): 414.99493895566377
>>
>> IBM
>> Composite Score: 259.8249218693683
>> FFT (1024): 296.8415012789055
>> SOR (100x100): 428.974881649179
>> Monte Carlo : 89.15159857584082
>> Sparse matmult (N=1000, nz=5000): 144.3524241203982
>> LU (100x100): 339.8042037225181
>>
>> Harmony-release
>> Composite Score: 124.6867792398682
>> FFT (1024): 214.17495491343038
>> SOR (100x100): 243.36384871876157
>> Monte Carlo : 14.886614559623572
>> Sparse matmult (N=1000, nz=5000): 71.70240430885481
>> LU (100x100): 79.30607369867067
>>
>> Harmony-debug
>> Composite Score: 113.65082278962575
>> FFT (1024): 203.76641991778123
>> SOR (100x100): 224.37761309236748
>> Monte Carlo : 9.063866256533116
>> Sparse matmult (N=1000, nz=5000): 65.4051866327227
>> LU (100x100): 65.6410280487242
>>
>> --------------------------------------------------------------------------
>>
>>
>> Test 2: Dhrystones (http://www.c-creators.co.jp/okayan/DhrystoneApplet/)
>>
>> command: java dhry 100000000
>>
>> sun6: 8552856 dhrystones/sec
>> sun5: 6605892
>> bea: 5678914
>> harmony-release 1132348
>> harmony-debug 669734
>> ibm: 501562
>>
>> --------------------------------------------------------------------------
>>
>>
>> Test 3: Sieve (part of http://www.sax.de/~adlibit/tya18.tgz)
>>
>> command: java Sieve 30
>>
>> sun6 8545 sieves/sec
>> sun5 8364
>> bea 6174
>> harmony-release 1848
>> harmony-debug 1836
>> ibm 225
>>
>>
>
>
--
Stefano.
Re: [performance] harmony 'release' results included
Posted by Robin Garner <ro...@anu.edu.au>.
Stefano,
Which iteration of the benchmark are you timing ? This can make a huge
difference (our postion is argued in our OOPSLA paper here
http://www.dacapobench.org/dacapo-oopsla-2006.pdf).
It would be really informative to see these results plotted for 1st, 2nd
and 3rd iteration, for a range of heap sizes.
cheers
Stefano Mazzocchi wrote:
> I've built a 'release' version of Harmony and I'm incorporating the new
> results in the previous runs.
>
> --------------------------------------------------------------------------
>
> harmony-debug
> -------------
> Apache Harmony Launcher : (c) Copyright 1991, 2006 The Apache Software
> Foundation or its licensors, as applicable.
> java version "1.5.0"
> pre-alpha : not complete or compatible
> svn = r476006, (Nov 16 2006), Linux/em64t/gcc 4.0.3, debug build
>
> harmony-release
> ---------------
> Apache Harmony Launcher : (c) Copyright 1991, 2006 The Apache Software
> Foundation or its licensors, as applicable.
> java version "1.5.0"
> pre-alpha : not complete or compatible
> svn = r476234, (Nov 17 2006), Linux/em64t/gcc 4.0.3, release build
>
> sun5
> ---
> java version "1.5.0_09"
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_09-b03)
> Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_09-b03, mixed mode)
>
> sun6
> ----
> java version "1.6.0-rc"
> Java(TM) SE Runtime Environment (build 1.6.0-rc-b104)
> Java HotSpot(TM) 64-Bit Server VM (build 1.6.0-rc-b104, mixed mode)
>
> ibm
> ---
> java version "1.5.0"
> Java(TM) 2 Runtime Environment, Standard Edition (build
> pxa64dev-20061002a (SR3) )
> IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux amd64-64
> j9vmxa6423-20061001 (JIT enabled)
> J9VM - 20060915_08260_LHdSMr
> JIT - 20060908_1811_r8
> GC - 20060906_AA)
> JCL - 20061002
>
> bea
> ---
> java version "1.5.0_06"
> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
> BEA JRockit(R) (build
> R26.4.0-63-63688-1.5.0_06-20060626-2259-linux-x86_64, )
>
>
> --------------------------------------------------------------------------
>
> Test #1: java scimark2 (http://math.nist.gov/scimark2/)
>
> command: java jnt.scimark2.commandline
>
> Sun6
> Composite Score: 364.5832265230057
> FFT (1024): 220.8458713892794
> SOR (100x100): 696.1542342357722
> Monte Carlo : 149.37978088875656
> Sparse matmult (N=1000, nz=5000): 326.37451873283845
> LU (100x100): 430.1617273683819
>
> BEA
> Composite Score: 359.13480378697835
> FFT (1024): 303.8746880751562
> SOR (100x100): 454.25628897202307
> Monte Carlo : 93.23913192138497
> Sparse matmult (N=1000, nz=5000): 530.44112637391
> LU (100x100): 413.8627835924175
>
> Sun5
> Composite Score: 332.84987587548574
> FFT (1024): 216.5144595799027
> SOR (100x100): 689.429322146947
> Monte Carlo : 25.791262124978065
> Sparse matmult (N=1000, nz=5000): 317.5193965699373
> LU (100x100): 414.99493895566377
>
> IBM
> Composite Score: 259.8249218693683
> FFT (1024): 296.8415012789055
> SOR (100x100): 428.974881649179
> Monte Carlo : 89.15159857584082
> Sparse matmult (N=1000, nz=5000): 144.3524241203982
> LU (100x100): 339.8042037225181
>
> Harmony-release
> Composite Score: 124.6867792398682
> FFT (1024): 214.17495491343038
> SOR (100x100): 243.36384871876157
> Monte Carlo : 14.886614559623572
> Sparse matmult (N=1000, nz=5000): 71.70240430885481
> LU (100x100): 79.30607369867067
>
> Harmony-debug
> Composite Score: 113.65082278962575
> FFT (1024): 203.76641991778123
> SOR (100x100): 224.37761309236748
> Monte Carlo : 9.063866256533116
> Sparse matmult (N=1000, nz=5000): 65.4051866327227
> LU (100x100): 65.6410280487242
>
> --------------------------------------------------------------------------
>
> Test 2: Dhrystones (http://www.c-creators.co.jp/okayan/DhrystoneApplet/)
>
> command: java dhry 100000000
>
> sun6: 8552856 dhrystones/sec
> sun5: 6605892
> bea: 5678914
> harmony-release 1132348
> harmony-debug 669734
> ibm: 501562
>
> --------------------------------------------------------------------------
>
> Test 3: Sieve (part of http://www.sax.de/~adlibit/tya18.tgz)
>
> command: java Sieve 30
>
> sun6 8545 sieves/sec
> sun5 8364
> bea 6174
> harmony-release 1848
> harmony-debug 1836
> ibm 225
>
>
--
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/