JikesRVM+Harmony: initial performance evaluation

[Aleksey Shipilev <al...@gmail.com>]

[crossposting to Harmony-dev and JikesRVM-research]

Hi,

I was eager to compare Harmony classlib and GNU Classpath running on
JikesRVM. Here's a little experiment I had conducted in this
environment:
 - P4 (Northwood) 2.8 Ghz / 1024 Mb PC-3200 running Gentoo x86, 2.6.23
 - JikesRVM r14857 + RVM-624 [1]
 - Harmony r683652
 - "production" config with "config.bootimage.compiler=opt",
"config.assertions=none"
 - -Xms512m -Xmx512m

I would appreciate comments on the configuration and measurement
methodology for JikesRVM. There are no full threading support for
Harmony in JikesRVM yet, so I run SPECjbb2005 in 1-threaded mode.
There are also failures on Dacapo, so the looking into the performance
can be regarded as premature, still - here we go with results:

1. ==== SPECjbb2005: ====
5 iterations with 1 thread, 4 minutes each
(can't provide exact numbers due to legal issues, normalized to GNU
Classpath, the more the better)

G: 1000
H: 1930 (+93%)

Yes, I mean it. Almost 2x boost relative to GNU Classpath. :)

2. ==== DaCapo ====
default workload, 5 warmup iterations
(msecs, the less the better, percents are boosts)

antlr:  G: 3397, H: 3935  (-13%)
fop: G: 3422, H: 5036 (-32%)
hsqldb: G: 3886, H: 3744 (+4%)
luindex: G: 14130, H: 13228 (+7%)
lusearch: G: 13491, H: 19084 (-29%)
pmd: G: 9990, H: 9166 (+9%)
bloat: failed due to [2]
chart: failed due to [3]
jython: failed due to [4]
xalan: failed due to [5]
eclipse: skipped [6]

AFAIK, there was almost zero research on DaCapo on Harmony last year,
so the picture is terrible. There's a way to go though :)

Thanks,
Aleksey.


[1] "NPE in java.util.logging.LogManager with Apache Harmony classlib"
http://jira.codehaus.org/browse/RVM-624

[2] "DaCapo bloat fails with Apache Harmony class library"
http://jira.codehaus.org/browse/RVM-617

[3] "DaCapo chart fails with Apache Harmony class library"
http://jira.codehaus.org/browse/RVM-618

[4] "DaCapo jython fails with Apache Harmony class library"
http://jira.codehaus.org/browse/RVM-620

[5] "DaCapo xalan fails with Apache Harmony class library"
http://jira.codehaus.org/browse/RVM-621

[6] "DaCapo eclipse fails with Apache Harmony class library"
http://jira.codehaus.org/browse/RVM-619