You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@xalan.apache.org by Tom Amiro <To...@Sun.COM> on 2002/07/18 20:49:46 UTC
XSLTMark benchmark results Xalan vs XSLTC
Hi,
I ran the XSLTMark benchmark with the latest versions of
Xalan and XSLTC off the main trunk at Apache and got results
that can't be true, or reflect a servious regression in
Xalan's performance.
In short, I get
KB's/sec
Xalan 2.5
XSLTC 7161
Something is really wrong somewhere.
Here's the detail.
*************************************************************
XALAN
*************************************************************
XSLTMark 2.0 (Feb 28 2001) running on SunOS 5.8 sparc
Java Runtime Environment is Sun Microsystems Inc. 1.3.1_03
Current time is 14:17:07 EDT, 18 Jul 2002
loading test driver: XalanJDriver... Loaded Xalan2JDriver
No configurations specified, using default
Reading configuration from testcases/default.conf
dummy initialization run:
xslbench3: xslbench3.xsl xslbenchdream.xml ... done in 3160ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
benchmark run:
alphabetize: alphabetize.xsl db100.xml ... done in 52170ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
attsets: attsets.xsl chart.xml ... done in 21806ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
avts: avts.xsl db100.xml ... done in 37321ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
axis: axis.xsl axis.xml ... done in 22164ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
backwards: backwards.xsl game.xml ... done in 26767ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
bottles: bottles.xsl bottles.xml ... done in 34419ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
breadth: find.xsl breadth.xml ... done in 23475ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
brutal: brutal.xsl brutal.xml ... done in 22260ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
chart: chart.xsl chart.xml ... done in 21782ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
creation: creation.xsl db100.xml ... done in 33204ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
current: current.xsl current.xml ... done in 19765ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
dbonerow: dbonerow.xsl db10000.xml ... done in 96359ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
dbtail: dbtail.xsl db100.xml ... done in 23368ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
decoy: decoy.xsl db100.xml ... done in 56668ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
depth: find.xsl depth.xml ... done in 20518ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
encrypt: encrypt.xsl db100.xml ... done in 4607ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
functions: functions.xsl db100.xml ... done in 54484ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
game: game.xsl game.xml ... done in 18754ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
html: html.xsl html.xml ... done in 17782ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
identity: identity.xsl db1000.xml ... done in 18522ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
inventory: inventory.xsl inventory.xml ... done in 21633ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
metric: metric.xsl metric.xml ... done in 23545ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
number: number.xsl number.xml ... done in 20917ms.
oddtemplate: oddtemplate.xsl oddtemplate.xml ... done in 20029ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
patterns: patterns.xsl db100.xml ... done in 47092ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
prettyprint: prettyprint.xsl db100.xml ... done in 12658ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
priority: priority.xsl priority.xml ... done in 20193ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
products: products.xsl products.xml ... done in 21631ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
queens: queens.xsl queens.xml ... done in 5009ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
reverser: reverser.xsl gettysburg.xml ... done in 20336ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
stringsort: stringsort.xsl db1000.xml ... done in 17345ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
summarize: summarize.xsl queens.xsl ... done in 15747ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
total: total.xsl chart.xml ... done in 14948ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
tower: tower.xsl tower.xml ... done in 12956ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
trend: trend.xsl trend.xml ... done in 86370ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
union: union.xsl union.xml ... done in 16682ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
xpath: xpath.xsl xpath.xml ... done in 16278ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
xslbench1: xslbench1.xsl xslbench1.xml ... done in 16932ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
xslbench2: xslbench2.xsl xslbenchdream.xml ... done in 18297ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
xslbench3: xslbench3.xsl xslbenchdream.xml ... done in 9782ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
XSLTMark 2.0 (Feb 28 2001) running on SunOS 5.8 sparc
Java Runtime Environment is Sun Microsystems Inc. 1.3.1_03
Current time is 14:17:07 EDT, 18 Jul 2002
Results for driver Xalan2JDriver
prep exec
Test Component iter ms ms KB In KB Out KB/s status
alphabetize 100 192 52170 1966 1566 33.86 CHK OUTPUT
attsets 100 94 21806 121 255 8.65 CHK OUTPUT
avts 100 209 37321 1966 1156 41.84 CHK OUTPUT
axis 100 138 22164 35 60 2.17 CHK OUTPUT
backwards 100 100 26767 260 261 9.76 CHK OUTPUT
bottles 100 131 34419 4 1130 16.49 CHK OUTPUT
breadth 100 54 23475 415 200 13.14 CHK OUTPUT
brutal 100 143 22260 130 394 11.78 CHK OUTPUT
chart 100 59 21782 121 359 11.04 CHK OUTPUT
creation 50 36 33204 983 1520 37.7 CHK OUTPUT
current 100 268 19765 18 29 1.24 CHK OUTPUT
dbonerow 10 45 96359 19621 8 101.86 CHK OUTPUT
dbtail 100 35 23368 1966 503 52.85 CHK OUTPUT
decoy 100 187 56668 1966 1974 34.77 CHK OUTPUT
depth 100 25 20518 588 373 23.46 CHK OUTPUT
encrypt 10 109 4607 196 196 42.69 CHK OUTPUT
functions 100 160 54484 1966 618 23.72 CHK OUTPUT
game 100 56 18754 260 43 8.1 CHK OUTPUT
html 100 52 17782 29 47 2.14 CHK OUTPUT
identity 10 30 18522 1962 1962 105.96 CHK OUTPUT
inventory 100 37 21633 184 202 8.94 CHK OUTPUT
metric 100 58 23545 88 65 3.28 CHK OUTPUT
number 100 38 20917 18 79 2.35 NO REF
oddtemplate 100 266 20029 16 15 0.81 CHK OUTPUT
patterns 100 34 47092 1966 1974 41.84 CHK OUTPUT
prettyprint 10 65 12658 196 1405 63.27 CHK OUTPUT
priority 100 98 20193 91 57 3.67 CHK OUTPUT
products 100 68 21631 196 40 5.47 CHK OUTPUT
queens 10 127 5009 0 17 1.75 CHK OUTPUT
reverser 100 37 20336 129 129 6.37 CHK OUTPUT
stringsort 10 22 17345 1962 1933 112.3 CHK OUTPUT
summarize 100 178 15747 712 29 23.56 CHK OUTPUT
total 100 25 14948 121 33 5.18 CHK OUTPUT
tower 20 66 12956 1 1381 53.35 CHK OUTPUT
trend 100 126 86370 176 806 5.69 CHK OUTPUT
union 100 123 16682 16 11 0.86 CHK OUTPUT
xpath 100 25 16278 28 20 1.51 CHK OUTPUT
xslbench1 100 176 16932 36 665 20.74 CHK OUTPUT
xslbench2 20 34 18297 2920 2139 138.26 CHK OUTPUT
xslbench3 20 38 9782 2920 25 150.56 CHK OUTPUT
Aggr. Results 100 38 20917 18 79 2.35
Arithmetic Mean 38 2.35
Geometric Mean 2.35
Harmonic Mean 2.35
Conformance Score (OK+NOREF+SKIPPED divided by TOTAL) 1/40 = 2.5%
*************************************************************
XSLTC
*************************************************************
XSLTMark 2.0 (Feb 28 2001) running on SunOS 5.8 sparc
Java Runtime Environment is Sun Microsystems Inc. 1.3.1_03
Current time is 14:37:47 EDT, 18 Jul 2002
loading test driver: XSLTCDriver... Loaded XSLTCDriver
No configurations specified, using default
Reading configuration from testcases/default.conf
dummy initialization run:
xslbench3: xslbench3.xsl xslbenchdream.xml ... done in 291ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
benchmark run:
alphabetize: alphabetize.xsl db100.xml ... done in 13844ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
attsets: attsets.xsl chart.xml ... done in 1002ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
avts: avts.xsl db100.xml ... done in 1522ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
axis: axis.xsl axis.xml ... done in 741ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
backwards: backwards.xsl game.xml ... done in 1458ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
bottles: bottles.xsl bottles.xml ... done in 1433ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
breadth: find.xsl breadth.xml ... done in 703ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
brutal: brutal.xsl brutal.xml ... done in 856ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
chart: chart.xsl chart.xml ... done in 836ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
creation: creation.xsl db100.xml ... done in 2024ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
current: current.xsl current.xml ... done in 287ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
dbonerow: dbonerow.xsl db10000.xml ... done in 2115ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
dbtail: dbtail.xsl db100.xml ... done in 746ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
decoy: decoy.xsl db100.xml ... done in 5665ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
depth: find.xsl depth.xml ... done in 596ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
encrypt: encrypt.xsl db100.xml ... done in 574ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
functions: functions.xsl db100.xml ... done in 7624ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
game: game.xsl game.xml ... done in 225ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
html: html.xsl html.xml ... done in 282ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
identity: identity.xsl db1000.xml ... done in 1701ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
inventory: inventory.xsl inventory.xml ... done in 487ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
metric: metric.xsl metric.xml ... done in 519ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
number: number.xsl number.xml ... done in 557ms.
oddtemplate: oddtemplate.xsl oddtemplate.xml ... done in 239ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
patterns: patterns.xsl db100.xml ... done in 2781ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
prettyprint: prettyprint.xsl db100.xml ... done in 5573ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
priority: priority.xsl priority.xml ... done in 264ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
products: products.xsl products.xml ... done in 675ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
queens: queens.xsl queens.xml ... done in 1348ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
reverser: reverser.xsl gettysburg.xml ... done in 778ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
stringsort: stringsort.xsl db1000.xml ... done in 2653ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
summarize: summarize.xsl queens.xsl ... done in 1013ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
total: total.xsl chart.xml ... done in 146ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
tower: tower.xsl tower.xml ... done in 6095ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
trend: trend.xsl trend.xml ... done in 11740ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
union: union.xsl union.xml ... done in 304ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
xpath: xpath.xsl xpath.xml ... done in 339ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
xslbench1: xslbench1.xsl xslbench1.xml ... done in 717ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
xslbench2: xslbench2.xsl xslbenchdream.xml ... done in 1901ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
xslbench3: xslbench3.xsl xslbenchdream.xml ... done in 171ms.
dgnorm or cksum invocation failed: java.io.IOException: dgnorm: not found
XSLTMark 2.0 (Feb 28 2001) running on SunOS 5.8 sparc
Java Runtime Environment is Sun Microsystems Inc. 1.3.1_03
Current time is 14:37:47 EDT, 18 Jul 2002
Results for driver XSLTCDriver
prep exec
Test Component iter ms ms KB In KB Out KB/s status
alphabetize 100 692 13844 1966 156699 5730.47 CHK OUTPUT
attsets 100 374 1002 121 25595 12832.83 CHK OUTPUT
avts 100 294 1522 1966 115693 38652.88 CHK OUTPUT
axis 100 485 741 35 6093 4135.56 CHK OUTPUT
backwards 100 462 1458 260 26181 9067.93 CHK OUTPUT
bottles 100 335 1433 4 113046 39445.66 CHK OUTPUT
breadth 100 147 703 415 20078 14576.13 CHK OUTPUT
brutal 100 944 856 130 39990 23434.99 CHK OUTPUT
chart 100 381 836 121 40732 24434.04 CHK OUTPUT
creation 50 268 2024 983 76042 19028.04 CHK OUTPUT
current 100 687 287 18 2998 5255.91 CHK OUTPUT
dbonerow 10 277 2115 19621 89 4659.7 CHK OUTPUT
dbtail 100 358 746 1966 50390 35091.58 CHK OUTPUT
decoy 100 377 5665 1966 197460 17601.67 CHK OUTPUT
depth 100 150 596 588 37363 31839.13 CHK OUTPUT
encrypt 10 177 574 196 1967 1885.09 CHK OUTPUT
functions 100 313 7624 1966 61845 4184.92 CHK OUTPUT
game 100 202 225 260 4658 10930.34 CHK OUTPUT
html 100 563 282 29 4863 8674.26 CHK OUTPUT
identity 10 254 1701 1962 19626 6345.96 CHK OUTPUT
inventory 100 322 487 184 20205 20934.23 CHK OUTPUT
metric 100 292 519 88 6582 6426.31 CHK OUTPUT
number 100 189 557 18 7958 7161.43 NO REF
oddtemplate 100 230 239 16 1552 3283.54 CHK OUTPUT
patterns 100 394 2781 1966 197460 35855.26 CHK OUTPUT
prettyprint 10 290 5573 196 14363 1306.32 CHK OUTPUT
priority 100 379 264 91 5722 11010.93 CHK OUTPUT
products 100 361 675 196 4101 3183.52 CHK OUTPUT
queens 10 357 1348 0 172 64.24 CHK OUTPUT
reverser 100 276 778 129 12939 8399.32 CHK OUTPUT
stringsort 10 605 2653 1962 19333 4013.55 CHK OUTPUT
summarize 100 362 1013 712 2939 1802.54 CHK OUTPUT
total 100 174 146 121 3408 12087.3 CHK OUTPUT
tower 20 281 6095 1 27628 2266.54 CHK OUTPUT
trend 100 1535 11740 176 80683 3443.78 CHK OUTPUT
union 100 922 304 16 1181 1971.27 CHK OUTPUT
xpath 100 1158 339 28 2021 3024.17 CHK OUTPUT
xslbench1 100 528 717 36 72285 50433.66 CHK OUTPUT
xslbench2 20 320 1901 2920 42801 12025.66 CHK OUTPUT
xslbench3 20 273 171 2920 503 10010.28 CHK OUTPUT
Aggr. Results 100 189 557 18 7958 7161.43
Arithmetic Mean 189 7161.43
Geometric Mean 7161.43
Harmonic Mean 7161.43
--
Tom Amiro -- SQA Engineer
Sun XML Technology Development
voice: 781-442-0589 Fax: 781-442-1437
eMail: tom.amiro@.sun.com