You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@xalan.apache.org by Stefano Mazzocchi <st...@apache.org> on 2002/02/16 17:58:20 UTC
some XSLT benchmarks
[crossposting on xalan-dev since they might be interested in these
results]
I want to have numbers to know whether or not a native implementation of
an XSLT transformer for Cocoon would make sense, so I did some
benchmarks.
I used XSLTMark and rerun the test on my machine (old laptop, but
anyway), here are the results, I'll commment them below.
Hardware
========
Sony VAIO PCG-F280
Intel Pentium II 366 Mhz
128 Mb RAM
Microsoft Windows 2000 - Service Pack 3 [1]
Xalan-J (built into JDK 1.4, uses crimson)
-------
Sun 1.4.0 (client) 108,73
Xalan-J 2.3 + Xerces 2.0 (latest and greatest)
-------
IBM 1.3.0 74,80
Sun 1.3.1_02 56,45
Sun 1.4.0 (client) 55,11
Xalan-J 2.3 + Crimson
---------------------
IBM 1.3.0 85,27
Sun 1.4.0 (client) 76,02[2]
Xalan-J 2.2.0 + Xerces 1.4.4 (shipped with Cocoon 2.0.1)
------------------------------
IBM 1.3.0 85,27
Sun 1.3.1_02 61,08
Sun 1.4.0 (client) 57,10
Saxon 6.5.1
-----------
IBM 1.3.0 147,78
Sun 1.3.1_02 120,99
Sun 1.4.0 (client) 103,58
Sun 1.4.0 (server) 79,52
XT + XP
-------
IBM 1.3.0 251,62
Sun 1.3.1_02 244,64
Sun 1.4.0 (client) 229,44
Sun 1.4.0 (server) 171,26
MSXML
-----
(native) 322,95
FastXML (http://www.geocities/fastxml/)
---------------------------------------
(native) 1056,99[2]
NOTES:
[1] all programs in the tray bar or background were removed, ethernet
disconnected, and mouse left untouched during the runs.
[2] this was obtained by moving 'xalan.jar' into the
java_home/lib/endorsed
directory, but without moving 'xerces-impl.jar' so that the default SAX
handler is the one shipped with the JDK 1.4 (which is crimson).
[2] this processor is an half-baked win32-only version which was
optimized
by hand for the x86 architecture and had some parsing and compliance
errors
(XSLTMark reports 70% compliance, against 94% for MSXML and 100% for
saxon and
Xalan)
---------------------------------------------------------------------
A few things to note:
1) Xalan is slow. Sorry people, but almost every serious parser beats
Xalan in terms of pure throughtput. The fact that XT (abandoned by
years) is still *way* faster than Xalan (twice as fast, normally)
worries me very much.
WARNING: I've run the tests as the guys in XSLTMark wrote them, so it is
entirely possible they didn't use optimization tricks. If you Xalan
folks know some and want to rewrite the Xalan driver for XSLTMark, I'm
sure everyone here will appreciate that.
2) XT is still spectacularly fast, even if abandoned a long time ago (I
used the source from www.jclark.com not the ones from 4xt.org). Cocoon
will continue to ship it as long as Xalan can match its speed (and I
still can't imagine why it doesn't!)
3) Xerces 2.0 appears slower than Xerces 1.4.4, and crimson appears
faster than Xerces in almost every case.
4) MSXML is faster than XT, but it't not using a SAX interface, so, as
Berin suggested, connecting a native SAX handler might waste the benefit
of native speed thru the JNI connection since lots of methods must be
called.
I think Ovidiu is right saying that we should concentrate on having a
faster (lighter?) and even non-completely-compliant XSLT processor that
is focused on speed.
5) i haven't tested Xalan Translets, which, along with compiled XML
might be *the* way to go for Cocoon production environments: when you
need speed, you compile your XML files (so get rid of the parsing stage)
and compile the XSLT into translets (then we could easily wrap them and
make them cocoon transformers) and we could be ready to kick ass.... but
there was no XSLTMark driver for the Xalan translets (only for old Sun's
XSLTC) and I didn't have time to write one myself (hint hint).
6) the FastXML parser is real, but I think it's cheating: writing an
XSLT processor in assembly is a total waste of time, with the price of
development compared with the price of hardware. Anyway, I asked the
author (who now works for Microsoft.. guess why :) if she cared to open
source it and donate the code to us. I'll let you know if anything
happens here.
At the end: we need faster XSLT processing and I'm sorry but Xalan
doesn't seem to be leading the way as I would like it to do :/
Flamesuit on, fire at need.
--
Stefano Mazzocchi One must still have chaos in oneself to be
able to give birth to a dancing star.
<st...@apache.org> Friedrich Nietzsche
--------------------------------------------------------------------
Re: some XSLT benchmarks
Posted by Torsten Curdt <tc...@dff.st>.
On Mon, 18 Feb 2002, Jacek R. Ambroziak wrote:
> Took me a while to recreate XSLTC driver for XSLTMark
> but it appears to be working now. Here are the results:
>
> Dell Inspiron 8000
> Intel Pentium III (Coppermine) 900 MHz
> 256 MB RAM
> Red Hat Linux 7.1, kernel 2.4.16
>
> "magic numbers" (w/ test 'dbonerow' disabled)
>
> XSLTC 493
> XT 377
> Saxon 235
> XalanJ 165
>
> ('dbonerow' enabled; 2 iterations)
>
> XT 452
> XSLTC 359
> Saxon 253
> XalanJ 168
>
> Today's implementation of XSLTC handles the 'dbonerow' really badly.
> What it gains elsewhere it loses on this benchmark.
>
> Otherwise, the results are not bad for XSLTC :-)
> [and remember: translets run on small devices]
>
> I am curious how Stefano's numbers will look like.
...this looks promising :) That's factor 2-3 faster...
--
Torsten
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: some XSLT benchmarks
Posted by "Jacek R. Ambroziak" <ja...@yahoo.com>.
Took me a while to recreate XSLTC driver for XSLTMark
but it appears to be working now. Here are the results:
Dell Inspiron 8000
Intel Pentium III (Coppermine) 900 MHz
256 MB RAM
Red Hat Linux 7.1, kernel 2.4.16
"magic numbers" (w/ test 'dbonerow' disabled)
XSLTC 493
XT 377
Saxon 235
XalanJ 165
('dbonerow' enabled; 2 iterations)
XT 452
XSLTC 359
Saxon 253
XalanJ 168
Today's implementation of XSLTC handles the 'dbonerow' really badly.
What it gains elsewhere it loses on this benchmark.
Otherwise, the results are not bad for XSLTC :-)
[and remember: translets run on small devices]
I am curious how Stefano's numbers will look like.
--Jacek
On Saturday 16 February 2002 11:58 am, Stefano Mazzocchi wrote:
> [crossposting on xalan-dev since they might be interested in these
> results]
>
> I want to have numbers to know whether or not a native implementation of
> an XSLT transformer for Cocoon would make sense, so I did some
> benchmarks.
>
> I used XSLTMark and rerun the test on my machine (old laptop, but
> anyway), here are the results, I'll commment them below.
>
> Hardware
> ========
>
> Sony VAIO PCG-F280
> Intel Pentium II 366 Mhz
> 128 Mb RAM
> Microsoft Windows 2000 - Service Pack 3 [1]
>
>
>
> Xalan-J (built into JDK 1.4, uses crimson)
> -------
>
> Sun 1.4.0 (client) 108,73
>
> Xalan-J 2.3 + Xerces 2.0 (latest and greatest)
> -------
>
> IBM 1.3.0 74,80
> Sun 1.3.1_02 56,45
> Sun 1.4.0 (client) 55,11
>
> Xalan-J 2.3 + Crimson
> ---------------------
>
> IBM 1.3.0 85,27
> Sun 1.4.0 (client) 76,02[2]
>
> Xalan-J 2.2.0 + Xerces 1.4.4 (shipped with Cocoon 2.0.1)
> ------------------------------
>
> IBM 1.3.0 85,27
> Sun 1.3.1_02 61,08
> Sun 1.4.0 (client) 57,10
>
> Saxon 6.5.1
> -----------
>
> IBM 1.3.0 147,78
> Sun 1.3.1_02 120,99
> Sun 1.4.0 (client) 103,58
> Sun 1.4.0 (server) 79,52
>
> XT + XP
> -------
>
> IBM 1.3.0 251,62
> Sun 1.3.1_02 244,64
> Sun 1.4.0 (client) 229,44
> Sun 1.4.0 (server) 171,26
>
>
> MSXML
> -----
>
> (native) 322,95
>
>
> FastXML (http://www.geocities/fastxml/)
> ---------------------------------------
>
> (native) 1056,99[2]
>
>
> NOTES:
>
> [1] all programs in the tray bar or background were removed, ethernet
> disconnected, and mouse left untouched during the runs.
>
> [2] this was obtained by moving 'xalan.jar' into the
> java_home/lib/endorsed
> directory, but without moving 'xerces-impl.jar' so that the default SAX
> handler is the one shipped with the JDK 1.4 (which is crimson).
>
> [2] this processor is an half-baked win32-only version which was
> optimized
> by hand for the x86 architecture and had some parsing and compliance
> errors
> (XSLTMark reports 70% compliance, against 94% for MSXML and 100% for
> saxon and
> Xalan)
>
> ---------------------------------------------------------------------
>
> A few things to note:
>
> 1) Xalan is slow. Sorry people, but almost every serious parser beats
> Xalan in terms of pure throughtput. The fact that XT (abandoned by
> years) is still *way* faster than Xalan (twice as fast, normally)
> worries me very much.
>
> WARNING: I've run the tests as the guys in XSLTMark wrote them, so it is
> entirely possible they didn't use optimization tricks. If you Xalan
> folks know some and want to rewrite the Xalan driver for XSLTMark, I'm
> sure everyone here will appreciate that.
>
> 2) XT is still spectacularly fast, even if abandoned a long time ago (I
> used the source from www.jclark.com not the ones from 4xt.org). Cocoon
> will continue to ship it as long as Xalan can match its speed (and I
> still can't imagine why it doesn't!)
>
> 3) Xerces 2.0 appears slower than Xerces 1.4.4, and crimson appears
> faster than Xerces in almost every case.
>
> 4) MSXML is faster than XT, but it't not using a SAX interface, so, as
> Berin suggested, connecting a native SAX handler might waste the benefit
> of native speed thru the JNI connection since lots of methods must be
> called.
>
> I think Ovidiu is right saying that we should concentrate on having a
> faster (lighter?) and even non-completely-compliant XSLT processor that
> is focused on speed.
>
> 5) i haven't tested Xalan Translets, which, along with compiled XML
> might be *the* way to go for Cocoon production environments: when you
> need speed, you compile your XML files (so get rid of the parsing stage)
> and compile the XSLT into translets (then we could easily wrap them and
> make them cocoon transformers) and we could be ready to kick ass.... but
> there was no XSLTMark driver for the Xalan translets (only for old Sun's
> XSLTC) and I didn't have time to write one myself (hint hint).
>
> 6) the FastXML parser is real, but I think it's cheating: writing an
> XSLT processor in assembly is a total waste of time, with the price of
> development compared with the price of hardware. Anyway, I asked the
> author (who now works for Microsoft.. guess why :) if she cared to open
> source it and donate the code to us. I'll let you know if anything
> happens here.
>
> At the end: we need faster XSLT processing and I'm sorry but Xalan
> doesn't seem to be leading the way as I would like it to do :/
>
> Flamesuit on, fire at need.
_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com
Re: some XSLT benchmarks
Posted by "Jacek R. Ambroziak" <ja...@yahoo.com>.
Took me a while to recreate XSLTC driver for XSLTMark
but it appears to be working now. Here are the results:
Dell Inspiron 8000
Intel Pentium III (Coppermine) 900 MHz
256 MB RAM
Red Hat Linux 7.1, kernel 2.4.16
"magic numbers" (w/ test 'dbonerow' disabled)
XSLTC 493
XT 377
Saxon 235
XalanJ 165
('dbonerow' enabled; 2 iterations)
XT 452
XSLTC 359
Saxon 253
XalanJ 168
Today's implementation of XSLTC handles the 'dbonerow' really badly.
What it gains elsewhere it loses on this benchmark.
Otherwise, the results are not bad for XSLTC :-)
[and remember: translets run on small devices]
I am curious how Stefano's numbers will look like.
--Jacek
On Saturday 16 February 2002 11:58 am, Stefano Mazzocchi wrote:
> [crossposting on xalan-dev since they might be interested in these
> results]
>
> I want to have numbers to know whether or not a native implementation of
> an XSLT transformer for Cocoon would make sense, so I did some
> benchmarks.
>
> I used XSLTMark and rerun the test on my machine (old laptop, but
> anyway), here are the results, I'll commment them below.
>
> Hardware
> ========
>
> Sony VAIO PCG-F280
> Intel Pentium II 366 Mhz
> 128 Mb RAM
> Microsoft Windows 2000 - Service Pack 3 [1]
>
>
>
> Xalan-J (built into JDK 1.4, uses crimson)
> -------
>
> Sun 1.4.0 (client) 108,73
>
> Xalan-J 2.3 + Xerces 2.0 (latest and greatest)
> -------
>
> IBM 1.3.0 74,80
> Sun 1.3.1_02 56,45
> Sun 1.4.0 (client) 55,11
>
> Xalan-J 2.3 + Crimson
> ---------------------
>
> IBM 1.3.0 85,27
> Sun 1.4.0 (client) 76,02[2]
>
> Xalan-J 2.2.0 + Xerces 1.4.4 (shipped with Cocoon 2.0.1)
> ------------------------------
>
> IBM 1.3.0 85,27
> Sun 1.3.1_02 61,08
> Sun 1.4.0 (client) 57,10
>
> Saxon 6.5.1
> -----------
>
> IBM 1.3.0 147,78
> Sun 1.3.1_02 120,99
> Sun 1.4.0 (client) 103,58
> Sun 1.4.0 (server) 79,52
>
> XT + XP
> -------
>
> IBM 1.3.0 251,62
> Sun 1.3.1_02 244,64
> Sun 1.4.0 (client) 229,44
> Sun 1.4.0 (server) 171,26
>
>
> MSXML
> -----
>
> (native) 322,95
>
>
> FastXML (http://www.geocities/fastxml/)
> ---------------------------------------
>
> (native) 1056,99[2]
>
>
> NOTES:
>
> [1] all programs in the tray bar or background were removed, ethernet
> disconnected, and mouse left untouched during the runs.
>
> [2] this was obtained by moving 'xalan.jar' into the
> java_home/lib/endorsed
> directory, but without moving 'xerces-impl.jar' so that the default SAX
> handler is the one shipped with the JDK 1.4 (which is crimson).
>
> [2] this processor is an half-baked win32-only version which was
> optimized
> by hand for the x86 architecture and had some parsing and compliance
> errors
> (XSLTMark reports 70% compliance, against 94% for MSXML and 100% for
> saxon and
> Xalan)
>
> ---------------------------------------------------------------------
>
> A few things to note:
>
> 1) Xalan is slow. Sorry people, but almost every serious parser beats
> Xalan in terms of pure throughtput. The fact that XT (abandoned by
> years) is still *way* faster than Xalan (twice as fast, normally)
> worries me very much.
>
> WARNING: I've run the tests as the guys in XSLTMark wrote them, so it is
> entirely possible they didn't use optimization tricks. If you Xalan
> folks know some and want to rewrite the Xalan driver for XSLTMark, I'm
> sure everyone here will appreciate that.
>
> 2) XT is still spectacularly fast, even if abandoned a long time ago (I
> used the source from www.jclark.com not the ones from 4xt.org). Cocoon
> will continue to ship it as long as Xalan can match its speed (and I
> still can't imagine why it doesn't!)
>
> 3) Xerces 2.0 appears slower than Xerces 1.4.4, and crimson appears
> faster than Xerces in almost every case.
>
> 4) MSXML is faster than XT, but it't not using a SAX interface, so, as
> Berin suggested, connecting a native SAX handler might waste the benefit
> of native speed thru the JNI connection since lots of methods must be
> called.
>
> I think Ovidiu is right saying that we should concentrate on having a
> faster (lighter?) and even non-completely-compliant XSLT processor that
> is focused on speed.
>
> 5) i haven't tested Xalan Translets, which, along with compiled XML
> might be *the* way to go for Cocoon production environments: when you
> need speed, you compile your XML files (so get rid of the parsing stage)
> and compile the XSLT into translets (then we could easily wrap them and
> make them cocoon transformers) and we could be ready to kick ass.... but
> there was no XSLTMark driver for the Xalan translets (only for old Sun's
> XSLTC) and I didn't have time to write one myself (hint hint).
>
> 6) the FastXML parser is real, but I think it's cheating: writing an
> XSLT processor in assembly is a total waste of time, with the price of
> development compared with the price of hardware. Anyway, I asked the
> author (who now works for Microsoft.. guess why :) if she cared to open
> source it and donate the code to us. I'll let you know if anything
> happens here.
>
> At the end: we need faster XSLT processing and I'm sorry but Xalan
> doesn't seem to be leading the way as I would like it to do :/
>
> Flamesuit on, fire at need.
_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org