[XALANJ2] interesting bench firgures

[Sebastien Sahuc <ss...@imediation.com>]

OK, I know Scott is not willing to stress the current Xalan2. But I thought
I might share with you some interesting figures I got. 

Basically I wanted to check if the difference from Xalan and XT in term of
performance was somewhat linked to the XML parser used. Therefore I did the
test with the JDK1.2.2 + Hotspot JVM 2.0 (win32) on purpose since I believe
most of the usage will be server side (at least for me :-) ).

----------------------------------------------------------------------------
--------
|  Combination       |    Warming up (200 first loops) | running (200 last
loops)  |
|--------------------|---------------------------------|--------------------
-------|
| Xalan2 + Aelfred   |    104 ms / loop                | 23 ms / loop
|
|--------------------|---------------------------------|--------------------
-------|
| Xalan2 + Xerces    |    110 ms / loop                | 32 ms / loop
|
|--------------------|---------------------------------|--------------------
-------|
| Xalan1.2 + Xerces  |    75 ms / loop                 | 14 ms / loop
|
|--------------------|---------------------------------|--------------------
-------|
| XT + Xerces        |    91  ms / loop                | 15 ms / loop
|
|--------------------|---------------------------------|--------------------
-------|
| XT + JAXP (sun)    |    36  ms / loop                | 13 ms / loop
|
|--------------------|---------------------------------|--------------------
-------|
| XT + Aelfred       |    37  ms / loop                | 16 ms / loop
|
----------------------------------------------------------------------------
--------

Each line is a combination of XSL processor + XMLprocessor. 

Each program starts by caching once the style sheet, and then it performs
the 400 loops before finishing.

There is two columns for results :
* One for the result from warming up. Indeed Hotspot server needs enough
loops to find out what are precisely the hot spots in the code that need to
be compiled. Therefore we can be sure that after these 200 loops of warming
up, the JVM has finished compiling hot spots.
* Then the process go into another 200 loops, and an average figure is
calculated. The corresponding figure show how fast the XSLT transformation
is performed in a Server side context.


So what can we conclude from this report :

* That you can build benchmark that best suits to your need :-) Indeed in
the present benchmark we are looping hundred times before getting the first
stable results

* That Xerces 1.3 doesn't perform so well in the Hotspot environment (but
the Xerces team already know that, and they'll take into account the fact
for Xerces 2). Indeed Xerces 1.3 is fully optimized for 1.1 JVM with JIT
compiler. Therefore the code is too complex for the Hotspot JVM to find out
quickly what the Hot spots are (and the code is also damn complex for
human).

* On the other side, the Aelfred parser and the parser from sun (former
Project-X, currently shiped with JAXP), have not been optimized, so a slower
that Xerces in 1.1 JVM. But more efficient in Server side environment when
used with HotSpot server JVM. Indeed these Parsers haven't been optimized
yet, therefore the code is much simpler to understand and easier optimize at
runtime.

* Concerning the XSL processor, we can end up with the same conclusion.
Indeed we need to wait for hundred loops before getting homogenized results
between different XSL transformer. Of course Xalan2 isn't optimized at all.
And warmimg up Xalan2 is longer than the other, but it's not too bad at the
end. So we can definitely trust Xalan team for building a world class XSLT
Processor...

All the best,

Sebastien

PS : For who ever needs the script and the sources of the bench, I'll be
more that happy to send it over.