Xalan 2.0.0 Source Tree Checked In

[Scott Boag/CAM/Lotus <Sc...@lotus.com>]

The Xalan 2.0.0 source tree is checked into the CVS repository.  Rather
than making a branch, it has been checked into the xml-xalan\java
directory, where we want to place all java code in the future. (The Xalan
1.x codebase is currently in the xml-xalan\src directory, which is
different from Xerces, so we want to fix this.)

First, normal users who just want to run XSLT transformations should not
run this stuff yet.  Second, please note that this is a work in progress
(which is somewhat behind schedule from where I wanted to be by now).  As
you can see below, we already have a long laundry list, so it's probably
not productive to report bugs yet.  I do want to hear design critiques, and
general brainstorming.

Please see http://xml.apache.org/xalan/design/design1_1_0.html for the
general design, though note that this document needs to be updated.  In
particular, it doesn't reflect the new XPath module redesign.  (It also
refers to Xalan 2.0.0 as Xalan 1.1.0).

Features:

1) TRaX interfaces.  (However, see laundry list below).
2) Build of stylesheet is separated from the template's classes, more like
a compiler.  It is table-driven, by an XSLT schema of sorts.  More stuff is
being done at compile time vs. run time.
3) The transformation package is now properly separated from the templates
package.  The templates are much more Java-Bean like now.
4) In general, many packages have been flattened out to the top-level
directory.
5) The XPath engine is undergoing radical redesign.  The opcode lists will
be used as an intermediate code for generating optimized runtime class
instances.  The location paths now use DOM2 iterators, and DOM2 Tree
walkers for the steps, so that searches can be totally incremental.  After
much pain, I have achieved depth-first document order searches without
peek-ahead, though this still needs optimizations added.
6) The default DOM is now STree (for Source Tree or SAX Tree, take your
pick), which is an incrementally SAX-built, fairly low overhead DOM, with
indexing capabilities.  [The DTM still works (though I haven't been testing
on it), and we'll do improvements on it down the line.]

Xalan 2.0.0 is currently NOT backwards compatible, in terms of interfaces,
with Xalan 1.x.x.  We'll try to provide some backwards compatibility down
the line.

Status with our test suite:
Warnings: 9
Non-conformance: 187
Crashes: 87
Success: 795

Most of crashes and non-conformance issues are fairly trivial, and should
be fixed in the next couple of weeks.

No performance testing has been done yet, and we won't begin doing this for
a couple weeks, after the XPath optimizations are put in (right now Xalan
2.0.0 is likely to be slower than Xalan 1.x.x... but this will change
fairly quickly).

Laundry list (in order of priority):
0) Fix remaining crashes and performance issues.
1) Ant builds (right now it only has makefiles).
2) Get Assaf to implement TRaX serializer interfaces, so I can point to
there, instead of Xerces serializers (I think this is just a package name
change).  Also, the TRaX interfaces contained in the Xalan 2.0.0 code have
changed a bit from what's in the openxml repository -- I haven't been able
to get past our firewall to check them in.
3) Get extensions working again.  This is related to some work going on in
the W3C XSL Working Group, so I need to pop this up on the priorty queue.
4) Rework XPath so that it uses the opcodes for intermediate code, and
generates fast runtime classes.  Put operations methods into xpath.ops
package.
7) Get STree incrementality working (right now it waits 'til doc is
complete)... on seperate thread?  ParseNext?  Both via option?
5) Optimize walkers so that they don't do traversal predicate checks when
location steps do not overlap.  Also, restructure patterns such as "//foo".
6) Do object pool for both LocPathIterators and AxesWalkers.  This is
absolutely critical for fast performance, since they hold state.
7) Work on error handling, which is currently not consistent.  State of
TRaX confuses matters.  More thinking to do...
8) Derive AxesWalkers to do STree fast index lookup.
9) Do caching of found values in KeyIterator (since xsl:keys are now
incrementally processed).
10) Test DTM.  Test generic DOM.
11) Work on SourceTreeManager so it can release documents if they are not
used.
12) Work on backwards compatibility interfaces.
13) XPointer interface!!!! (including range functionality)  ** Please,
please, someone volunteer for this. **
14) Schema-aware walkers in Stree, so that we can take advantage of knowing
if an element can occur within a subtree or not.
15) XSLT-level XPath analysis - common sub-expression elimination,
streaming analysis, possible template pre-selection.  (Other compiler-like
optimizations?)
16) Schema-aware extensions -- predicates and formatting functions.
17) Compile stylesheet into stand-alone class, which means making much of
the XPath runtime, templates package, and transformer package, be separable
from the rest of the engine (i.e. the processor and XPath parser).  This
can be done partly via an XSLT stylesheet that processes a stylesheet to
create the class source code.
18) PAX (Patterns for XML) interface for the XPath stuff.

I would LOVE to get help on this stuff.  Especially in the area of the
extension mechanism (which should be shared with the Cocoon XSP stuff,
IMHO), the schema extensions (!), optimization analysis, XPointer API (!),
Stree schema knowledge, and general design ideas.  Please get involved!!!

-scott