[report] Classloading problems between Catalina and Cocoon

[Stefano Mazzocchi <st...@apache.org>]

[sorry for the cross post, but all of you should be aware of this]

Cocoon is a devilishly complex java application for mainly three
reasons:

 1) it uses a huge number of dependencies to external APIs and
applications
 2) it recompiles parts of itself at startup for optimization
 3) by design, it wants to be a completely portable servlet

these three points make it nearly impossible to design such a thing
today, namely because:

today's java compilation technology stinks!

It doesn't rely on classloading but directly on classpaths, which are
simply *ONE* of the possible implementation of classloading. So, there
is no way to say to the java compiler "hey, compile this stream I pass
you importing the classes available on your classloader". Too damn easy:
you need to stream the source on a file, then add the file to the
classpath, then instantiate the javac classes passing an array of
parameters.

All right, it was designed for command line, but is it so hard to shape
it into something more modern than 30-year-old command-line stuff?

Pier and I started talking about a JSR for Java Compilation API months
ago and I even wrote a JSR-ignition document but the 'javac' team sucked
it, well, I don't know anything about it.

So, 6 months later, we are still with the same stinking problem of
having to pass classpaths to the compiler.

Ok, those of you who worked with classloaders know that classpaths are
poor-man implementation of the beautiful classloading concept. But now
you are required to transform a beautiful concept into a stupid one and
try to provide an 'classpath' equivalence of your classloading
capabilities.

Imagine having a classloader that remotely connects to a Jini-based
storage system with hot-swap capabilities and all that, tell me, how can
I compile my server page so that it includes that class loaded from my
jini-classloader?

No shit there is no Java CPAN and making one is a pain in the ass: all
stinking classes need to be there on your disk at compilation time, no
way around it (yes, folks, I even tried to decompile the whole javac to
see if I could take parts out and hook the internals, but nothing, no
way! the design faults are deep into the very core of the compiler for
what I could tell)

Sorry for the rant, but grrr....

Anyway, we are left with this decades-old concept of classpath and we
must keep it for now.

Ok, so, hopefully, there is a standard way to access the classpath:
yeah, a system property, but guess what? each servlet engine will
probably sandbox you into your own classloader which has probably
nothing to do with the system classpath (in fact, the system classpath
should be empty or more or less so for a good system!)

Catalina does exactly this.

But Jasper works, right? yeah, it does, at the expense of not being
portable across servlet engines: it uses a catalina-specific context
attribute to obtain its own classloader.

Cocoon does the same, hoping that either the Servlet API expert group
comes up with a standard attribute for context classpath (which is a
stupid idea since it will need to be deprecated in the future when
better compilation technologies emerge, but solves the problem for now),
or somebody writes a "really-modern" java compiler (I sware, I might
well start writing one when I have some time!)

Ok, so far so good, the problem is that Catalina expects that JSP are
such a great thing that all web applications will want to use them. So,
instead of placing jasper.jar into a WEB-INF/lib where it should reside
if there was parity for technologies, it resides on the catalina main
library /lib and gets 'spoiled' by this fact.

Cocoon doesn't pretend to be "the" server pages solution and more
modestly reside on its own WEB-INF/lib. Great but the problem is that
Catalina tells us our classpath based on our WEB-INF/lib, but forgets to
tell us where the Servlet API is! Since it's not part of the system
classloader because Catalina is bootstrapped, javac is *NOT* able to
compile because it doesn't find the servlets.

Easy, you say: add servlet.jar to your WEB-INF/lib. WRONG! the same
class loaded on different classloaders is considered a different class,
so Cocoon extends javax.servlet.Servlet but *MY* version not Catalina's!
Thus, Catalina pukes "you're  not a servlet!" and fails to run us.

But we are not finished.

Even if you add "servlet.jar" to Catalina system classpath, another big
problem remains: sealed jar files.

The Tomcat people are fully aware of this, but I tried to see what
caused the problems with Cocoon in particular.

So, it results that if you remove "crimson.jar" from Catalina's /lib it
works! Why so? Because crimson.jar is sealed.

Now, I thought, what the hell? I'm not touching the /lib directory and
I'm fully happy with crimson.jar being the default JAXP parser for all
the webapps hosted, but why should this limit my ability to add *MY OWN*
JAXP parser to my own library?

Ok, so, if it's a sealing problem, we just have to 'unseal' crimson. So
I 'unsealed' it by extracting the files, modifying the manifest by hand,
and repackaging the jar. Result: same exception!!! How about that?

So I checked *all* jars and only JAXP.jar is sealed, but when I remove
'crimson.jar' and leave JAXP.jar there, Cocoon works just fine (unlike
JSPs, of course, since they can't find the DOM API which is included in
Crimson).

So, I placed xerces instead of crimson and, then again, 'sealing
violation exception'.

So, I unsealed JAXP.jar and BANG, it worked.

Solution: unseal both JAXP.jar and crimson.jar to avoid security
exceptions and allow webapps to provide their own implementations for
parsers and everything else.

The questions results pretty obvious: if unsealing a package is a matter
of removing a "sealing: true" in a manifest and creates all these
problems, why bother?

Well, mysteries of life.

Stay tuned for the next episode of "living on the bleeding edge". Coming
up next: "where when will XXX be out?" a sci-fi production where
software development is always on time and goes smoother the more
project dependencies you add. :)

What a great way to waste an afternoon, anyway... NOT!

-- 
Stefano Mazzocchi      One must still have chaos in oneself to be
                          able to give birth to a dancing star.
<st...@apache.org>                             Friedrich Nietzsche
--------------------------------------------------------------------