classloader issues (ClassCastException on org.xml.sax.Parser)

[Ed Korthof <ed...@apache.org>]

Hi --

At my company, we've been using tomcat out of cvs for our development.
(We'll be releasing software which uses Tomcat 4.0 rather than
Apache-JServ shortly.)  During the last update which we did, we found
our XML-RPC service stopped working -- it started giving us
ClassCastExceptions.  Eventually, we traced it down to classloading
issues which were introduced in 1.30 of WebappClassLoader.java :


----------------------------
revision 1.30
date: 2002/01/03 16:36:09;  author: remm;  state: Exp;  lines: +88 -11

- Adds some filtering on the JARs, similar to what is done by the
  ClassLoaderFactory.  That should prevent overriding classes which
  shouldn't be overridden (and which is actually a spec requirement).
  For example, it is now possible to actually run JSPs even with an old
  servlet.jar in the /WEB-INF/lib directory.
- I think this patch should be ported to the 4.0.x branch, since it
  prevents lots of user errors.
----------------------------

(reformatting done so this will be clean in < 80 columns).

Anyway -- here's the way our problem works.  We have xml-rpc.jar in the
webapp classloader.  xml-rpc.jar needs an xml parser -- and we were
using xercex.jar for that.  However, xerces.jar has the
javax.xml.parsers.DocumentBuilder class in it -- which means that the
webapp classloader *silently* (a loud warning would be appropriate even
when debugging is off) ignores that class.

xml-rpc.jar has org.xml.sax.Parser in it -- so when classname resolution
is done for xml-rpc, it finds Parser in the WebappClassLoader.  However,
when xml-rpc looks for its XML parser, it finds xerces -- and now xerces
isn't defined in the webapp classloader, it's defined in the next level
up (the shared classloader, I believe).  That would work fine -- except
that when xerces resolve org.xml.sax.Parser, it cannot find the version
which is in the WebappClassloader -- it can only find the version which
is in the shared classloader (it's in xerces.jar).  Two classes differ
if their classloader differs, so a ClassCastException is the correct
error.

AFAICT our solutions are:
1) use a different XML parser (and ensure it doesn't have
   org.xml.parsers.DocumentBuilder in it -- which is fine only unless
   WebappClassLoader is not changed to forbid other classes which 
   XML parsers are likely to include with them)
2) move xml-rpc.jar entirely out of the WebappClassloader and put it
   in the shared class loader (this would require that we move
   everything which uses an XML parser into the shared classpath --
   definitely not ideal)
3) remove org.xml.parser.DocumentBuilder from xerces.jar (similar to #1,
   but we know in advance that we have to muck with the .jar file in
   a problematic way).
4) toggle the behavior of the WebappClassloader in server.xml by
   telling it 'delegate="true"', so it delegates to its parent first.
   
The last one of these seems to work fine for us -- and it does actually
seems like a reasonable approach.  But it won't be available to at least
some people (why else would you attempt to allow users to override
classes in the webapp classloader?) ...

I think the fundamental problem is a bad specification (allowing people
to override some classes in a child classloader -- and trying to prevent
them from loading others -- is just silly); but it could be improved in
tomcat in a couple of ways.  I haven't yet tested this, but I think you
would have *far* better protection from bad .jar files if you moved
those security checks (which are currently used to disable .jar files
entirely -- IMO, that's wrong) into loadClass -- and change them so that
they check the class name against a list of packages, rather than just
looking for a few specific packages.

That would avoid encouraging stupid user tricks (#3 up above) and
actually be rather reliable, in terms of catching classes which
shouldn't be redefined in the child classloader.  It would also have
avoided the ClassCastException which we were seeing.

Failing that -- at least tomcat should do more in the way of logging
it if it decides not to accept a given .jar file because of the classes
the .jar file contains ... and this feature should be documented in
either case.

I'm happy to provide patches for consideration (as soon as I can get a
tomcat dev environment set up -- but I think I can get some help from
some folks at CollabNet who have build environments set up).

thanks --

Ed



--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: classloader issues (ClassCastException on org.xml.sax.Parser)

[Daniel Rall <dl...@finemaltcoding.com>]

"Remy Maucherat" <re...@apache.org> writes:

> Also, you can't change the delegation and still be spec compliant. We could
> avoid implementing the requirement of preventing loading core libraries, if
> it turns out it's not implementable.

If the spec says to look in the current class loader before delegating
to the parent, then the spec is broken.  The ClassCastException we
were seeing as a result of this behavior illustrates the breakage
lucidly.

Dan

--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: classloader issues (ClassCastException on org.xml.sax.Parser)

[Ed Korthof <ed...@apache.org>]

On Wed, Jan 23, 2002 at 02:28:52AM -0800, Remy Maucherat wrote:
> > On Tue, Jan 22, 2002 at 06:14:18PM -0800, Remy Maucherat wrote:
> >
> > Do you mean you're spawning another process to do compilation?  I
> > thought that javac's core class had been fixed so that it'd be possible
> > to run it in a thread in an existing server ... for a number of reasons,
> > including performance and the awkwardness of spawning processes on
> > win32.  (I know there was talk about that years ago, when I last worked
> > on a page compilation system, but I don't know what happened with it.)
> 
> Jasper uses the javac API which doesn't spawn a process, but it still
> behaves the same way it does if you actually spawn a process.

You said the reason that we can't do what I suggested is that it would
"make JSPs behave differently from servlets" -- so I went to look at
Jasper to see how similar the behavior is right now, in terms of
classloading.  It looks like there's no guarantee at all about
consistency:

Jasper already assumes that the classpath for JSP compilation may be
different from that used in servlets.  There are a number of
inconsistencies, including at least the following:

1) jars accessed by URLs using protocols other than the file protocol
will be silently left out (JspEngineContext.java:155)

2) the order of classname resolution does not match the special logic in
WebappClassLoader, since the webapp classloader's jars aren't ever
preptended to the system classpath, they're always appended
(Compiler.java:233).  I haven't yet followed the logic all the way back
in terms of how the data gets into the context for use by JspServlet --
it is possible that the order is reversed somewhere else ... but if so,
the logic isn't in WebappClassLoader, which is the only class which will
know if delegation is true or false.  So I suspect it's not being done.

Note that #2 may mean that excluding the jars from the classpath to
Jasper might not be such a big deal, as the only reason for the
filtering of those jars in WebappClassloader is the fact that they could
override classes defined in their parent classloaders.

I can imagine that there may be other good reasons not to allow the
changes which I suggested, but AFAICT there's very little consistency
left to be preserved, when comparing classpath handling for JSP pages
and servlets.

> Also, you can't change the delegation and still be spec compliant. We could
> avoid implementing the requirement of preventing loading core libraries, if
> it turns out it's not implementable.

I don't mind configuring tomcat so that my install isn't spec compliant
(that's already possible, with the setDelgate toggle).  I tend to feel
as Daniel said -- the spec is broken.  But I can understand wanting to
build tomcat in such a way that it satisfies the spec as well as is
possible.

It is possible to be compliant with the specification, but I don't think
the current implementation is.  I can override classes in each of the
"protected" packages, so long as I take care to leave out the trigger
class, for which tomcat looks, in my .jar files -- so if it's tomcat's
job to avoid loading core libraries in the webapp classloaders, I don't
think it's doing it reliably.

Anyway -- one possibility would be to do filtering of .jars before
handing them to the JSP engine (much as it's being done now, thought it
might make more sense to read through the contents of the jars and look
for any classes which are in packages they're not allowed to be in), but
to do the kind of filtering which I described in my first message, in
loadClass.  And I'd suggest adding one or more toggles, to configure how
filtering is done.  (I'm not sure if one or two is appropriate because
I'm not sure if #2 -- up above -- should be fixed.)

thanks --

Ed

--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: classloader issues (ClassCastException on org.xml.sax.Parser)

[Remy Maucherat <re...@apache.org>]

> On Tue, Jan 22, 2002 at 06:14:18PM -0800, Remy Maucherat wrote:
>
> Do you mean you're spawning another process to do compilation?  I
> thought that javac's core class had been fixed so that it'd be possible
> to run it in a thread in an existing server ... for a number of reasons,
> including performance and the awkwardness of spawning processes on
> win32.  (I know there was talk about that years ago, when I last worked
> on a page compilation system, but I don't know what happened with it.)

Jasper uses the javac API which doesn't spawn a process, but it still
behaves the same way it does if you actually spawn a process.

Also, you can't change the delegation and still be spec compliant. We could
avoid implementing the requirement of preventing loading core libraries, if
it turns out it's not implementable.

Remy


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: classloader issues (ClassCastException on org.xml.sax.Parser)

[Ed Korthof <ed...@apache.org>]

On Tue, Jan 22, 2002 at 06:14:18PM -0800, Remy Maucherat wrote:
> > Hi --
> >
> > I think the fundamental problem is a bad specification (allowing people
> > to override some classes in a child classloader -- and trying to prevent
> > them from loading others -- is just silly); but it could be improved in
> > tomcat in a couple of ways.  I haven't yet tested this, but I think you
> > would have *far* better protection from bad .jar files if you moved
> > those security checks (which are currently used to disable .jar files
> > entirely -- IMO, that's wrong) into loadClass -- and change them so that
> > they check the class name against a list of packages, rather than just
> > looking for a few specific packages.
> 
> Unfortunately, you can't do that, since JSP uses javac to compile the pages.
> javac accesses JARs as a whole, so either you have to filter out the whole
> JAR or you allow Jasper to load all the classes it contains, which will make
> JSPs behave differently from servlets (bad).
> It's a complex issue, so I really don't know what to do ...

Do you mean you're spawning another process to do compilation?  I
thought that javac's core class had been fixed so that it'd be possible
to run it in a thread in an existing server ... for a number of reasons,
including performance and the awkwardness of spawning processes on
win32.  (I know there was talk about that years ago, when I last worked
on a page compilation system, but I don't know what happened with it.)

I think it would be hard to avoid having javac work different from
servlets if it runs in a separate process and it doesn't use a
classloader like the webapp classloader -- unless you change the order
of the jar files iff delegate is set to false.  (AFAICT
WebappClassloader will delegate to the parent classloader regardless --
the question is just if it does so before or after looking at the local
repositories, which is what setDelegate toggles.)  Is this being done?

thanks --

Ed



--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>

Re: classloader issues (ClassCastException on org.xml.sax.Parser)

[Remy Maucherat <re...@apache.org>]

> Hi --
>
> I think the fundamental problem is a bad specification (allowing people
> to override some classes in a child classloader -- and trying to prevent
> them from loading others -- is just silly); but it could be improved in
> tomcat in a couple of ways.  I haven't yet tested this, but I think you
> would have *far* better protection from bad .jar files if you moved
> those security checks (which are currently used to disable .jar files
> entirely -- IMO, that's wrong) into loadClass -- and change them so that
> they check the class name against a list of packages, rather than just
> looking for a few specific packages.

Unfortunately, you can't do that, since JSP uses javac to compile the pages.
javac accesses JARs as a whole, so either you have to filter out the whole
JAR or you allow Jasper to load all the classes it contains, which will make
JSPs behave differently from servlets (bad).
It's a complex issue, so I really don't know what to do ...

Remy


--
To unsubscribe, e-mail:   <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>