Still have XML loading problems

[Kevin Jones <ke...@develop.com>]

Craig,

I'm playing with the 22nd March drop of Catalina, and I've come across a
scenario where the new classloading architecture doesn't quite work.

I'm using Xerces and Xalan (although Xalan is irrelevant to this). If I
access a servlet that uses XML and I don't put Xerces in my
web-app/myapp/lib my code fails (exactly what I'd expect). If I put
Xerces.jar into web-app/myapp/lib it works, again what I'd expect.

However, if I put my XML code into an application listener then Jasper fails
to load - I get a Security Exception, sealing violation, while it's load the
'jsp' servlet. It seems that what's happening is that my listener loads,
loads Xerces and executes OK. The jsp Servlet then tries to load
crimson/JAXP and *bang* sealing violation.

Sorry about this - the XML stuff had been going so well until then!

Kevin Jones
DevelopMentor
www.develop.com

RE: Still have XML loading problems

[Kevin Jones <ke...@develop.com>]

> The Jasper page compiler needs to have access to all of the bean classes
> that you might have placed in WEB-INF/classes and WEB-INF/lib.  This could
> be "faked" by setting up yet another class loader that simulates the
> "real" webapp class loader (in terms of what repositories it gives you
> visibility to), or by making the special Jasper class loader a sibling of
> the webapp class loader (rather than a child), and adding the WEB-INF
> repositories to it as well.
>
> This will work as long as you don't create an object in the Jasper class
> loader and then expect it to be visible in the webapp class loader (or
> vice versa) -- you get class cast exceptions at that point.

Craig, can we do this for now - it will solve these problems in the short
term and we can then think about re-writing Jasper?

Kevin Jones
DevelopMentor
www.develop.com

RE: Still have XML loading problems

[Kevin Jones <ke...@develop.com>]

> I'm interested in seeing if we can gather enough folks to start a "Jasper
> 2.0" effort, on a completely fresh slate.  The design of this would take
> into account the two input syntaxes and everything we've learned, and the
> implementation could implement many obvious optimizations that would speed
> up the runtime code as well.
>
> What do you think?

I think this is a great idea - I'd certainly be interested,

Kevin Jones
DevelopMentor
www.develop.com

RE: Still have XML loading problems

["Craig R. McClanahan" <cr...@apache.org>]

On Sat, 24 Mar 2001, Kevin Jones wrote:

> Craig,
> 
> bear with my ignorance of how Catalina works at this point.
> I'm assuming that the JSP servlet compiles the 'raw' JSP->XML->Java and then
> compiles this to a .class file.
> 

It doesn't actually go to XML if the original syntax was the JSP
syntax.  There are two parsers (sharing a lot of underlying code), that
each go directly to Java.

> Why does the Jasper class-loader delegate to the web-apps classloader, isn't
> the Jasper step entirely independent. If so, couldn't the JSP servlet create
> its own CL that doesn't delegate to the webapps loader, but delegates
> straight to the shared classloader. In fact we could take tools.jar off-of
> the classpath and make it available to this classloader only.
> 

The Jasper page compiler needs to have access to all of the bean classes
that you might have placed in WEB-INF/classes and WEB-INF/lib.  This could
be "faked" by setting up yet another class loader that simulates the
"real" webapp class loader (in terms of what repositories it gives you
visibility to), or by making the special Jasper class loader a sibling of
the webapp class loader (rather than a child), and adding the WEB-INF
repositories to it as well.

This will work as long as you don't create an object in the Jasper class
loader and then expect it to be visible in the webapp class loader (or
vice versa) -- you get class cast exceptions at that point.

> So the Jasper CL would pick up tools.jar, crimson.jar and jaxp.jar and
> everything from my webapp lib and classes, but doesn't delegate to the
> webapp CL. This should have the effect of making Jasper independent of my
> webapp CL.
> 

The Cocoon folks would not be happy if we made tools.jar invisible to
webapps :-).

> The other CLs would work the same.
> 
> It makes the CL architecture slightly more complicated (but it already is
> complicated).
> 

I'll say.

> Will this work?
> 
> Another thought. If Jasper is a two step process (.jsp->XML, XML->Java) as
> step 1, java->.class as step 2, only the first step needs an XML parser, can
> we isolate that first step with a separate CL?
> 

As above, it's not a two step process today -- we took the expedient of
minimal work in creating the original parser so we basically didn't touch
the existing JSP->Java translations.

I'm interested in seeing if we can gather enough folks to start a "Jasper
2.0" effort, on a completely fresh slate.  The design of this would take
into account the two input syntaxes and everything we've learned, and the
implementation could implement many obvious optimizations that would speed
up the runtime code as well.

What do you think?

> Kevin (waiting to be shot down in flames) Jones
> 
> 
> 

Craig

RE: Still have XML loading problems

[Kevin Jones <ke...@develop.com>]

Craig,

bear with my ignorance of how Catalina works at this point.
I'm assuming that the JSP servlet compiles the 'raw' JSP->XML->Java and then
compiles this to a .class file.

Why does the Jasper class-loader delegate to the web-apps classloader, isn't
the Jasper step entirely independent. If so, couldn't the JSP servlet create
its own CL that doesn't delegate to the webapps loader, but delegates
straight to the shared classloader. In fact we could take tools.jar off-of
the classpath and make it available to this classloader only.

So the Jasper CL would pick up tools.jar, crimson.jar and jaxp.jar and
everything from my webapp lib and classes, but doesn't delegate to the
webapp CL. This should have the effect of making Jasper independent of my
webapp CL.

The other CLs would work the same.

It makes the CL architecture slightly more complicated (but it already is
complicated).

Will this work?

Another thought. If Jasper is a two step process (.jsp->XML, XML->Java) as
step 1, java->.class as step 2, only the first step needs an XML parser, can
we isolate that first step with a separate CL?

Kevin (waiting to be shot down in flames) Jones

Re: Still have XML loading problems

["Craig R. McClanahan" <Cr...@eng.sun.com>]

Kevin Jones wrote:

> > It (the JSP servlet) is loaded early already.
> >
> > It (and its XML parser) are now loaded by another classloader (not sure
> > that was
> > true on the 22nd -- if your Tomcat 4.0 has "jasper-compiler.jar" and
> > "jasper-runtime.jar" separated, then it is for you).
>
> I have jasper-compiler.jar in 'jasper' and jasper-runtime.jar in 'lib'
>

If I had looked at my calendar I would have figured that one out ...


> >
> > The problem is the following scenario:
> >
> > * JSP servlet loads early, parses TLDs from the web.xml file (which
> >   causes some JAXP parser classes to be loaded).
> >
> > * You do something with Xerces that causes Xerces classes to
> >   be loaded, from your webapps class loader (parent of the one
> >   used by the JSP servlet).
> >
> > * You do something with a JSP page that causes more parser
> >   classes to be loaded (from the JSP servlet's classloader), including
> >   a class with the same fully qualified name as one of the Xerces
> >   classes you just loaded.
> >
> > * Under 1.2 this worked.  Under 1.3 it gives sealing violation errors
> >
>
> I'm not seeing this sequence (I don't think)
>
> * I'm loading Xerces in my listener (in fact all I'm doing is using the
> DocumentBuilder and DocumentBuilderFactory), in the contextInitialized
> method, this is getting loaded first.
>

Yep, Listeners (and Filters) get initialized before any <load-on-startup>
servlets do.

>
> * JSP servlet is trying to load - in it's init() method I get the exception
>
> No TLDs but I see a call to TldLocationsCache.<init>
>

The constructor calls processWebDotXml(), which scans web.xml looking for
<taglib> directives (using an XML parse, of course).

You're seeing the same symptom from operations in a different order, but the
cause is identical -- when a JAXP class is loaded *after* a Xerces class (even
though the Xerces class is loaded from a parent class loader), you get the error
on a 1.3 JVM.

> Kevin Jones
>

Craig

RE: Still have XML loading problems

[Kevin Jones <ke...@develop.com>]

> It (the JSP servlet) is loaded early already.
>
> It (and its XML parser) are now loaded by another classloader (not sure
> that was
> true on the 22nd -- if your Tomcat 4.0 has "jasper-compiler.jar" and
> "jasper-runtime.jar" separated, then it is for you).

I have jasper-compiler.jar in 'jasper' and jasper-runtime.jar in 'lib'
>
> The problem is the following scenario:
>
> * JSP servlet loads early, parses TLDs from the web.xml file (which
>   causes some JAXP parser classes to be loaded).
>
> * You do something with Xerces that causes Xerces classes to
>   be loaded, from your webapps class loader (parent of the one
>   used by the JSP servlet).
>
> * You do something with a JSP page that causes more parser
>   classes to be loaded (from the JSP servlet's classloader), including
>   a class with the same fully qualified name as one of the Xerces
>   classes you just loaded.
>
> * Under 1.2 this worked.  Under 1.3 it gives sealing violation errors
>

I'm not seeing this sequence (I don't think)

* I'm loading Xerces in my listener (in fact all I'm doing is using the
DocumentBuilder and DocumentBuilderFactory), in the contextInitialized
method, this is getting loaded first.

* JSP servlet is trying to load - in it's init() method I get the exception

No TLDs but I see a call to TldLocationsCache.<init>

This is part of the trace I'm seeing

ContextConfig[/KRJ]: Added certificates -> request attribute Valve
xmlSetupListener: enter contextInitialized
xmlSetupListener: leave contextInitialized
StandardWrapper[/KRJ:default]: Loading container servlet default
default: init
StandardWrapper[/KRJ:invoker]: Loading container servlet invoker
invoker: init
StandardWrapper[/KRJ:jsp]: Using Jasper classloader for servlet jsp
jsp: init
StandardContext[/KRJ]: Servlet /KRJ threw load() exception
javax.servlet.ServletException: Servlet.init() for servlet jsp threw
exception
        at
org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:791)
        at
org.apache.catalina.core.StandardContext.start(StandardContext.java:3135)

... lines omitted for brevity ...

----- Root Cause -----
java.lang.SecurityException: sealing violation
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:234)
        at java.net.URLClassLoader.access$100(URLClassLoader.java:56)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
        at
org.apache.catalina.loader.StandardClassLoader.findClass(StandardClassLoader
.java:646)
        at
org.apache.catalina.loader.StandardClassLoader.loadClass(StandardClassLoader
.java:1013)
        at
org.apache.catalina.loader.StandardClassLoader.loadClass(StandardClassLoader
.java:912)
        at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:313)
        at
org.apache.jasper.compiler.TldLocationsCache.processWebDotXml(TldLocationsCa
che.java:161)


> The only other possible workaround I can think if is to exhaustively
> load every
> single class file out of jaxp.jar and crimson.jar at startup time.  That
> seems
> horrendously wasteful of memory (especially when you consider that this
> has to
> be done for each web app).
>

Agreed

Kevin Jones
DevelopMentor
www.develop.com

Re: Still have XML loading problems

["Craig R. McClanahan" <Cr...@eng.sun.com>]

Kevin Jones wrote:

> Yep 1.3
>
> > I think the only workaround for this is to ship our own copy of
> > the JAXP JAR
> > files, with the "sealed" attribute removed.  I'm also going to be
> > talking with
> > the JAXP folks about removing that in their next release.
>
> :-(
>
> What would happen if you made a special case of the JSP servlet and loaded
> it early?
> Could you preload jasper somewhere else?
>

It (the JSP servlet) is loaded early already.

It (and its XML parser) are now loaded by another classloader (not sure
that was
true on the 22nd -- if your Tomcat 4.0 has "jasper-compiler.jar" and
"jasper-runtime.jar" separated, then it is for you).

The problem is the following scenario:

* JSP servlet loads early, parses TLDs from the web.xml file (which
  causes some JAXP parser classes to be loaded).

* You do something with Xerces that causes Xerces classes to
  be loaded, from your webapps class loader (parent of the one
  used by the JSP servlet).

* You do something with a JSP page that causes more parser
  classes to be loaded (from the JSP servlet's classloader), including
  a class with the same fully qualified name as one of the Xerces
  classes you just loaded.

* Under 1.2 this worked.  Under 1.3 it gives sealing violation errors

The only other possible workaround I can think if is to exhaustively
load every
single class file out of jaxp.jar and crimson.jar at startup time.  That
seems
horrendously wasteful of memory (especially when you consider that this
has to
be done for each web app).

>
> Kevin Jones
> DevelopMentor
> www.develop.com

Craig

RE: Still have XML loading problems

[Kevin Jones <ke...@develop.com>]

Yep 1.3

> I think the only workaround for this is to ship our own copy of
> the JAXP JAR
> files, with the "sealed" attribute removed.  I'm also going to be
> talking with
> the JAXP folks about removing that in their next release.

:-(

What would happen if you made a special case of the JSP servlet and loaded
it early?
Could you preload jasper somewhere else?

Kevin Jones
DevelopMentor
www.develop.com

> -----Original Message-----
> From: Craig R. McClanahan [mailto:Craig.McClanahan@eng.sun.com]
> Sent: 23 March 2001 19:05
> To: Kevin Jones
> Cc: Tomcat-Dev
> Subject: Re: Still have XML loading problems
>
>
> Kevin Jones wrote:
>
> > Craig,
> >
> > I'm playing with the 22nd March drop of Catalina, and I've come across a
> > scenario where the new classloading architecture doesn't quite work.
> >
> > I'm using Xerces and Xalan (although Xalan is irrelevant to this). If I
> > access a servlet that uses XML and I don't put Xerces in my
> > web-app/myapp/lib my code fails (exactly what I'd expect). If I put
> > Xerces.jar into web-app/myapp/lib it works, again what I'd expect.
> >
> > However, if I put my XML code into an application listener then
> Jasper fails
> > to load - I get a Security Exception, sealing violation, while
> it's load the
> > 'jsp' servlet. It seems that what's happening is that my listener loads,
> > loads Xerces and executes OK. The jsp Servlet then tries to load
> > crimson/JAXP and *bang* sealing violation.
> >
>
> I will bet you're using a JDK 1.3 platform, right?  It turns out
> that all the
> changes we made are very effective for JDK 1.2, but the 1.3 version of
> URLClassLoader still doesn't like it.  :-(
>
> >
> > Sorry about this - the XML stuff had been going so well until then!
> >
>
> I think the only workaround for this is to ship our own copy of
> the JAXP JAR
> files, with the "sealed" attribute removed.  I'm also going to be
> talking with
> the JAXP folks about removing that in their next release.
>
>
> >
> > Kevin Jones
> > DevelopMentor
> > www.develop.com
>
> Craig
>

Re: Still have XML loading problems

["Craig R. McClanahan" <Cr...@eng.sun.com>]

Kevin Jones wrote:

> Craig,
>
> I'm playing with the 22nd March drop of Catalina, and I've come across a
> scenario where the new classloading architecture doesn't quite work.
>
> I'm using Xerces and Xalan (although Xalan is irrelevant to this). If I
> access a servlet that uses XML and I don't put Xerces in my
> web-app/myapp/lib my code fails (exactly what I'd expect). If I put
> Xerces.jar into web-app/myapp/lib it works, again what I'd expect.
>
> However, if I put my XML code into an application listener then Jasper fails
> to load - I get a Security Exception, sealing violation, while it's load the
> 'jsp' servlet. It seems that what's happening is that my listener loads,
> loads Xerces and executes OK. The jsp Servlet then tries to load
> crimson/JAXP and *bang* sealing violation.
>

I will bet you're using a JDK 1.3 platform, right?  It turns out that all the
changes we made are very effective for JDK 1.2, but the 1.3 version of
URLClassLoader still doesn't like it.  :-(

>
> Sorry about this - the XML stuff had been going so well until then!
>

I think the only workaround for this is to ship our own copy of the JAXP JAR
files, with the "sealed" attribute removed.  I'm also going to be talking with
the JAXP folks about removing that in their next release.


>
> Kevin Jones
> DevelopMentor
> www.develop.com

Craig