You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cocoon.apache.org by Stefano Mazzocchi <st...@apache.org> on 2002/06/17 12:09:19 UTC
[help needed] Patching the Servlet API on classloading
People,
I've asked Jason Hunter (one of our representatives in the Servlet API
JSR) if Servlet 2.4 (which is going to be finalized soon) is being more
explicit on the way the classloaders are supposed to load classes from
WAR files.
If Servlet 2.4 was explicit on the 'direction' that classloading
fall-back should work, than Cocoon (and any other WARs) will be much
more portable across containers and classloading hells will simply be a
thing of the past (except for those libraries where static calls are
used, see log4j).
He said that nothing has been changed to fix that, but he'll be happy to
run a proposal on my (our?) behalf thru the list. Having been a member
of that JSR for years, you can be sure the proposal will be heard.
So, I'll try to write something down here, if you have any
comments/suggestions, please, let me know ASAP.
---- cut here ----
Fixing WAR portability issues
=============================
Author: Stefano Mazzocchi (stefano@apache.org) [former member of
JSR-053]
[your names will be here]
State of the art
----------------
Interoperability between servlet containers has always been a goal of
the Servlet API. The WAR concept was, in fact, introduced to allow
servlet/jsp-based web applications to be easily deployable on a
container without further hassle.
Unfortunately, the Servlet API failed to describe in explicit terms
*how* the servlet container must treat this WAR file. I'm not talking
about 'how to deploy', since this is obviously an implementation detail,
but about 'how to use a deployed WAR'.
Current problems
----------------
With the advent of server-side XML technology, WAR file portability
suffers from what it is now known as "xml parser hell" or simply "stupid
classloader ignores my jar files!"
The problem is simple: when a class contained in a WAR file wants to
load a class, a classloader is invoqued. The Java Language Specification
indicates that the 'normal' flow of classloading fallback is the
following:
system -> custom classloader -> custom classloader
so, first of all, the class is loaded from the system and, if not found,
it is asked to the application-defined classloader and so on.
Unfortunately, this behavior breaks WAR portability in those containers
that expose the classes that are overwritten by the WAR file.
NOTE: JDK 1.4 fixed this by providing an 'endorsed' directory where you
place your jars that you want to overwrite the classes shipped with the
rt.jar of the JVM, so for JDK 1.4, the classloading flow becomes
endorsed classloader -> system classloader -> application classloader
Still, implies that 'deploy and forget' of WAR files is not possible
across containers.
Current Solutions
-----------------
This problem was identified and solved in Tomcat 4.x by changing the
fallback direction of classloading. So, Tomcat 4.x does
[war classloader -> tomcat classloader -> system classloader]
this allows several nice things:
1) 'deploy and forget' works.
2) you can have different WARs including different versions of the same
library (say a JAXP 1.0 parser and a JAXP 1.1 parser), without
collision.
Proposal
--------
Our proposal is to include a section in the Servlet API 2.4 that makes
the classloading fallback direction explicit and forces servlet
containers to implement this classloading behavior to be said
'compliant' with this version of the API.
We want to enphasize that no changes *whatsoever* are required both to
the Servlet API, nor to the DTDs describing the various deployment
descriptors. Those are just instructions to the servlet container
implementors, but that guarantee a common classloading behavior across
containers that increases interoperability.
Also, we'd like to point out that impact on exising functionality and
implementations is minimal: a JSP engine is very likely to provide
already a nested classloading implementation, so changing the direction
of fallback is a matter of changing the order of call between system
classloader and the custom classloaders.
Also, the impact on previous WAR deployment behavior is minimal: all
WARS that don't contain jars or classes are not affected by this change
and those who do, it doesn't make any difference for them if their
classes are looked up immediately instead of after failing looking them
up in the system classpath and falling back to the web-inf/lib
directory.
Result
------
If the Servlet API 2.4 makes this classloading behavior explicit and
forces all containers to 'behave' the same way, WAR interoperability
will be incredibly augmented, expecially for servlets that need lots of
libraries (and this is become more and more the case with servlet-based
web frameworks)
Thanks for your time and consideration.
--
Stefano Mazzocchi One must still have chaos in oneself to be
able to give birth to a dancing star.
<st...@apache.org> Friedrich Nietzsche
--------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [help needed] Patching the Servlet API on classloading
Posted by David Haraburda <da...@haraburda.com>.
Sylvain Wallez wrote:
> I don't agree with you : findResources() is the method that a given
> ClassLoader can redefine to enumerate *its own resources*.
> getResources() aggreates the enumerations of all resources found in the
> classloader hierarchy.
Actually, I think maybe you do agree with me :-) -- this is basically what I was saying (sorry if I created
confusion), all I am adding is that getResources being final and getResource not being final don't provide
for an inconsistency, IMO.
> But it does this using the "standard" order which
> is system classloader resources first and custom classloader resources
> last. So you can't change the order of the enumeration to give a higher
> priority to custom classloader resources by enumerating them first.
The "standard" classloading delegation/algorithm is for security's sake, and I think it
makes sense most of the time (see http://java.sun.com/j2se/1.4/docs/guide/security/spec/security-spec.doc5.html
for more info), although it's obvious there is a problem that needs solving here with webapp classloading.
My reason for posting is to clarify to make sure people don't think that the solution needs to involve changing
JDK system classes (and I don't think that is what you were suggesting). I agree with both you
and Stefano that making a change to the Serlvet spec to mandate servlet containers implement
the heirarchy like tomcat does is a good idea -- I just wanted to throw in there that I don't think
anything else beyond this is needed.
Again, just my $0.02
Thanks,
David
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [help needed] Patching the Servlet API on classloading
Posted by Sylvain Wallez <sy...@anyware-tech.com>.
David Haraburda wrote:
>Sylvain Wallez wrote:
>
>
>
>>. . .
>>Another related item : while updating the ParanoidClassloader to make it
>>really paranoid, I encountered an inconsistency in java.lang.ClassLoader
>>class : getResource() can be overriden, but getResources() is declared
>>final, meaning there is no way to change the order in which e.g.
>>indentical service definitions in META-INF/services are enumerated.
>>Although this shouldn't cause a problem with JAXP (at least the Apache
>>implementation) which uses getResource() (no enumeration), we should be
>>aware that this is a potential pitfall.
>>
>>
>
>This isn't an inconsistency, it's intentional -- getResources() is a final method that calls findResources, a protected method on the current instance of the ClassLoader. ClassLoader implementations should override findResources(). The purpose of getResources() is to get a set of resources from all the
>classloaders in the classloader heirarchy.
>
>
I don't agree with you : findResources() is the method that a given
ClassLoader can redefine to enumerate *its own resources*.
getResources() aggreates the enumerations of all resources found in the
classloader hierarchy. But it does this using the "standard" order which
is system classloader resources first and custom classloader resources
last. So you can't change the order of the enumeration to give a higher
priority to custom classloader resources by enumerating them first.
Sylvain
--
Sylvain Wallez
Anyware Technologies Apache Cocoon
http://www.anyware-tech.com mailto:sylvain@apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [help needed] Patching the Servlet API on classloading
Posted by David Haraburda <da...@haraburda.com>.
Sylvain Wallez wrote:
>
> . . .
> Another related item : while updating the ParanoidClassloader to make it
> really paranoid, I encountered an inconsistency in java.lang.ClassLoader
> class : getResource() can be overriden, but getResources() is declared
> final, meaning there is no way to change the order in which e.g.
> indentical service definitions in META-INF/services are enumerated.
> Although this shouldn't cause a problem with JAXP (at least the Apache
> implementation) which uses getResource() (no enumeration), we should be
> aware that this is a potential pitfall.
This isn't an inconsistency, it's intentional -- getResources() is a final
method
that calls findResources, a protected method on the current instance of the
ClassLoader. ClassLoader implementations should override findResources().
The purpose of getResources() is to get a set of resources from all the
classloaders
in the classloader heirarchy.
>
>
> As this item impacts one of the core classes of the JDK, it may be
> difficult to convice people to fix it, but this would allow to uniformly
> override the search order in specialized classloaders.
>
> Sylvain
David
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [help needed] Patching the Servlet API on classloading
Posted by Stefano Mazzocchi <st...@apache.org>.
Sylvain Wallez wrote:
>
> Stefano Mazzocchi wrote:
>
> >People,
> >
> >I've asked Jason Hunter (one of our representatives in the Servlet API
> >JSR) if Servlet 2.4 (which is going to be finalized soon) is being more
> >explicit on the way the classloaders are supposed to load classes from
> >WAR files.
> >
> >If Servlet 2.4 was explicit on the 'direction' that classloading
> >fall-back should work, than Cocoon (and any other WARs) will be much
> >more portable across containers and classloading hells will simply be a
> >thing of the past (except for those libraries where static calls are
> >used, see log4j).
> >
> >He said that nothing has been changed to fix that, but he'll be happy to
> >run a proposal on my (our?) behalf thru the list. Having been a member
> >of that JSR for years, you can be sure the proposal will be heard.
> >
> >So, I'll try to write something down here, if you have any
> >comments/suggestions, please, let me know ASAP.
> >
>
> Your note clearly explains the problem and its solution. However, Tomcat
> being the reference implementation of servlets 2.3, I thought it
> classloading strategy was written in the spec.
>
> Going back to reading, I found at paragraph 9.7.2 "It is recommended
> also that the application class loader be implemented so that classes
> and resources packaged within the WAR are loaded in preference to
> classes and resources residing in container-wide library JARs."
>
> So what you want to propose is that this "recommended" is changed to
> "required", right ?
Yes, exactly.
> Another related item : while updating the ParanoidClassloader to make it
> really paranoid, I encountered an inconsistency in java.lang.ClassLoader
> class : getResource() can be overriden, but getResources() is declared
> final, meaning there is no way to change the order in which e.g.
> indentical service definitions in META-INF/services are enumerated.
> Although this shouldn't cause a problem with JAXP (at least the Apache
> implementation) which uses getResource() (no enumeration), we should be
> aware that this is a potential pitfall.
Yes, that's very nice information to submit along with the request for
change.
> As this item impacts one of the core classes of the JDK, it may be
> difficult to convice people to fix it, but this would allow to uniformly
> override the search order in specialized classloaders.
Gosh, one thing is proposing changes to the Servlet API, another one is
to propose changes to something in java.lang
Jason, what do you think?
> Sylvain
>
> > ---- cut here ----
> >
> > Fixing WAR portability issues
> > =============================
> >
> > Author: Stefano Mazzocchi (stefano@apache.org) [former member of
> >JSR-053]
> > [your names will be here]
> >
> >
> > State of the art
> > ----------------
> >
> >Interoperability between servlet containers has always been a goal of
> >the Servlet API. The WAR concept was, in fact, introduced to allow
> >servlet/jsp-based web applications to be easily deployable on a
> >container without further hassle.
> >
> >Unfortunately, the Servlet API failed to describe in explicit terms
> >*how* the servlet container must treat this WAR file. I'm not talking
> >about 'how to deploy', since this is obviously an implementation detail,
> >but about 'how to use a deployed WAR'.
> >
> > Current problems
> > ----------------
> >
> >With the advent of server-side XML technology, WAR file portability
> >suffers from what it is now known as "xml parser hell" or simply "stupid
> >classloader ignores my jar files!"
> >
> >The problem is simple: when a class contained in a WAR file wants to
> >load a class, a classloader is invoqued. The Java Language Specification
> >indicates that the 'normal' flow of classloading fallback is the
> >following:
> >
> > system -> custom classloader -> custom classloader
> >
> >so, first of all, the class is loaded from the system and, if not found,
> >it is asked to the application-defined classloader and so on.
> >
> >Unfortunately, this behavior breaks WAR portability in those containers
> >that expose the classes that are overwritten by the WAR file.
> >
> >NOTE: JDK 1.4 fixed this by providing an 'endorsed' directory where you
> >place your jars that you want to overwrite the classes shipped with the
> >rt.jar of the JVM, so for JDK 1.4, the classloading flow becomes
> >
> > endorsed classloader -> system classloader -> application classloader
> >
> >Still, implies that 'deploy and forget' of WAR files is not possible
> >across containers.
> >
> > Current Solutions
> > -----------------
> >
> >This problem was identified and solved in Tomcat 4.x by changing the
> >fallback direction of classloading. So, Tomcat 4.x does
> >
> > [war classloader -> tomcat classloader -> system classloader]
> >
> >this allows several nice things:
> >
> > 1) 'deploy and forget' works.
> > 2) you can have different WARs including different versions of the same
> >library (say a JAXP 1.0 parser and a JAXP 1.1 parser), without
> >collision.
> >
> > Proposal
> > --------
> >
> >Our proposal is to include a section in the Servlet API 2.4 that makes
> >the classloading fallback direction explicit and forces servlet
> >containers to implement this classloading behavior to be said
> >'compliant' with this version of the API.
> >
> >We want to enphasize that no changes *whatsoever* are required both to
> >the Servlet API, nor to the DTDs describing the various deployment
> >descriptors. Those are just instructions to the servlet container
> >implementors, but that guarantee a common classloading behavior across
> >containers that increases interoperability.
> >
> >Also, we'd like to point out that impact on exising functionality and
> >implementations is minimal: a JSP engine is very likely to provide
> >already a nested classloading implementation, so changing the direction
> >of fallback is a matter of changing the order of call between system
> >classloader and the custom classloaders.
> >
> >Also, the impact on previous WAR deployment behavior is minimal: all
> >WARS that don't contain jars or classes are not affected by this change
> >and those who do, it doesn't make any difference for them if their
> >classes are looked up immediately instead of after failing looking them
> >up in the system classpath and falling back to the web-inf/lib
> >directory.
> >
> > Result
> > ------
> >
> >If the Servlet API 2.4 makes this classloading behavior explicit and
> >forces all containers to 'behave' the same way, WAR interoperability
> >will be incredibly augmented, expecially for servlets that need lots of
> >libraries (and this is become more and more the case with servlet-based
> >web frameworks)
> >
> >Thanks for your time and consideration.
> >
> >
> >
>
> --
> Sylvain Wallez
> Anyware Technologies Apache Cocoon
> http://www.anyware-tech.com mailto:sylvain@apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
> For additional commands, email: cocoon-dev-help@xml.apache.org
--
Stefano Mazzocchi One must still have chaos in oneself to be
able to give birth to a dancing star.
<st...@apache.org> Friedrich Nietzsche
--------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [help needed] Patching the Servlet API on classloading
Posted by Sylvain Wallez <sy...@anyware-tech.com>.
Stefano Mazzocchi wrote:
>People,
>
>I've asked Jason Hunter (one of our representatives in the Servlet API
>JSR) if Servlet 2.4 (which is going to be finalized soon) is being more
>explicit on the way the classloaders are supposed to load classes from
>WAR files.
>
>If Servlet 2.4 was explicit on the 'direction' that classloading
>fall-back should work, than Cocoon (and any other WARs) will be much
>more portable across containers and classloading hells will simply be a
>thing of the past (except for those libraries where static calls are
>used, see log4j).
>
>He said that nothing has been changed to fix that, but he'll be happy to
>run a proposal on my (our?) behalf thru the list. Having been a member
>of that JSR for years, you can be sure the proposal will be heard.
>
>So, I'll try to write something down here, if you have any
>comments/suggestions, please, let me know ASAP.
>
Your note clearly explains the problem and its solution. However, Tomcat
being the reference implementation of servlets 2.3, I thought it
classloading strategy was written in the spec.
Going back to reading, I found at paragraph 9.7.2 "It is recommended
also that the application class loader be implemented so that classes
and resources packaged within the WAR are loaded in preference to
classes and resources residing in container-wide library JARs."
So what you want to propose is that this "recommended" is changed to
"required", right ?
Another related item : while updating the ParanoidClassloader to make it
really paranoid, I encountered an inconsistency in java.lang.ClassLoader
class : getResource() can be overriden, but getResources() is declared
final, meaning there is no way to change the order in which e.g.
indentical service definitions in META-INF/services are enumerated.
Although this shouldn't cause a problem with JAXP (at least the Apache
implementation) which uses getResource() (no enumeration), we should be
aware that this is a potential pitfall.
As this item impacts one of the core classes of the JDK, it may be
difficult to convice people to fix it, but this would allow to uniformly
override the search order in specialized classloaders.
Sylvain
> ---- cut here ----
>
> Fixing WAR portability issues
> =============================
>
> Author: Stefano Mazzocchi (stefano@apache.org) [former member of
>JSR-053]
> [your names will be here]
>
>
> State of the art
> ----------------
>
>Interoperability between servlet containers has always been a goal of
>the Servlet API. The WAR concept was, in fact, introduced to allow
>servlet/jsp-based web applications to be easily deployable on a
>container without further hassle.
>
>Unfortunately, the Servlet API failed to describe in explicit terms
>*how* the servlet container must treat this WAR file. I'm not talking
>about 'how to deploy', since this is obviously an implementation detail,
>but about 'how to use a deployed WAR'.
>
> Current problems
> ----------------
>
>With the advent of server-side XML technology, WAR file portability
>suffers from what it is now known as "xml parser hell" or simply "stupid
>classloader ignores my jar files!"
>
>The problem is simple: when a class contained in a WAR file wants to
>load a class, a classloader is invoqued. The Java Language Specification
>indicates that the 'normal' flow of classloading fallback is the
>following:
>
> system -> custom classloader -> custom classloader
>
>so, first of all, the class is loaded from the system and, if not found,
>it is asked to the application-defined classloader and so on.
>
>Unfortunately, this behavior breaks WAR portability in those containers
>that expose the classes that are overwritten by the WAR file.
>
>NOTE: JDK 1.4 fixed this by providing an 'endorsed' directory where you
>place your jars that you want to overwrite the classes shipped with the
>rt.jar of the JVM, so for JDK 1.4, the classloading flow becomes
>
> endorsed classloader -> system classloader -> application classloader
>
>Still, implies that 'deploy and forget' of WAR files is not possible
>across containers.
>
> Current Solutions
> -----------------
>
>This problem was identified and solved in Tomcat 4.x by changing the
>fallback direction of classloading. So, Tomcat 4.x does
>
> [war classloader -> tomcat classloader -> system classloader]
>
>this allows several nice things:
>
> 1) 'deploy and forget' works.
> 2) you can have different WARs including different versions of the same
>library (say a JAXP 1.0 parser and a JAXP 1.1 parser), without
>collision.
>
> Proposal
> --------
>
>Our proposal is to include a section in the Servlet API 2.4 that makes
>the classloading fallback direction explicit and forces servlet
>containers to implement this classloading behavior to be said
>'compliant' with this version of the API.
>
>We want to enphasize that no changes *whatsoever* are required both to
>the Servlet API, nor to the DTDs describing the various deployment
>descriptors. Those are just instructions to the servlet container
>implementors, but that guarantee a common classloading behavior across
>containers that increases interoperability.
>
>Also, we'd like to point out that impact on exising functionality and
>implementations is minimal: a JSP engine is very likely to provide
>already a nested classloading implementation, so changing the direction
>of fallback is a matter of changing the order of call between system
>classloader and the custom classloaders.
>
>Also, the impact on previous WAR deployment behavior is minimal: all
>WARS that don't contain jars or classes are not affected by this change
>and those who do, it doesn't make any difference for them if their
>classes are looked up immediately instead of after failing looking them
>up in the system classpath and falling back to the web-inf/lib
>directory.
>
> Result
> ------
>
>If the Servlet API 2.4 makes this classloading behavior explicit and
>forces all containers to 'behave' the same way, WAR interoperability
>will be incredibly augmented, expecially for servlets that need lots of
>libraries (and this is become more and more the case with servlet-based
>web frameworks)
>
>Thanks for your time and consideration.
>
>
>
--
Sylvain Wallez
Anyware Technologies Apache Cocoon
http://www.anyware-tech.com mailto:sylvain@apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: [help needed] Patching the Servlet API on classloading
Posted by David Haraburda <da...@haraburda.com>.
Stefano Mazzocchi wrote:
> . . .
> The problem is simple: when a class contained in a WAR file wants to
> load a class, a classloader is invoqued. The Java Language Specification
> indicates that the 'normal' flow of classloading fallback is the
> following:
>
> system -> custom classloader -> custom classloader
>
> so, first of all, the class is loaded from the system and, if not found,
> it is asked to the application-defined classloader and so on.
A better way to describe this behavior (IMHO anyway :-) is the parent-child
heirarchy.
Basically, the Java specification says that when a ClassLoader is called
upon to load a class,
it asks it's parent classloader for the resource before looking for the
resource itself.
>
> . . .
>
> This problem was identified and solved in Tomcat 4.x by changing the
> fallback direction of classloading. So, Tomcat 4.x does
>
> [war classloader -> tomcat classloader -> system classloader]
Actually, it goes:
WebApp (WAR) classloader -> Bootstrap/JVM classloader -> "System"
classloader -> Tomcat classloader
(See http://jakarta.apache.org/tomcat/tomcat-4.0-doc/class-loader-howto.html
for a better
description)
I agree, the way Tomcat does it is good, and I think your proposal is a good
idea.
I would speculate that a lot of classloader issues (with other servlet
containers) are created by
users editing the CLASSPATH variable, to point to their jar files which they
have placed wherever
they want, or worse, placed in the JDK's directories, which creates problems
because then the system
classloader trys to load them. The spec already states that necessary JARs
and classes should be placed
in the webapp/lib and webapp/classes directories. Tomcat realizes this
problem as well, and ignores the
CLASSPATH environment variable, which I think is good too.
David
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org