You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by youngm <yo...@gmail.com> on 2010/01/14 02:09:57 UTC
Classloader between Web application and system loader
I am designing my production Tomcat 6 system and would like to have a
classloader where I can put emergency "patch" jars (e.g. database drivers,
etc) and configuration (e.g. tweaked spring config).
These artifacts would need to be loaded in a classloader between the System
Classloader and the Webapp classloader so that the patch classpath resources
are picked up instead of the application resources. Looking at
http://tomcat.apache.org/tomcat-6.0-doc/class-loader-howto.html there does
not appear to be a standard place where I can put these artifacts. Is
anyone aware of any solution to my problem?
Thanks,
Mike
--
View this message in context: http://old.nabble.com/Classloader-between-Web-application-and-system-loader-tp27154908p27154908.html
Sent from the Tomcat - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Classloader between Web application and system loader
Posted by Mark Thomas <ma...@apache.org>.
On 14/01/2010 23:45, youngm wrote:
>> Won't help. The servlet spec requires that classes in the web
>> application take priority.
>
> The only quote I was able to find from the servlet spec was:
>
> "It is recommended also that the application class loader be
> implemented so that classes and resources packaged within the WAR are loaded
> in
> preference to classes and resources residing in container-wide library
> JARs."
>
> Seems to me that there is some wiggle room there. Besides Websphere
> actually by default deploys its web application "parent first" completely
> ignoring the above recommendation. :)
Fair enough, requires was too strong. You can change the default Tomcat
behaviour. Look for the delegate attribute on the loader.
The spec recommends not using parent first for good reasons - memory
leaks for one. Why WebSphere goes against that I have no idea.
>> Even if the solutions would work (which they won't) they seem a lot more
>> complex than
>> - unpack war
>> - replace driver jar
>> - pack war
>> - deploy through manager
>
> Here are a couple of the problems I have with the above approach:
>
> * If I'm administering an environment with 20-30 tomcat apps all of which
> require the new database driver this is a rather more difficult process for
> middleware engineers (not developers) to perform than simply (Copy this file
> into *this* directory, restart app)
That depends how much you process (and associated automation) you have.
I'd have a staging area with the original wars and the patches and a
script that generated the patched wars.
> * It would seem that the change tracking of a process like that above would
> be very difficult. It would be hard to answer questions like "What patches
> have I applied to these 20 applications in production?" If there were
> simply a folder with the patches right there then it becomes very easier to
> see how this running application's configuration differs from the
> configuration the archive was built with. This becomes more and more
> important the older an application gets and the more patches applied to it.
I can see where you are coming from with this. Unless you have only one
application per instance, the solution is going to have to be at the
web-app level and there isn't anything that springs immediately to mind.
Neither is there an easy place this could be added.
With one application per instance, changing the delegation to parent
first and placing the "patches" in lib might work. You could also enable
the shared class loader and use that. I'd be wary of memory leaks on
reload but you say you aren't reloading so that shouldn't be an issue.
I'd also be concerned about any apps that used XML parsers or anything
else where a different version might be found in a parent class loader.
>> That would update the driver for the app with zero downtime. What is the
>> issue with this approach?
>
> This may just be my organization's standard but we never hot deploy in
> production. We're too scared of permgen errors and other potential
> inconsistencies plus we have a load balancer. However, this may work for
> others so point taken.
PermGen on reload is a whole other topic :)
Getting back to your issue, with some restrictions you could probably
get something along the lines you describe working but it would be a bit
of a hack. I think a staging area with a script is a safer solution that
meets your requirements. Something like:
/staging/
/source
app1.war
app2.war
...
/patches
/app1
/WEB-INF
/lib
new.jar
/app2
...
/output
/app1.war
/app2.war
...
with a single script that iterates through source and
- expands foo.war to a tmp location
- copies /patches/foo over the top of the temp location
- creates /output/foo.war from the tmp location
You have single WARs which Tomcat work with Tomcat (and any other
container) without having to hack around
You have one place to look for patches applied to an app
The patch process is add patch files, run script, copy output WARs.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Classloader between Web application and system loader
Posted by Pid <pi...@pidster.com>.
On 14/01/2010 23:45, youngm wrote:
>
> * It would seem that the change tracking of a process like that above would
> be very difficult. It would be hard to answer questions like "What patches
> have I applied to these 20 applications in production?" If there were
> simply a folder with the patches right there then it becomes very easier to
> see how this running application's configuration differs from the
> configuration the archive was built with. This becomes more and more
> important the older an application gets and the more patches applied to it.
I think there's a fundamental misunderstanding here.
In some environments it is possible to load application code, followed
by patch code which replaces sections of the previously loaded code.
Thus the environment may load multiple versions of the same code, in a
particular order.
A running JVM is not one of those environments AFAIK. (Because, amongst
other reasons, you can't specify the load order of classes/jars in a
given classloader.)
The concept of sequential patching isn't something that can be readily
applied to deployed Java web applications. This is something better
achieved prior to deployment, e.g. in a source code repository.
If you have multiple versions of the same class in the classloader tree
you *will* experience problems at some point. We frequently see
problems here caused by duplicate jars and classes.
You seem to acknowledge that different servlet containers exhibit
different behaviour here, which makes the whole scheme you're proposing
even more fragile.
Your idea might /seem/ like a good solution, but it is a square peg in a
round hole.
People vastly more experienced than me are suggesting things to you that
they think are a better solution. Listen to them.
p
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Classloader between Web application and system loader
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: youngm [mailto:youngm@gmail.com]
> Subject: Re: Classloader between Web application and system loader
>
> * If I'm administering an environment with 20-30 tomcat apps
> all of which require the new database driver this is a rather
> more difficult process for middleware engineers (not developers)
> to perform than simply (Copy this file into *this* directory,
> restart app)
Why aren't you telling them "Run this script"? One step, no fuss, and you control the actions of the script.
> * It would seem that the change tracking of a process like that
> above would be very difficult.
Only if you refuse to automate it. Whatever tracking you want would be built into the script.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Classloader between Web application and system loader
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Mike,
On 1/14/2010 6:45 PM, youngm wrote:
> * It would seem that the change tracking of a process like that above would
> be very difficult. It would be hard to answer questions like "What patches
> have I applied to these 20 applications in production?" If there were
> simply a folder with the patches right there then it becomes very easier to
> see how this running application's configuration differs from the
> configuration the archive was built with. This becomes more and more
> important the older an application gets and the more patches applied to it.
I would think that going through a complete release cycle with the patch
would be /more/ trackable than just tossing a bunch of new JAR files
into a directory. You would also get the benefit of /testing/ the patch
instead of just crossing your fingers and doing a blind update on a
production machine.
But that's just me.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAktQiZMACgkQ9CaO5/Lv0PCBdgCgmODUCBeM79Be4RJ+YueZ3POV
dRYAoMBoNxITMINeWQcfRGDr/GRS0H/A
=GKvr
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Classloader between Web application and system loader
Posted by youngm <yo...@gmail.com>.
>Very bad idea. Search the archives for a long list of things that go
>wrong when you start changing the class path. Not to mention that would
>make the new driver visible to all web applications not just the web app
>that needed the new jar file. And you'd need to restart Tomcat for the
>changes to take effect. That means downtime unless you use load-balancing.
I'll do a search and see what I can find what kinds of things go wrong with
changing the class path. This is a bummer because this was going to be my
fallback solution. :(
> Won't help. The servlet spec requires that classes in the web
> application take priority.
The only quote I was able to find from the servlet spec was:
"It is recommended also that the application class loader be
implemented so that classes and resources packaged within the WAR are loaded
in
preference to classes and resources residing in container-wide library
JARs."
Seems to me that there is some wiggle room there. Besides Websphere
actually by default deploys its web application "parent first" completely
ignoring the above recommendation. :)
> Even if the solutions would work (which they won't) they seem a lot more
> complex than
> - unpack war
> - replace driver jar
> - pack war
> - deploy through manager
Here are a couple of the problems I have with the above approach:
* If I'm administering an environment with 20-30 tomcat apps all of which
require the new database driver this is a rather more difficult process for
middleware engineers (not developers) to perform than simply (Copy this file
into *this* directory, restart app)
* It would seem that the change tracking of a process like that above would
be very difficult. It would be hard to answer questions like "What patches
have I applied to these 20 applications in production?" If there were
simply a folder with the patches right there then it becomes very easier to
see how this running application's configuration differs from the
configuration the archive was built with. This becomes more and more
important the older an application gets and the more patches applied to it.
> That would update the driver for the app with zero downtime. What is the
> issue with this approach?
This may just be my organization's standard but we never hot deploy in
production. We're too scared of permgen errors and other potential
inconsistencies plus we have a load balancer. However, this may work for
others so point taken.
Mike
--
View this message in context: http://old.nabble.com/Classloader-between-Web-application-and-system-loader-tp27154908p27170181.html
Sent from the Tomcat - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Classloader between Web application and system loader
Posted by Mark Thomas <ma...@apache.org>.
On 14/01/2010 22:19, youngm wrote:
> I personally don't like any of the options.
If you articulated your concerns, then we might be able to show you
solutions to them.
> Here are a couple of solutions
> that I might like a little more. I'd be curious if anyone likes any of
> these.
>
> 1. modify the tomcat startup scripts to provide a standard hook for adding
> to the system classloader (CATALINA_CLASSPATH)???
Very bad idea. Search the archives for a long list of things that go
wrong when you start changing the class path. Not to mention that would
make the new driver visible to all web applications not just the web app
that needed the new jar file. And you'd need to restart Tomcat for the
changes to take effect. That means downtime unless you use load-balancing.
> 2. Add an optional Classloader similar to the common.loader that is
> positioned between the system loader and the webapp loader. This loader
> could be configured the same way the common.loader is configured through a
> catalina.properties property?
Won't help. The servlet spec requires that classes in the web
application take priority.
> Thoughts?
Even if the solutions would work (which they won't) they seem a lot more
complex than
- unpack war
- replace driver jar
- pack war
- deploy through manager
That would update the driver for the app with zero downtime. What is the
issue with this approach?
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Classloader between Web application and system loader
Posted by youngm <yo...@gmail.com>.
Ok, so to sum up this is what I've gathered.
Here is my hypothetical situation. I have a .war running in production.
This war embeds an oracle driver in its WEB-INF/lib. Middleware discovers
that there is a security vulnerability in this oracle driver and need to do
an emergency upgrade of the database driver to a new patch version. My
options are:
1. Modify the .war or the exploded .war deleting the old oracle driver and
replacing the new one.
2. Involve developers and release management to create a new version of the
tag and do a new build. Assuming that my build and release management
processes are mature enough to do this and that the .war isn't a thirdparty
.war
3. Perhaps modify the tomcat startup scripts to add the new oracle driver to
the system classloader?
Does anyone see any other options?
What are people's thoughts on these options?
I personally don't like any of the options. Here are a couple of solutions
that I might like a little more. I'd be curious if anyone likes any of
these.
1. modify the tomcat startup scripts to provide a standard hook for adding
to the system classloader (CATALINA_CLASSPATH)???
2. Add an optional Classloader similar to the common.loader that is
positioned between the system loader and the webapp loader. This loader
could be configured the same way the common.loader is configured through a
catalina.properties property?
Thoughts?
Mike
--
View this message in context: http://old.nabble.com/Classloader-between-Web-application-and-system-loader-tp27154908p27169031.html
Sent from the Tomcat - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Classloader between Web application and system loader
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Pid [mailto:pid@pidster.com]
> Subject: Re: Classloader between Web application and system loader
>
> You might get away with adding individual classes to the classloader
> tree, but multiple jars containing the same class is likely to cause
> problems. In fact, I'm not even sure the former is foolproof.
>
> I don't the classloader hierarchy is designed to handle patching.
Correct, and the whole exercise is somewhat pointless, since the webapp being "patched" will have to be restarted anyway. Might as well replace the .war file (or directory) with the updated one. Tracking patches should be done by the webapp build mechanism, not Tomcat.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Classloader between Web application and system loader
Posted by Pid <pi...@pidster.com>.
On 14/01/2010 02:40, youngm wrote:
>
>
>> Put expanded classes into WEB-INF/classes.
>> Likewise, in CATALINA_BASE/lib
>
> So CATALINA_BASE/lib jars and classes are used before the application
> (WEB-INF/lib and /classes)?
>
> http://tomcat.apache.org/tomcat-6.0-doc/class-loader-howto.html under Class
> Loader Definitions seems to indicate otherwise. The way I read it the
> application's WEB-INF/lib and classes are used before CATALINA_BASE/lib.
> Though perhaps I'm reading it wrong?
>
> I like the idea of placing my patch jars and classes in CATALINA_BASE/lib
> over WEB-INF so that I can more easily see what patches have been applied to
> an environment I just didn't think those would be loaded before the
> application's.
>
> Thanks,
> Mike
You might get away with adding individual classes to the classloader
tree, but multiple jars containing the same class is likely to cause
problems. In fact, I'm not even sure the former is foolproof.
I don't the classloader hierarchy is designed to handle patching.
p
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Classloader between Web application and system loader
Posted by Konstantin Kolinko <kn...@gmail.com>.
2010/1/14 youngm <yo...@gmail.com>:
>
>
>>Put expanded classes into WEB-INF/classes.
>>Likewise, in CATALINA_BASE/lib
>
> So CATALINA_BASE/lib jars and classes are used before the application
> (WEB-INF/lib and /classes)?
>
No, they are not.
WEB-INF is used before CATALINA_BASE, but actually that is a more
complicated thing.
If you need to patch a web application, the only thing that I can
recommend is, as I wrote, to place your classes into WEB-INF/classes.
jar -u allows to add files to an existing jar/war archive, or you
may place them into the expanded webapp directory
Best regards,
Konstantin Kolinko
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Classloader between Web application and system loader
Posted by youngm <yo...@gmail.com>.
>Put expanded classes into WEB-INF/classes.
>Likewise, in CATALINA_BASE/lib
So CATALINA_BASE/lib jars and classes are used before the application
(WEB-INF/lib and /classes)?
http://tomcat.apache.org/tomcat-6.0-doc/class-loader-howto.html under Class
Loader Definitions seems to indicate otherwise. The way I read it the
application's WEB-INF/lib and classes are used before CATALINA_BASE/lib.
Though perhaps I'm reading it wrong?
I like the idea of placing my patch jars and classes in CATALINA_BASE/lib
over WEB-INF so that I can more easily see what patches have been applied to
an environment I just didn't think those would be loaded before the
application's.
Thanks,
Mike
--
View this message in context: http://old.nabble.com/Classloader-between-Web-application-and-system-loader-tp27154908p27155608.html
Sent from the Tomcat - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Classloader between Web application and system loader
Posted by Konstantin Kolinko <kn...@gmail.com>.
2010/1/14 Rob S. <ro...@tintri.com>:
> Konstantin, do you have to modify one of the loaders in catalina.properties
> to check in CATALINA_BASE/lib? Or it does that automagically?
>
In 6.0.20 it is by default:
What comes first takes priority in an URLClassLoader.
common.loader=${catalina.home}/lib,${catalina.home}/lib/*.jar
In 6.0.21 and later it is
common.loader=${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar
Regarding WEB-INF/lib vs. WEB-INF/classes the ordering is defined in
the Servlet spec.
Best regards,
Konstantin Kolinko
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Classloader between Web application and system loader
Posted by "Rob S." <ro...@tintri.com>.
Konstantin, do you have to modify one of the loaders in catalina.properties
to check in CATALINA_BASE/lib? Or it does that automagically?
On Wed, Jan 13, 2010 at 5:21 PM, Konstantin Kolinko
<kn...@gmail.com>wrote:
> 2010/1/14 youngm <yo...@gmail.com>:
> >
> > I am designing my production Tomcat 6 system and would like to have a
> > classloader where I can put emergency "patch" jars (e.g. database
> drivers,
> > etc) and configuration (e.g. tweaked spring config).
> >
> > These artifacts would need to be loaded in a classloader between the
> System
> > Classloader and the Webapp classloader so that the patch classpath
> resources
> > are picked up instead of the application resources. Looking at
> > http://tomcat.apache.org/tomcat-6.0-doc/class-loader-howto.html there
> does
> > not appear to be a standard place where I can put these artifacts. Is
> > anyone aware of any solution to my problem?
> >
> > Thanks,
> > Mike
>
> Put expanded classes into WEB-INF/classes.
> Likewise, in CATALINA_BASE/lib
>
> The expanded classes always have precedence over *.jar s.
>
> Best regards,
> Konstantin Kolinko
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Classloader between Web application and system loader
Posted by Konstantin Kolinko <kn...@gmail.com>.
2010/1/14 youngm <yo...@gmail.com>:
>
> I am designing my production Tomcat 6 system and would like to have a
> classloader where I can put emergency "patch" jars (e.g. database drivers,
> etc) and configuration (e.g. tweaked spring config).
>
> These artifacts would need to be loaded in a classloader between the System
> Classloader and the Webapp classloader so that the patch classpath resources
> are picked up instead of the application resources. Looking at
> http://tomcat.apache.org/tomcat-6.0-doc/class-loader-howto.html there does
> not appear to be a standard place where I can put these artifacts. Is
> anyone aware of any solution to my problem?
>
> Thanks,
> Mike
Put expanded classes into WEB-INF/classes.
Likewise, in CATALINA_BASE/lib
The expanded classes always have precedence over *.jar s.
Best regards,
Konstantin Kolinko
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Classloader between Web application and system loader
Posted by youngm <yo...@gmail.com>.
Thanks for your help guys. I've never really considered modifying a .war
outside of the normal build process an acceptable thing to do. But, it
appears that is the recommended approach from the Tomcat community so I'll
work with my middleware organization to see if we can get something like
that to work. Obviously applying the patch to the project source and
rebuilding the source is the optimal solution, however, I'm not sure how
much faith I have in our build system to be able to produce builds on demand
of an application years after it has gone into maintenance mode.
I still think the general idea of being able to "patch" an application
without modifying the war is a useful approach in a number of situations but
I see that it would not be very easy to implement/rationalize it into the
platform.
Thanks again for everyones help.
Mike
--
View this message in context: http://old.nabble.com/Classloader-between-Web-application-and-system-loader-tp27154908p27183997.html
Sent from the Tomcat - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org