You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by David Perez <cr...@yahoo.es> on 2005/05/18 08:56:33 UTC
Commons logging and memory leaks
Hi,
This is my 1st message to the list.
I have created a web app, integrated with a web server called Jetty.
Both of them use commons-logging 1.5 alpha1.
There is just one set of jars shared between them.
The optional package is included, so weak references must be active.
I have implemente webapp reload, by throwing the old class loader and
creating a new one.
I have observed with the help of a memory profiler, that the old class
loader isn't memory garbaged.
The cause is the log references. The garbage roots are that maintain
live references are:
* static HIERARCHY in org.apache.log.Hierarchy
* static log in org.mortbay.util.URLResource and some other Jetty
classes that use Commons Logging.
I have tested under Java 1.5.0.03 in Windows XP.
Can someone please give me some indications?
Is it best to have separate commons-logging instances between my web app
and the web server?
Best regards
David Pérez
Re: Commons logging and memory leaks
Posted by David Perez <cr...@yahoo.es>.
Thanks Simon,
Simon Kitching schrieb:
>On Wed, 2005-05-18 at 12:20 +0200, David Perez wrote:
>
>
>>You're right, I have changed the commons-logging version to the latest one.
>>Now I have reverted to commons-logging-1.0.2
>>
>>
>
>I would recommend using commons-logging-1.0.4. After all, that is the
>most recent official release.
>
>
>
I have already done so successfully. See below for more details.
>
>
>>>I suggest registering a ServletContextListener. Declare a <listener>
>>>element in the web.xml file, with a nested <listener-class> element
>>>containing the fully qualified classname of a logging-cleanup class. In
>>>the contextDestroyed method of that class do this:
>>>* LogFactory.release(
>>> Thread.currentThread().getContextClassLoader())
>>>* somehow cleaning up the underlying logging library if needed.
>>>
>>>If that doesn't help, let me know and we can look into it a bit
>>>further.
>>>
>>>
>>>
>>>
>> From my code I call LogFactory.releaseAll() after removing the old web
>>app context, and no luck :-(
>>
>>
>
>Do you mean that you use a ServletContextListener as I described? Or
>something else?
>
>Note also that LogFactory.releaseAll may not do what you want. If your
>webapp has bound to commons-logging that is loaded via a shared
>classloader, then calling releaseAll will shut down *all* instances of
>commons-logging - which other webapps running in the same container may
>not appreciate :-). The code I recommended will only shut down logging
>for the webapp whose context is being destroyed, assuming you are
>calling the method from a SerletContextListener.contextDestroyed, ie
>from code where Thread.currentThread.getContextClassLoader() returns the
>classloader of the webapp being unloaded.
>
>
>
Now I use LogFactory.release()
>>Should my webapp use another copy of commons-logging.jar?
>>
>>
>
>That depends.
>
>If your webapp has all the libraries it needs in WEB-INF/lib, then yes.
>Deploying commons-logging.jar in WEB-INF/lib in that situation makes
>things very clean and consistent. There's no need to use call
>LogFactory.release() in this case, as undeploying the webapp
>automatically undeploys commons-logging and the underlying logging lib.
>
>But if your webapp is calling into code that is deployed via a shared
>classloader, and that code uses commons-logging, then you must *not*
>also deploy commons-logging in WEB-INF/lib. If you do (and you are using
>release 1.0.4), you will get the message
> Invalid class loader hierarchy.
> You have more than one version of org.apache.commons.logging.Log
> visible, which is not allowed.
>If you are using an earlier release, you will probably get something
>like:
> AvalonLogger does not implement Log
>
>
>
>
Finally I use a shared instance.
>
>
>>Here is the logging jars I use in my app:
>>
>>jars used by the web server (Jetty):
>>
>> * commons-logging.jar and
>> * log4j.jar
>>
>>jars used by the webapp:
>>
>> * logkit.jar
>>
>> My web app uses Cocoon which uses Avalon LogKit, and makes me
>>include logkit.jar, even though logging is redirected to log4j.
>>
>>
>
>What do you mean "logging is redirected to log4j"?
>
>
>
Avalon LogKit can be configured to use the Log4J subsystem. I like more
Log4J than Avalon LogKit.
>> Some other classes use directly commons-logging
>>
>>With commons-logging-1.0.4 and 1.05 alpha1, I have to make logkit.jar
>>available to Jetty, otherwise commons-loggings throws a
>>ClassNotFoundException, because it cannot find
>>
>>org/apache/log/Hierarchy. I wonder why it tries to load LogKit if Log4j is already available?
>>
>>
>
>Commons-logging does *not* auto-discover LogKit; it doesn't even try. So
>somewhere something in your webapp must be *ordering* commons-logging to
>use AvalonLogger (aka LogKit) as the logger.
>
>Which logging lib commons-logging uses can be explicitly set via a
>number of methods; .properties files, jdk1.3 service files, system
>properties. See the commons-logging documentation for more information.
>
>
>
You were right, I had a WEB-INF/classes/common-logging.properties file,
of my webapp, that I used to use before I decided to use log4j instead
of LogKit.
Commons-loggind 1.02 did ignore this file, but version 1.0.4 didn't.
This what has confused me.
>
>
>>Caused by: java.lang.NoClassDefFoundError: org/apache/log/Hierarchy
>> at org.apache.commons.logging.impl.LogKitLogger.getLogger(LogKitLogger.java:77)
>> at org.apache.commons.logging.impl.LogKitLogger.<init>(LogKitLogger.java:64)
>> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>> at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
>> at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:529)
>>
>>
>
>Yep, commons-logging's LogFactoryImpl is explicitly trying to
>instantiate a LogKitLogger object.
>
>
>
>Can you please:
>(a) look for what is ordering commons-logging to use LogKit and remove
>that?
>(b) use commons-logging 1.0.4; I will be able to help you much better
>than if you use the very old 1.0.2.
>
>Also, can you please confirm that Jetty (and your webapp) use the
>standard child-first (aka parent-last) classloading order? I know the
>Resin webserver doesn't, and that makes things quite different...
>
>
>And by the way, where do you put your log4j configuration file?
>
>
>
>
I load it programmatically with a call to
PropertyConfigurator.configureAndWatch(new File(basedir,
"logs/"+System.getProperty("loggging.file",
"log-production.properties")).toString());
in this way I can
1. change dynamically the logging configuration
2. have two sets of logging configurations: one for production (the
default) and other for development (overriden through a system
property in the debug launch). In this way, increasing log level
for testing purposes never goes to production inadvertently.
>Good luck!
>
>
Now I have only problems with log4j and some other static instances, but
not with commons-logging!!
Thanks for your invaluable help.
David
Re: Commons logging and memory leaks
Posted by Simon Kitching <sk...@apache.org>.
On Wed, 2005-05-18 at 12:20 +0200, David Perez wrote:
> You're right, I have changed the commons-logging version to the latest one.
> Now I have reverted to commons-logging-1.0.2
I would recommend using commons-logging-1.0.4. After all, that is the
most recent official release.
> >I suggest registering a ServletContextListener. Declare a <listener>
> >element in the web.xml file, with a nested <listener-class> element
> >containing the fully qualified classname of a logging-cleanup class. In
> >the contextDestroyed method of that class do this:
> > * LogFactory.release(
> > Thread.currentThread().getContextClassLoader())
> > * somehow cleaning up the underlying logging library if needed.
> >
> >If that doesn't help, let me know and we can look into it a bit
> >further.
> >
> >
> From my code I call LogFactory.releaseAll() after removing the old web
> app context, and no luck :-(
Do you mean that you use a ServletContextListener as I described? Or
something else?
Note also that LogFactory.releaseAll may not do what you want. If your
webapp has bound to commons-logging that is loaded via a shared
classloader, then calling releaseAll will shut down *all* instances of
commons-logging - which other webapps running in the same container may
not appreciate :-). The code I recommended will only shut down logging
for the webapp whose context is being destroyed, assuming you are
calling the method from a SerletContextListener.contextDestroyed, ie
from code where Thread.currentThread.getContextClassLoader() returns the
classloader of the webapp being unloaded.
> Should my webapp use another copy of commons-logging.jar?
That depends.
If your webapp has all the libraries it needs in WEB-INF/lib, then yes.
Deploying commons-logging.jar in WEB-INF/lib in that situation makes
things very clean and consistent. There's no need to use call
LogFactory.release() in this case, as undeploying the webapp
automatically undeploys commons-logging and the underlying logging lib.
But if your webapp is calling into code that is deployed via a shared
classloader, and that code uses commons-logging, then you must *not*
also deploy commons-logging in WEB-INF/lib. If you do (and you are using
release 1.0.4), you will get the message
Invalid class loader hierarchy.
You have more than one version of org.apache.commons.logging.Log
visible, which is not allowed.
If you are using an earlier release, you will probably get something
like:
AvalonLogger does not implement Log
> Here is the logging jars I use in my app:
>
> jars used by the web server (Jetty):
>
> * commons-logging.jar and
> * log4j.jar
>
> jars used by the webapp:
>
> * logkit.jar
>
> My web app uses Cocoon which uses Avalon LogKit, and makes me
> include logkit.jar, even though logging is redirected to log4j.
What do you mean "logging is redirected to log4j"?
> Some other classes use directly commons-logging
>
> With commons-logging-1.0.4 and 1.05 alpha1, I have to make logkit.jar
> available to Jetty, otherwise commons-loggings throws a
> ClassNotFoundException, because it cannot find
>
> org/apache/log/Hierarchy. I wonder why it tries to load LogKit if Log4j is already available?
Commons-logging does *not* auto-discover LogKit; it doesn't even try. So
somewhere something in your webapp must be *ordering* commons-logging to
use AvalonLogger (aka LogKit) as the logger.
Which logging lib commons-logging uses can be explicitly set via a
number of methods; .properties files, jdk1.3 service files, system
properties. See the commons-logging documentation for more information.
> Caused by: java.lang.NoClassDefFoundError: org/apache/log/Hierarchy
> at org.apache.commons.logging.impl.LogKitLogger.getLogger(LogKitLogger.java:77)
> at org.apache.commons.logging.impl.LogKitLogger.<init>(LogKitLogger.java:64)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
> at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:529)
Yep, commons-logging's LogFactoryImpl is explicitly trying to
instantiate a LogKitLogger object.
Can you please:
(a) look for what is ordering commons-logging to use LogKit and remove
that?
(b) use commons-logging 1.0.4; I will be able to help you much better
than if you use the very old 1.0.2.
Also, can you please confirm that Jetty (and your webapp) use the
standard child-first (aka parent-last) classloading order? I know the
Resin webserver doesn't, and that makes things quite different...
And by the way, where do you put your log4j configuration file?
Good luck!
Simon
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: Commons logging and memory leaks
Posted by David Perez <cr...@yahoo.es>.
Hi Simon,
Thanks for your answer.
See my comments below....
Simon Kitching wrote:
>
>Firstly, I presume you mean commons-logging-1.0.5-alpha1? Note that this
>really is rather experimental; I presume jetty didn't come with this
>version of commons-logging?
>
>
>
You're right, I have changed the commons-logging version to the latest one.
Now I have reverted to commons-logging-1.0.2
>There are some cases which the weak-reference stuff cannot handle. And
>even the weak-reference stuff works, commons logging cleans up but the
>underlying logging library may not.
>
>
>I suggest registering a ServletContextListener. Declare a <listener>
>element in the web.xml file, with a nested <listener-class> element
>containing the fully qualified classname of a logging-cleanup class. In
>the contextDestroyed method of that class do this:
> * LogFactory.release(
> Thread.currentThread().getContextClassLoader())
> * somehow cleaning up the underlying logging library if needed.
>
>If that doesn't help, let me know and we can look into it a bit
>further.
>
>
From my code I call LogFactory.releaseAll() after removing the old web
app context, and no luck :-(
Should my webapp use another copy of commons-logging.jar?
------------------------------------------------------------------------
Another issue:
I have noticed a strange class loading behaviour new to the latest
commons-logging releases.
Here is the logging jars I use in my app:
jars used by the web server (Jetty):
* commons-logging.jar and
* log4j.jar
jars used by the webapp:
* logkit.jar
My web app uses Cocoon which uses Avalon LogKit, and makes me
include logkit.jar, even though logging is redirected to log4j.
Some other classes use directly commons-logging
With commons-logging-1.0.4 and 1.05 alpha1, I have to make logkit.jar
available to Jetty, otherwise commons-loggings throws a
ClassNotFoundException, because it cannot find
org/apache/log/Hierarchy. I wonder why it tries to load LogKit if Log4j is already available?
F 18 12:08:54 <main> [fcc.ima.servidor2.ServidorIma] Error al iniciar
servidor
java.lang.ExceptionInInitializerError
at org.mortbay.util.Resource.newResource(Resource.java:73)
at org.mortbay.util.Resource.newSystemResource(Resource.java:186)
at org.mortbay.jetty.servlet.XMLConfiguration.configureDefaults(XMLConfiguration.java:151)
at org.mortbay.jetty.servlet.WebApplicationContext.configureDefaults(WebApplicationContext.java:415)
at org.mortbay.jetty.servlet.WebApplicationContext.doStart(WebApplicationContext.java:464)
at org.mortbay.util.Container.start(Container.java:72)
at org.mortbay.http.HttpServer.doStart(HttpServer.java:695)
at org.mortbay.util.Container.start(Container.java:72)
at fcc.ima.servidor2.ServidorIma.main(ServidorIma.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.mortbay.start.Main.invokeMain(Main.java:153)
at org.mortbay.start.Main.start(Main.java:483)
at org.mortbay.start.Main.main(Main.java:91)
at fcc.ima.servidor.Inicializacion.main(Inicializacion.java:34)
Caused by: org.apache.commons.logging.LogConfigurationException: java.lang.NoClassDefFoundError: org/apache/log/Hierarchy (Caused by java.lang.NoClassDefFoundError: org/apache/log/Hierarchy)
at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:538)
at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:235)
at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:209)
at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:351)
at org.mortbay.util.JarResource.<clinit>(JarResource.java:33)
... 17 more
Caused by: java.lang.NoClassDefFoundError: org/apache/log/Hierarchy
at org.apache.commons.logging.impl.LogKitLogger.getLogger(LogKitLogger.java:77)
at org.apache.commons.logging.impl.LogKitLogger.<init>(LogKitLogger.java:64)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:494)
at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:529)
... 21 more
On the other side, commons-logging-1.02 works ok.
Regards
David
Re: Commons logging and memory leaks
Posted by Simon Kitching <sk...@apache.org>.
Hi David,
On Wed, 2005-05-18 at 08:56 +0200, David Perez wrote:
> I have created a web app, integrated with a web server called Jetty.
> Both of them use commons-logging 1.5 alpha1.
> There is just one set of jars shared between them.
> The optional package is included, so weak references must be active.
> I have implemente webapp reload, by throwing the old class loader and
> creating a new one.
> I have observed with the help of a memory profiler, that the old class
> loader isn't memory garbaged.
>
> The cause is the log references. The garbage roots are that maintain
> live references are:
>
> * static HIERARCHY in org.apache.log.Hierarchy
> * static log in org.mortbay.util.URLResource and some other Jetty
> classes that use Commons Logging.
>
>
> I have tested under Java 1.5.0.03 in Windows XP.
>
> Can someone please give me some indications?
> Is it best to have separate commons-logging instances between my web app
> and the web server?
Firstly, I presume you mean commons-logging-1.0.5-alpha1? Note that this
really is rather experimental; I presume jetty didn't come with this
version of commons-logging?
There are some cases which the weak-reference stuff cannot handle. And
even the weak-reference stuff works, commons logging cleans up but the
underlying logging library may not.
I suggest registering a ServletContextListener. Declare a <listener>
element in the web.xml file, with a nested <listener-class> element
containing the fully qualified classname of a logging-cleanup class. In
the contextDestroyed method of that class do this:
* LogFactory.release(
Thread.currentThread().getContextClassLoader())
* somehow cleaning up the underlying logging library if needed.
If that doesn't help, let me know and we can look into it a bit
further.
Regards,
Simon
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org