You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by saumil shah <sa...@hotmail.com> on 2013/04/02 05:01:07 UTC
RE: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Hello there,
I recently deployed one of the COTS products SAP Business Objects. When the product was deployed , everything seemed to run fine but yesterday we started experiencing "Service is unavailable" error , upon enabling DEBUG logs in Tomcat , we saw the error below with memory leak message. We were wondering what could have caused it ....
The Tomcat is 64 bit , JVM is 64 bit but the applications deployed are 32 bit webapps .... we say Tomcat6 process becoming unresponsive around 2GB mark.
Your help is appreiated.
Many thanks,Sam
Apr 1, 2013 8:53:18 PM org.apache.coyote.http11.Http11Protocol pauseINFO: Pausing Coyote HTTP/1.1 on http-8080Apr 1, 2013 8:53:19 PM org.apache.catalina.core.StandardService stopINFO: Stopping service CatalinaApr 1, 2013 8:53:19 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/AnalyticalReporting] appears to have started a thread named [Business Objects - Sessions Clean up] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:19 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/AnalyticalReporting] appears to have started a thread named [AWT-Windows] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:19 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/AnalyticalReporting] appears to have started a thread named [ORBacus:Client:SenderThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:19 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/AnalyticalReporting] appears to have started a thread named [ORBacus:Client:ReceiverThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:19 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/AnalyticalReporting] appears to have started a thread named [ORBacus:Client:SenderThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:19 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/AnalyticalReporting] appears to have started a thread named [ORBacus:Client:ReceiverThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:19 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/AnalyticalReporting] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@63e3d2dd]) and a value of type [com.businessobjects.wp.xml.jaxp.XMLJaxpParser] (value [com.businessobjects.wp.xml.jaxp.XMLJaxpParser@228de67]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:19 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/AnalyticalReporting] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@63e3d2dd]) and a value of type [com.businessobjects.wp.xml.jaxp.XMLJaxpParser] (value [com.businessobjects.wp.xml.jaxp.XMLJaxpParser@49213d4c]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:19 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/AnalyticalReporting] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@63e3d2dd]) and a value of type [com.businessobjects.wp.xml.jaxp.XMLJaxpParser] (value [com.businessobjects.wp.xml.jaxp.XMLJaxpParser@253f6e16]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:19 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/AnalyticalReporting] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@63e3d2dd]) and a value of type [com.businessobjects.wp.xml.jaxp.XMLJaxpParser] (value [com.businessobjects.wp.xml.jaxp.XMLJaxpParser@67547974]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:19 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/AnalyticalReporting] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@63e3d2dd]) and a value of type [com.businessobjects.wp.xml.jaxp.XMLJaxpParser] (value [com.businessobjects.wp.xml.jaxp.XMLJaxpParser@5af1e3ab]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:19 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/AnalyticalReporting] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@63e3d2dd]) and a value of type [com.businessobjects.wp.xml.jaxp.XMLJaxpParser] (value [com.businessobjects.wp.xml.jaxp.XMLJaxpParser@3e9a1e32]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:19 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/AnalyticalReporting] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@63e3d2dd]) and a value of type [com.businessobjects.wp.xml.jaxp.XMLJaxpParser] (value [com.businessobjects.wp.xml.jaxp.XMLJaxpParser@783484b9]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:19 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/AnalyticalReporting] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@63e3d2dd]) and a value of type [com.businessobjects.wp.xml.jaxp.XMLJaxpParser] (value [com.businessobjects.wp.xml.jaxp.XMLJaxpParser@3ff5cb56]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/VoyagerClient] created a ThreadLocal with key of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@13123b1c]) and a value of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@11e6b50c]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/VoyagerClient] created a ThreadLocal with key of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@41783002]) and a value of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@3997f4e2]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/VoyagerClient] created a ThreadLocal with key of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@2081ff1]) and a value of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@63690841]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/VoyagerClient] created a ThreadLocal with key of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@13123b1c]) and a value of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@49d1664]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/VoyagerClient] created a ThreadLocal with key of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@41783002]) and a value of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@65290199]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/VoyagerClient] created a ThreadLocal with key of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@2081ff1]) and a value of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@672817b1]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/VoyagerClient] created a ThreadLocal with key of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@13123b1c]) and a value of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@792b9a5f]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/VoyagerClient] created a ThreadLocal with key of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@41783002]) and a value of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@1da4111f]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/VoyagerClient] created a ThreadLocal with key of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@2081ff1]) and a value of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@7e60196c]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/VoyagerClient] created a ThreadLocal with key of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@13123b1c]) and a value of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@57254245]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/VoyagerClient] created a ThreadLocal with key of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@41783002]) and a value of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@4f21ecb5]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/VoyagerClient] created a ThreadLocal with key of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@2081ff1]) and a value of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@3c3b87a9]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/VoyagerClient] created a ThreadLocal with key of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@13123b1c]) and a value of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@68478723]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/VoyagerClient] created a ThreadLocal with key of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@41783002]) and a value of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@2ff94051]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/VoyagerClient] created a ThreadLocal with key of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@2081ff1]) and a value of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@14ed9e72]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/VoyagerClient] created a ThreadLocal with key of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@13123b1c]) and a value of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@7c3d5919]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/VoyagerClient] created a ThreadLocal with key of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@41783002]) and a value of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@1fa4b808]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/VoyagerClient] created a ThreadLocal with key of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@2081ff1]) and a value of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@772e2572]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMapSEVERE: The web application [/VoyagerClient] created a ThreadLocal with key of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@13123b1c]) and a value of type [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter] (value [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@769c9c7e]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/CrystalReports] appears to have started a thread named [Business Objects - Sessions Clean up] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/CrystalReports] appears to have started a thread named [Thread-18] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/CrystalReports] appears to have started a thread named [ORBacus:Client:SenderThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/CrystalReports] appears to have started a thread named [ORBacus:Client:ReceiverThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/CrystalReports] appears to have started a thread named [ORBacus:Client:SenderThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/CrystalReports] appears to have started a thread named [ORBacus:Client:ReceiverThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/CrystalReports] appears to have started a thread named [ORBacus:Server:StarterThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/CmcApp] appears to have started a thread named [Business Objects - Sessions Clean up] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/CmcApp] appears to have started a thread named [ORBacus:Client:SenderThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/CmcApp] appears to have started a thread named [ORBacus:Client:ReceiverThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/CmcApp] appears to have started a thread named [ORBacus:Client:SenderThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/CmcApp] appears to have started a thread named [ORBacus:Client:ReceiverThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/AdminTools] appears to have started a thread named [Business Objects - Sessions Clean up] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/AdminTools] appears to have started a thread named [ORBacus:Client:SenderThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/AdminTools] appears to have started a thread named [ORBacus:Client:ReceiverThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/AdminTools] appears to have started a thread named [ORBacus:Client:SenderThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:20 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/AdminTools] appears to have started a thread named [ORBacus:Client:ReceiverThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:21 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/InfoViewApp] appears to have started a thread named [Business Objects - Sessions Clean up] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:21 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/InfoViewApp] appears to have started a thread named [ORBacus:Client:SenderThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:21 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/InfoViewApp] appears to have started a thread named [ORBacus:Client:ReceiverThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:21 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/InfoViewApp] appears to have started a thread named [ORBacus:Client:SenderThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:21 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/InfoViewApp] appears to have started a thread named [ORBacus:Client:ReceiverThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:21 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/PerformanceManagement] appears to have started a thread named [Business Objects - Sessions Clean up] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:21 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/PerformanceManagement] appears to have started a thread named [ORBacus:Client:SenderThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:21 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/PerformanceManagement] appears to have started a thread named [ORBacus:Client:ReceiverThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:21 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/PerformanceManagement] appears to have started a thread named [ORBacus:Client:SenderThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:21 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/PerformanceManagement] appears to have started a thread named [ORBacus:Client:ReceiverThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:21 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/InfoViewAppActions] appears to have started a thread named [Business Objects - Sessions Clean up] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:21 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/InfoViewAppActions] appears to have started a thread named [ORBacus:Client:SenderThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:21 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/InfoViewAppActions] appears to have started a thread named [ORBacus:Client:ReceiverThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:21 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/InfoViewAppActions] appears to have started a thread named [ORBacus:Client:SenderThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:21 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/InfoViewAppActions] appears to have started a thread named [ORBacus:Client:ReceiverThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:21 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/dswsbobje] appears to have started a thread named [Timer-1] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:22 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/CmcAppActions] appears to have started a thread named [Business Objects - Sessions Clean up] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:22 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/PlatformServices] appears to have started a thread named [Business Objects - Sessions Clean up] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:22 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/PlatformServices] appears to have started a thread named [ORBacus:Client:SenderThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:22 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/PlatformServices] appears to have started a thread named [ORBacus:Client:ReceiverThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:22 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/PlatformServices] appears to have started a thread named [ORBacus:Client:SenderThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:22 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreadsSEVERE: The web application [/PlatformServices] appears to have started a thread named [ORBacus:Client:ReceiverThread] but has failed to stop it. This is very likely to create a memory leak.Apr 1, 2013 8:53:22 PM org.apache.coyote.http11.Http11Protocol destroyINFO: Stopping Coyote HTTP/1.1 on http-8080Apr 1, 2013 8:53:26 PM org.apache.catalina.core.AprLifecycleListener initINFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:\Windows\SysWOW64\;D:\Apps\Business Objects\BusinessObjects Enterprise 12.0\win32_x86\Apr 1, 2013 8:53:26 PM org.apache.coyote.http11.Http11Protocol initINFO: Initializing Coyote HTTP/1.1 on http-8080Apr 1, 2013 8:53:26 PM org.apache.catalina.startup.Catalina loadINFO: Initialization processed in 490 msApr 1, 2013 8:53:26 PM org.apache.catalina.core.StandardService startINFO: Starting service CatalinaApr 1, 2013 8:53:26 PM org.apache.catalina.core.StandardEngine startINFO: Starting Servlet Engine: Apache TomcatApr 1, 2013 8:53:26 PM org.apache.catalina.startup.HostConfig deployWARINFO: Deploying web application archive AdminTools.warApr 1, 2013 8:53:26 PM org.apache.catalina.startup.HostConfig deployWARINFO: Deploying web application archive AnalyticalReporting.warApr 1, 2013 8:53:27 PM org.apache.catalina.core.StandardContext addApplicationListenerINFO: The listener "com.sun.faces.config.ConfigureListener" is already configured for this context. The duplicate definition has been ignored.Apr 1, 2013 8:53:28 PM org.apache.catalina.startup.HostConfig deployWARINFO: Deploying web application archive BusinessProcessBI.warApr 1, 2013 8:53:29 PM org.apache.catalina.startup.HostConfig deployWARINFO: Deploying web application archive CmcApp.warApr 1, 2013 8:53:29 PM org.apache.catalina.core.StandardContext addApplicationListenerINFO: The listener "com.sun.faces.config.ConfigureListener" is already configured for this context. The duplicate definition has been ignored.Apr 1, 2013 8:53:29 PM org.apache.catalina.startup.HostConfig deployWARINFO: Deploying web application archive CmcAppActions.warApr 1, 2013 8:53:29 PM org.apache.catalina.core.StandardContext addApplicationListenerINFO: The listener "com.sun.faces.config.ConfigureListener" is already configured for this context. The duplicate definition has been ignored.Apr 1, 2013 8:53:29 PM org.apache.catalina.startup.HostConfig deployWARINFO: Deploying web application archive CrystalReports.warApr 1, 2013 8:53:30 PM org.apache.catalina.core.StandardContext addApplicationListenerINFO: The listener "com.sun.faces.config.ConfigureListener" is already configured for this context. The duplicate definition has been ignored.Apr 1, 2013 8:53:30 PM org.apache.catalina.startup.HostConfig deployWARINFO: Deploying web application archive dswsbobje.warApr 1, 2013 8:53:32 PM org.apache.catalina.startup.HostConfig deployWARINFO: Deploying web application archive InfoViewApp.warApr 1, 2013 8:53:32 PM org.apache.catalina.core.StandardContext addApplicationListenerINFO: The listener "com.sun.faces.config.ConfigureListener" is already configured for this context. The duplicate definition has been ignored.Apr 1, 2013 8:53:33 PM org.apache.catalina.startup.HostConfig deployWARINFO: Deploying web application archive InfoViewAppActions.warApr 1, 2013 8:53:33 PM org.apache.catalina.core.StandardContext addApplicationListenerINFO: The listener "com.sun.faces.config.ConfigureListener" is already configured for this context. The duplicate definition has been ignored.Apr 1, 2013 8:53:33 PM org.apache.catalina.startup.HostConfig deployWARINFO: Deploying web application archive jsfplatform.warApr 1, 2013 8:53:33 PM org.apache.catalina.core.StandardContext addApplicationListenerINFO: The listener "com.sun.faces.config.ConfigureListener" is already configured for this context. The duplicate definition has been ignored.Apr 1, 2013 8:53:34 PM org.apache.catalina.startup.HostConfig deployWARINFO: Deploying web application archive OpenDocument.warApr 1, 2013 8:53:34 PM org.apache.catalina.startup.HostConfig deployWARINFO: Deploying web application archive PerformanceManagement.warApr 1, 2013 8:53:34 PM org.apache.catalina.core.StandardContext addApplicationListenerINFO: The listener "com.sun.faces.config.ConfigureListener" is already configured for this context. The duplicate definition has been ignored.Apr 1, 2013 8:56:19 PM org.apache.catalina.startup.HostConfig deployWARINFO: Deploying web application archive PlatformServices.warApr 1, 2013 8:56:20 PM org.apache.catalina.core.StandardContext addApplicationListenerINFO: The listener "com.sun.faces.config.ConfigureListener" is already configured for this context. The duplicate definition has been ignored.Apr 1, 2013 8:56:20 PM org.apache.catalina.startup.HostConfig deployWARINFO: Deploying web application archive PMC_Help.warApr 1, 2013 8:56:20 PM org.apache.catalina.startup.HostConfig deployWARINFO: Deploying web application archive VoyagerClient.warApr 1, 2013 8:56:20 PM org.apache.catalina.core.StandardContext addApplicationListenerINFO: The listener "com.sun.faces.config.ConfigureListener" is already configured for this context. The duplicate definition has been ignored.Apr 1, 2013 8:56:21 PM org.apache.catalina.startup.HostConfig deployWARINFO: Deploying web application archive Xcelsius.warApr 1, 2013 8:56:21 PM org.apache.catalina.startup.HostConfig deployWARINFO: Deploying web application archive XCTemplateUploader.warApr 1, 2013 8:56:21 PM org.apache.catalina.core.StandardContext addApplicationListenerINFO: The listener "com.sun.faces.config.ConfigureListener" is already configured for this context. The duplicate definition has been ignored.Apr 1, 2013 8:56:21 PM com.sun.faces.config.rules.ConverterRule endWARNING: [ConverterRule]{faces-config/converter} Merge(null,java.math.BigDecimal)Apr 1, 2013 8:56:21 PM com.sun.faces.config.rules.ConverterRule endWARNING: [ConverterRule]{faces-config/converter} Merge(null,java.math.BigInteger)Apr 1, 2013 8:56:21 PM com.sun.faces.config.rules.ManagedBeanRule endWARNING: [ManagedBeanRule]{faces-config/managed-bean} Merge(xcuser)Apr 1, 2013 8:56:21 PM com.sun.faces.config.rules.ManagedBeanRule endWARNING: [ManagedBeanRule]{faces-config/managed-bean} Merge(xctemplate)Apr 1, 2013 8:56:21 PM com.sun.faces.config.rules.ManagedBeanRule endWARNING: [ManagedBeanRule]{faces-config/managed-bean} Merge(xctemplatezip)Apr 1, 2013 8:56:21 PM com.sun.faces.config.rules.ManagedBeanRule endWARNING: [ManagedBeanRule]{faces-config/managed-bean} Merge(xctemplatelist)Apr 1, 2013 8:56:21 PM com.sun.faces.config.rules.NavigationRuleRule endWARNING: [NavigationRuleRule]{faces-config/navigation-rule} Merge(/pages/login.jsp)Apr 1, 2013 8:56:21 PM com.sun.faces.config.rules.NavigationRuleRule endWARNING: [NavigationRuleRule]{faces-config/navigation-rule} Merge(/pages/uploaderForm.jsp)Apr 1, 2013 8:56:21 PM com.sun.faces.config.rules.NavigationRuleRule endWARNING: [NavigationRuleRule]{faces-config/navigation-rule} Merge(/pages/uploaded.jsp)Apr 1, 2013 8:56:21 PM com.sun.faces.config.rules.NavigationRuleRule endWARNING: [NavigationRuleRule]{faces-config/navigation-rule} Merge(/pages/customPropsForm.jsp)Apr 1, 2013 8:56:21 PM com.sun.faces.config.rules.NavigationRuleRule endWARNING: [NavigationRuleRule]{faces-config/navigation-rule} Merge(/pages/uploaderZipForm.jsp)Apr 1, 2013 8:56:21 PM com.sun.faces.config.rules.NavigationRuleRule endWARNING: [NavigationRuleRule]{faces-config/navigation-rule} Merge(*)Apr 1, 2013 8:56:21 PM com.sun.faces.config.rules.NavigationRuleRule endWARNING: [NavigationRuleRule]{faces-config/navigation-rule} Merge(/pages/mainMenu.jsp)Apr 1, 2013 8:56:21 PM org.apache.coyote.http11.Http11Protocol startINFO: Starting Coyote HTTP/1.1 on http-8080Apr 1, 2013 8:56:21 PM org.apache.jk.common.ChannelSocket initINFO: JK: ajp13 listening on /0.0.0.0:8009Apr 1, 2013 8:56:21 PM org.apache.jk.server.JkMain startINFO: Jk running ID=0 time=0/16 config=nullApr 1, 2013 8:56:21 PM org.apache.catalina.startup.Catalina startINFO: Server startup in 175414 ms
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by Ognjen Blagojevic <og...@gmail.com>.
Saumil,
On 2.4.2013 5:01, saumil shah wrote:
> I recently deployed one of the COTS products SAP Business Objects. When the product was deployed , everything seemed to run fine but yesterday we started experiencing "Service is unavailable" error , upon enabling DEBUG logs in Tomcat , we saw the error below with memory leak message. We were wondering what could have caused it ....
> The Tomcat is 64 bit , JVM is 64 bit but the applications deployed are 32 bit webapps .... we say Tomcat6 process becoming unresponsive around 2GB mark.
There is a Wiki page [1] that descibes most couses of memory leaks.
Until webapp developers check and fix those leaks, you might mitigate
them by restarting Tomcat every time you need to redeploy or restart any
of your webapps. If that does not help, then you have some other
problem. You could investigate further by analyzing thread activity
during hangs using thread dumps [2] or monitoring tools like JConsole.
-Ognjen
[1] https://wiki.apache.org/tomcat/MemoryLeakProtection
[2]
https://wiki.apache.org/tomcat/HowTo#How_do_I_obtain_a_thread_dump_of_my_running_webapp_.3F
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by André Warnier <aw...@ice-sa.com>.
saumil shah wrote:
> Hello there,
>
> I recently deployed one of the COTS products SAP Business Objects. When the product was deployed , everything seemed to run fine but yesterday we started experiencing "Service is unavailable" error , upon enabling DEBUG logs in Tomcat , we saw the error below with memory leak message. We were wondering what could have caused it ....
> The Tomcat is 64 bit
(huh ?)
, JVM is 64 bit but the applications deployed are 32 bit webapps
(huh ?)
.... we say Tomcat6 process becoming unresponsive around 2GB mark.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by "Howard W. Smith, Jr." <sm...@gmail.com>.
On Mon, Apr 15, 2013 at 11:18 AM, Mark Eggers <it...@yahoo.com> wrote:
> On 4/15/2013 7:25 AM, David kerber wrote:
>
>> On 4/15/2013 10:10 AM, Howard W. Smith, Jr. wrote:
>>
>>> On Mon, Apr 15, 2013 at 7:40 AM, David kerber<dc...@verizon.net>
>>> wrote:
>>>
>>> On 4/14/2013 11:10 PM, Howard W. Smith, Jr. wrote:
>>>>
>>>> On Sun, Apr 14, 2013 at 10:52 PM, Mark Thomas<ma...@apache.org>
>>>>> wrote:
>>>>>
>>>>> On 14/04/2013 21:53, Howard W. Smith, Jr. wrote:
>>>>>
>>>>>>
>>>>>> On Sun, Apr 14, 2013 at 6:51 PM, Christopher Schultz<
>>>>>>
>>>>>>> chris@christopherschultz.net> wrote:
>>>>>>>
>>>>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>>>>>
>>>>>>> Hash: SHA256
>>>>>>>>
>>>>>>>> Howard,
>>>>>>>>
>>>>>>>> On 4/11/13 10:38 PM, Howard W. Smith, Jr. wrote:
>>>>>>>>
>>>>>>>> On Thu, Apr 4, 2013 at 2:32 PM, Christopher Schultz<
>>>>>>>>
>>>>>>>>> chris@christopherschultz.net> wrote:
>>>>>>>>>
>>>>>>>>> Your heap settings should be tailored to your environment and
>>>>>>>>>
>>>>>>>>> usage scenarios.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Interesting. I suppose 'your environment' means memory available,
>>>>>>>>> operating system, hardware. Usage scenarios? hmmm... please clarify
>>>>>>>>> with a brief example, thanks. :)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Here's an example: Let's say that your webapp doesn't use
>>>>>>>> HttpSessions
>>>>>>>> and does no caching. You need to be able to handle 100 simultaneous
>>>>>>>> connections that do small fetches/inserts from/into a relational
>>>>>>>> database. Your pages are fairly simple and don't have any kind of
>>>>>>>> heavyweight app framework taking-up a whole bunch of memory to do
>>>>>>>> simple things.
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks Chris for the example. This is definitely not my app. I am
>>>>>>>>
>>>>>>> definitely relying on user HttpSessions, and I do JPA-level caching
>>>>>>> (statement cache and query results cache). pages are PrimeFaces and
>>>>>>> primefaces = xhtml, html, jquery, and MyFaces/OpenWebBeans to help
>>>>>>> with
>>>>>>> speed/performance. And right now, the app handles on a 'few'
>>>>>>> simultaneous
>>>>>>> connections/users that do small and large fetches/inserts from/into
>>>>>>> relational database. :)
>>>>>>>
>>>>>>> Hopefully one day, my app will be support 100+ simultaneous
>>>>>>> connections/users.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> For this situation, you can probably get away with a 64MiB
>>>>>>> heap. If
>>>>>>>
>>>>>>> your webapp uses more than 64MiB, there is probably some kind of
>>>>>>>> problem. If you only need a 64MiB heap, then you can probably run on
>>>>>>>> fairly modest hardware: there's no need to lease that 128GiB server
>>>>>>>> your vendor is trying to talk you into.
>>>>>>>>
>>>>>>>>
>>>>>>>> Understood, thanks. I have Xms/Xmx = 1024m, and I rarely see used
>>>>>>>>
>>>>>>> memory
>>>>>>> get over 400 or 500m. the production server has 32GB RAM.
>>>>>>>
>>>>>>>
>>>>>>> I'll summarize a number of JavaOne sesisons I've been to on GC and
>>>>>> performance (caveat - this was a couple of years ago and GC design has
>>>>>> moved on since then).
>>>>>>
>>>>>> - GC pause time
>>>>>> - throughput
>>>>>> - small memory footprint
>>>>>>
>>>>>> You can optimise for any two of the above at the expense of the third.
>>>>>>
>>>>>> Assuming you opt for min GC pause time and max throughput the question
>>>>>> then becomes how much heap do you need? If you look at your steady
>>>>>> state
>>>>>> heap usage graph (it should be a saw-tooth) then take the heap
>>>>>> usage at
>>>>>> the
>>>>>> bottom of the saw-tooth and multiply it by 5 - that is the heap
>>>>>> size you
>>>>>> should use for the GC to work optimally.
>>>>>>
>>>>>> HTH,
>>>>>>
>>>>>> Mark
>>>>>>
>>>>>>
>>>>>> Interesting, that does help, Mark, thanks. 250 x 5 = 1,250. I
>>>>>> guess I
>>>>>>
>>>>> was
>>>>> pretty close on target when I set Xms/Xmx = 1024m.
>>>>>
>>>>> Prior to seeing your email/response, I checked the server again, and it
>>>>> was
>>>>> no saw-tooth at all, it was at 250 (bottom), and then saw-tooth
>>>>> graph came
>>>>> into play...minutes later.
>>>>>
>>>>>
>>>> Make sure you give it enough time for the memory use to stabilize.
>>>>
>>>
>>>
>>> Will do (and doing that), thanks. :)
>>>
>>>
>>> Depending on your app and usage patterns, it can take up to days for the
>>>> sawtooth to stabilize and start showing. One of mine takes a couple of
>>>> hours, and another a few days for that pattern to become visible.
>>>>
>>>
>>>
>>> I see it stabilize 'in minutes' (after/during usage of the app).
>>>
>>> Just now (prior to writing this email), I was looking at the app's usage
>>> (via monitoring the app's own data/record audit trail page), and then
>>> decided to check-on the app to see how it is doing/performing via
>>> jvisualvm, and voila, again, I saw no saw-tooth[1].
>>>
>>> I saw this, 5 to 15 minutes after a period of inactivity in the app, but
>>> before I logged into the app, as I stated above, I checked the app's
>>> audit
>>> trail (which can definitely be a 'heavy-lifting' database query,
>>> depending
>>> on work done within the app on selected date, default = current date),
>>> and
>>> it[1] still showed no activity (or saw-tooth); I assume activity
>>> within the
>>> app can = definite/obvious saw-tooth graph (which also means, GC is
>>> working
>>> while app is being used).
>>>
>>> What I mentioned above is very normal behavior for my app.
>>>
>>> [1] http://img805.imageshack.us/**img805/8453/**20130415jvisualvm01.png<http://img805.imageshack.us/img805/8453/20130415jvisualvm01.png>
>>>
>>
>> These graphs are only showing ~40 seconds of data. I'll bet if you let
>> the app run for several minutes or hours, you'll see it.
>>
>>
> Yep, there's no history in that data.
>
Agreed! :)
>
> What you can do (probably in a test environment) is the following:
>
> 1. Set up monitoring (visualvm, psi-probe, jconsole)
> 2. Abuse your application with well-crafted JMeter (or other) tests
> 3. Watch memory
>
Interesting. I will have to try that, at some point. Thanks.
>
> This becomes a little more challenging with AJAX-style applications (yours
> is a PrimeFaces / JSF / AJAX one, right?), but I've seen some notes on
> this. Google is your friend.
>
Yes, PrimeFaces, but not much AJAX at all. my endusers went from using an
almost 20-year-old MS-DOS dBase IV app to a Java/JSF web application. They
don't need AJAX (partial page refreshes), so they are very happy with full
page refreshes. :)
I implemented security and session management in my app via some approaches
that I got from BalusC on stackoverflow.com. so, Full page refresh (FPR) is
necessary to refresh the page,
<meta http-equiv="refresh"
content="#{session.maxInactiveInterval};url=#{pf_usersController.isLoggedInViaAndroid()
? 'pf_viewExpiredOnMobile.jsf' : 'pf_viewExpired.jsf'}" />
so, session can timeout/expire according to the follow context param, set
in web.xml.
<!-- session-timeout = 15 minutes -->
<session-config>
<session-timeout>
15
</session-timeout>
</session-config>
this meets my requirements for session timeout/expire/management.
> . . . . just my two cents.
> /mde/
>
>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.**apache.org<us...@tomcat.apache.org>
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by Mark Eggers <it...@yahoo.com>.
On 4/15/2013 7:25 AM, David kerber wrote:
> On 4/15/2013 10:10 AM, Howard W. Smith, Jr. wrote:
>> On Mon, Apr 15, 2013 at 7:40 AM, David kerber<dc...@verizon.net>
>> wrote:
>>
>>> On 4/14/2013 11:10 PM, Howard W. Smith, Jr. wrote:
>>>
>>>> On Sun, Apr 14, 2013 at 10:52 PM, Mark Thomas<ma...@apache.org>
>>>> wrote:
>>>>
>>>> On 14/04/2013 21:53, Howard W. Smith, Jr. wrote:
>>>>>
>>>>> On Sun, Apr 14, 2013 at 6:51 PM, Christopher Schultz<
>>>>>> chris@christopherschultz.net> wrote:
>>>>>>
>>>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>>>>
>>>>>>> Hash: SHA256
>>>>>>>
>>>>>>> Howard,
>>>>>>>
>>>>>>> On 4/11/13 10:38 PM, Howard W. Smith, Jr. wrote:
>>>>>>>
>>>>>>> On Thu, Apr 4, 2013 at 2:32 PM, Christopher Schultz<
>>>>>>>> chris@christopherschultz.net> wrote:
>>>>>>>>
>>>>>>>> Your heap settings should be tailored to your environment and
>>>>>>>>
>>>>>>>>> usage scenarios.
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Interesting. I suppose 'your environment' means memory available,
>>>>>>>> operating system, hardware. Usage scenarios? hmmm... please clarify
>>>>>>>> with a brief example, thanks. :)
>>>>>>>>
>>>>>>>>
>>>>>>> Here's an example: Let's say that your webapp doesn't use
>>>>>>> HttpSessions
>>>>>>> and does no caching. You need to be able to handle 100 simultaneous
>>>>>>> connections that do small fetches/inserts from/into a relational
>>>>>>> database. Your pages are fairly simple and don't have any kind of
>>>>>>> heavyweight app framework taking-up a whole bunch of memory to do
>>>>>>> simple things.
>>>>>>>
>>>>>>>
>>>>>>> Thanks Chris for the example. This is definitely not my app. I am
>>>>>> definitely relying on user HttpSessions, and I do JPA-level caching
>>>>>> (statement cache and query results cache). pages are PrimeFaces and
>>>>>> primefaces = xhtml, html, jquery, and MyFaces/OpenWebBeans to help
>>>>>> with
>>>>>> speed/performance. And right now, the app handles on a 'few'
>>>>>> simultaneous
>>>>>> connections/users that do small and large fetches/inserts from/into
>>>>>> relational database. :)
>>>>>>
>>>>>> Hopefully one day, my app will be support 100+ simultaneous
>>>>>> connections/users.
>>>>>>
>>>>>>
>>>>>>
>>>>>> For this situation, you can probably get away with a 64MiB
>>>>>> heap. If
>>>>>>
>>>>>>> your webapp uses more than 64MiB, there is probably some kind of
>>>>>>> problem. If you only need a 64MiB heap, then you can probably run on
>>>>>>> fairly modest hardware: there's no need to lease that 128GiB server
>>>>>>> your vendor is trying to talk you into.
>>>>>>>
>>>>>>>
>>>>>>> Understood, thanks. I have Xms/Xmx = 1024m, and I rarely see used
>>>>>> memory
>>>>>> get over 400 or 500m. the production server has 32GB RAM.
>>>>>>
>>>>>>
>>>>> I'll summarize a number of JavaOne sesisons I've been to on GC and
>>>>> performance (caveat - this was a couple of years ago and GC design has
>>>>> moved on since then).
>>>>>
>>>>> - GC pause time
>>>>> - throughput
>>>>> - small memory footprint
>>>>>
>>>>> You can optimise for any two of the above at the expense of the third.
>>>>>
>>>>> Assuming you opt for min GC pause time and max throughput the question
>>>>> then becomes how much heap do you need? If you look at your steady
>>>>> state
>>>>> heap usage graph (it should be a saw-tooth) then take the heap
>>>>> usage at
>>>>> the
>>>>> bottom of the saw-tooth and multiply it by 5 - that is the heap
>>>>> size you
>>>>> should use for the GC to work optimally.
>>>>>
>>>>> HTH,
>>>>>
>>>>> Mark
>>>>>
>>>>>
>>>>> Interesting, that does help, Mark, thanks. 250 x 5 = 1,250. I
>>>>> guess I
>>>> was
>>>> pretty close on target when I set Xms/Xmx = 1024m.
>>>>
>>>> Prior to seeing your email/response, I checked the server again, and it
>>>> was
>>>> no saw-tooth at all, it was at 250 (bottom), and then saw-tooth
>>>> graph came
>>>> into play...minutes later.
>>>>
>>>
>>> Make sure you give it enough time for the memory use to stabilize.
>>
>>
>> Will do (and doing that), thanks. :)
>>
>>
>>> Depending on your app and usage patterns, it can take up to days for the
>>> sawtooth to stabilize and start showing. One of mine takes a couple of
>>> hours, and another a few days for that pattern to become visible.
>>
>>
>> I see it stabilize 'in minutes' (after/during usage of the app).
>>
>> Just now (prior to writing this email), I was looking at the app's usage
>> (via monitoring the app's own data/record audit trail page), and then
>> decided to check-on the app to see how it is doing/performing via
>> jvisualvm, and voila, again, I saw no saw-tooth[1].
>>
>> I saw this, 5 to 15 minutes after a period of inactivity in the app, but
>> before I logged into the app, as I stated above, I checked the app's
>> audit
>> trail (which can definitely be a 'heavy-lifting' database query,
>> depending
>> on work done within the app on selected date, default = current date),
>> and
>> it[1] still showed no activity (or saw-tooth); I assume activity
>> within the
>> app can = definite/obvious saw-tooth graph (which also means, GC is
>> working
>> while app is being used).
>>
>> What I mentioned above is very normal behavior for my app.
>>
>> [1] http://img805.imageshack.us/img805/8453/20130415jvisualvm01.png
>
> These graphs are only showing ~40 seconds of data. I'll bet if you let
> the app run for several minutes or hours, you'll see it.
>
Yep, there's no history in that data.
What you can do (probably in a test environment) is the following:
1. Set up monitoring (visualvm, psi-probe, jconsole)
2. Abuse your application with well-crafted JMeter (or other) tests
3. Watch memory
This becomes a little more challenging with AJAX-style applications
(yours is a PrimeFaces / JSF / AJAX one, right?), but I've seen some
notes on this. Google is your friend.
. . . . just my two cents.
/mde/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by David kerber <dc...@verizon.net>.
On 4/15/2013 10:10 AM, Howard W. Smith, Jr. wrote:
> On Mon, Apr 15, 2013 at 7:40 AM, David kerber<dc...@verizon.net> wrote:
>
>> On 4/14/2013 11:10 PM, Howard W. Smith, Jr. wrote:
>>
>>> On Sun, Apr 14, 2013 at 10:52 PM, Mark Thomas<ma...@apache.org> wrote:
>>>
>>> On 14/04/2013 21:53, Howard W. Smith, Jr. wrote:
>>>>
>>>> On Sun, Apr 14, 2013 at 6:51 PM, Christopher Schultz<
>>>>> chris@christopherschultz.net> wrote:
>>>>>
>>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>>>
>>>>>> Hash: SHA256
>>>>>>
>>>>>> Howard,
>>>>>>
>>>>>> On 4/11/13 10:38 PM, Howard W. Smith, Jr. wrote:
>>>>>>
>>>>>> On Thu, Apr 4, 2013 at 2:32 PM, Christopher Schultz<
>>>>>>> chris@christopherschultz.net> wrote:
>>>>>>>
>>>>>>> Your heap settings should be tailored to your environment and
>>>>>>>
>>>>>>>> usage scenarios.
>>>>>>>>
>>>>>>>>
>>>>>>> Interesting. I suppose 'your environment' means memory available,
>>>>>>> operating system, hardware. Usage scenarios? hmmm... please clarify
>>>>>>> with a brief example, thanks. :)
>>>>>>>
>>>>>>>
>>>>>> Here's an example: Let's say that your webapp doesn't use HttpSessions
>>>>>> and does no caching. You need to be able to handle 100 simultaneous
>>>>>> connections that do small fetches/inserts from/into a relational
>>>>>> database. Your pages are fairly simple and don't have any kind of
>>>>>> heavyweight app framework taking-up a whole bunch of memory to do
>>>>>> simple things.
>>>>>>
>>>>>>
>>>>>> Thanks Chris for the example. This is definitely not my app. I am
>>>>> definitely relying on user HttpSessions, and I do JPA-level caching
>>>>> (statement cache and query results cache). pages are PrimeFaces and
>>>>> primefaces = xhtml, html, jquery, and MyFaces/OpenWebBeans to help with
>>>>> speed/performance. And right now, the app handles on a 'few'
>>>>> simultaneous
>>>>> connections/users that do small and large fetches/inserts from/into
>>>>> relational database. :)
>>>>>
>>>>> Hopefully one day, my app will be support 100+ simultaneous
>>>>> connections/users.
>>>>>
>>>>>
>>>>>
>>>>> For this situation, you can probably get away with a 64MiB heap. If
>>>>>
>>>>>> your webapp uses more than 64MiB, there is probably some kind of
>>>>>> problem. If you only need a 64MiB heap, then you can probably run on
>>>>>> fairly modest hardware: there's no need to lease that 128GiB server
>>>>>> your vendor is trying to talk you into.
>>>>>>
>>>>>>
>>>>>> Understood, thanks. I have Xms/Xmx = 1024m, and I rarely see used
>>>>> memory
>>>>> get over 400 or 500m. the production server has 32GB RAM.
>>>>>
>>>>>
>>>> I'll summarize a number of JavaOne sesisons I've been to on GC and
>>>> performance (caveat - this was a couple of years ago and GC design has
>>>> moved on since then).
>>>>
>>>> - GC pause time
>>>> - throughput
>>>> - small memory footprint
>>>>
>>>> You can optimise for any two of the above at the expense of the third.
>>>>
>>>> Assuming you opt for min GC pause time and max throughput the question
>>>> then becomes how much heap do you need? If you look at your steady state
>>>> heap usage graph (it should be a saw-tooth) then take the heap usage at
>>>> the
>>>> bottom of the saw-tooth and multiply it by 5 - that is the heap size you
>>>> should use for the GC to work optimally.
>>>>
>>>> HTH,
>>>>
>>>> Mark
>>>>
>>>>
>>>> Interesting, that does help, Mark, thanks. 250 x 5 = 1,250. I guess I
>>> was
>>> pretty close on target when I set Xms/Xmx = 1024m.
>>>
>>> Prior to seeing your email/response, I checked the server again, and it
>>> was
>>> no saw-tooth at all, it was at 250 (bottom), and then saw-tooth graph came
>>> into play...minutes later.
>>>
>>
>> Make sure you give it enough time for the memory use to stabilize.
>
>
> Will do (and doing that), thanks. :)
>
>
>> Depending on your app and usage patterns, it can take up to days for the
>> sawtooth to stabilize and start showing. One of mine takes a couple of
>> hours, and another a few days for that pattern to become visible.
>
>
> I see it stabilize 'in minutes' (after/during usage of the app).
>
> Just now (prior to writing this email), I was looking at the app's usage
> (via monitoring the app's own data/record audit trail page), and then
> decided to check-on the app to see how it is doing/performing via
> jvisualvm, and voila, again, I saw no saw-tooth[1].
>
> I saw this, 5 to 15 minutes after a period of inactivity in the app, but
> before I logged into the app, as I stated above, I checked the app's audit
> trail (which can definitely be a 'heavy-lifting' database query, depending
> on work done within the app on selected date, default = current date), and
> it[1] still showed no activity (or saw-tooth); I assume activity within the
> app can = definite/obvious saw-tooth graph (which also means, GC is working
> while app is being used).
>
> What I mentioned above is very normal behavior for my app.
>
> [1] http://img805.imageshack.us/img805/8453/20130415jvisualvm01.png
These graphs are only showing ~40 seconds of data. I'll bet if you let
the app run for several minutes or hours, you'll see it.
>
>
>
>>
>>
>> ------------------------------**------------------------------**---------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.**apache.org<us...@tomcat.apache.org>
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by "Howard W. Smith, Jr." <sm...@gmail.com>.
On Mon, Apr 15, 2013 at 7:40 AM, David kerber <dc...@verizon.net> wrote:
> On 4/14/2013 11:10 PM, Howard W. Smith, Jr. wrote:
>
>> On Sun, Apr 14, 2013 at 10:52 PM, Mark Thomas<ma...@apache.org> wrote:
>>
>> On 14/04/2013 21:53, Howard W. Smith, Jr. wrote:
>>>
>>> On Sun, Apr 14, 2013 at 6:51 PM, Christopher Schultz<
>>>> chris@christopherschultz.net> wrote:
>>>>
>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>>
>>>>> Hash: SHA256
>>>>>
>>>>> Howard,
>>>>>
>>>>> On 4/11/13 10:38 PM, Howard W. Smith, Jr. wrote:
>>>>>
>>>>> On Thu, Apr 4, 2013 at 2:32 PM, Christopher Schultz<
>>>>>> chris@christopherschultz.net> wrote:
>>>>>>
>>>>>> Your heap settings should be tailored to your environment and
>>>>>>
>>>>>>> usage scenarios.
>>>>>>>
>>>>>>>
>>>>>> Interesting. I suppose 'your environment' means memory available,
>>>>>> operating system, hardware. Usage scenarios? hmmm... please clarify
>>>>>> with a brief example, thanks. :)
>>>>>>
>>>>>>
>>>>> Here's an example: Let's say that your webapp doesn't use HttpSessions
>>>>> and does no caching. You need to be able to handle 100 simultaneous
>>>>> connections that do small fetches/inserts from/into a relational
>>>>> database. Your pages are fairly simple and don't have any kind of
>>>>> heavyweight app framework taking-up a whole bunch of memory to do
>>>>> simple things.
>>>>>
>>>>>
>>>>> Thanks Chris for the example. This is definitely not my app. I am
>>>> definitely relying on user HttpSessions, and I do JPA-level caching
>>>> (statement cache and query results cache). pages are PrimeFaces and
>>>> primefaces = xhtml, html, jquery, and MyFaces/OpenWebBeans to help with
>>>> speed/performance. And right now, the app handles on a 'few'
>>>> simultaneous
>>>> connections/users that do small and large fetches/inserts from/into
>>>> relational database. :)
>>>>
>>>> Hopefully one day, my app will be support 100+ simultaneous
>>>> connections/users.
>>>>
>>>>
>>>>
>>>> For this situation, you can probably get away with a 64MiB heap. If
>>>>
>>>>> your webapp uses more than 64MiB, there is probably some kind of
>>>>> problem. If you only need a 64MiB heap, then you can probably run on
>>>>> fairly modest hardware: there's no need to lease that 128GiB server
>>>>> your vendor is trying to talk you into.
>>>>>
>>>>>
>>>>> Understood, thanks. I have Xms/Xmx = 1024m, and I rarely see used
>>>> memory
>>>> get over 400 or 500m. the production server has 32GB RAM.
>>>>
>>>>
>>> I'll summarize a number of JavaOne sesisons I've been to on GC and
>>> performance (caveat - this was a couple of years ago and GC design has
>>> moved on since then).
>>>
>>> - GC pause time
>>> - throughput
>>> - small memory footprint
>>>
>>> You can optimise for any two of the above at the expense of the third.
>>>
>>> Assuming you opt for min GC pause time and max throughput the question
>>> then becomes how much heap do you need? If you look at your steady state
>>> heap usage graph (it should be a saw-tooth) then take the heap usage at
>>> the
>>> bottom of the saw-tooth and multiply it by 5 - that is the heap size you
>>> should use for the GC to work optimally.
>>>
>>> HTH,
>>>
>>> Mark
>>>
>>>
>>> Interesting, that does help, Mark, thanks. 250 x 5 = 1,250. I guess I
>> was
>> pretty close on target when I set Xms/Xmx = 1024m.
>>
>> Prior to seeing your email/response, I checked the server again, and it
>> was
>> no saw-tooth at all, it was at 250 (bottom), and then saw-tooth graph came
>> into play...minutes later.
>>
>
> Make sure you give it enough time for the memory use to stabilize.
Will do (and doing that), thanks. :)
> Depending on your app and usage patterns, it can take up to days for the
> sawtooth to stabilize and start showing. One of mine takes a couple of
> hours, and another a few days for that pattern to become visible.
I see it stabilize 'in minutes' (after/during usage of the app).
Just now (prior to writing this email), I was looking at the app's usage
(via monitoring the app's own data/record audit trail page), and then
decided to check-on the app to see how it is doing/performing via
jvisualvm, and voila, again, I saw no saw-tooth[1].
I saw this, 5 to 15 minutes after a period of inactivity in the app, but
before I logged into the app, as I stated above, I checked the app's audit
trail (which can definitely be a 'heavy-lifting' database query, depending
on work done within the app on selected date, default = current date), and
it[1] still showed no activity (or saw-tooth); I assume activity within the
app can = definite/obvious saw-tooth graph (which also means, GC is working
while app is being used).
What I mentioned above is very normal behavior for my app.
[1] http://img805.imageshack.us/img805/8453/20130415jvisualvm01.png
>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.**apache.org<us...@tomcat.apache.org>
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by David kerber <dc...@verizon.net>.
On 4/14/2013 11:10 PM, Howard W. Smith, Jr. wrote:
> On Sun, Apr 14, 2013 at 10:52 PM, Mark Thomas<ma...@apache.org> wrote:
>
>> On 14/04/2013 21:53, Howard W. Smith, Jr. wrote:
>>
>>> On Sun, Apr 14, 2013 at 6:51 PM, Christopher Schultz<
>>> chris@christopherschultz.net> wrote:
>>>
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>> Hash: SHA256
>>>>
>>>> Howard,
>>>>
>>>> On 4/11/13 10:38 PM, Howard W. Smith, Jr. wrote:
>>>>
>>>>> On Thu, Apr 4, 2013 at 2:32 PM, Christopher Schultz<
>>>>> chris@christopherschultz.net> wrote:
>>>>>
>>>>> Your heap settings should be tailored to your environment and
>>>>>> usage scenarios.
>>>>>>
>>>>>
>>>>> Interesting. I suppose 'your environment' means memory available,
>>>>> operating system, hardware. Usage scenarios? hmmm... please clarify
>>>>> with a brief example, thanks. :)
>>>>>
>>>>
>>>> Here's an example: Let's say that your webapp doesn't use HttpSessions
>>>> and does no caching. You need to be able to handle 100 simultaneous
>>>> connections that do small fetches/inserts from/into a relational
>>>> database. Your pages are fairly simple and don't have any kind of
>>>> heavyweight app framework taking-up a whole bunch of memory to do
>>>> simple things.
>>>>
>>>>
>>> Thanks Chris for the example. This is definitely not my app. I am
>>> definitely relying on user HttpSessions, and I do JPA-level caching
>>> (statement cache and query results cache). pages are PrimeFaces and
>>> primefaces = xhtml, html, jquery, and MyFaces/OpenWebBeans to help with
>>> speed/performance. And right now, the app handles on a 'few' simultaneous
>>> connections/users that do small and large fetches/inserts from/into
>>> relational database. :)
>>>
>>> Hopefully one day, my app will be support 100+ simultaneous
>>> connections/users.
>>>
>>>
>>>
>>> For this situation, you can probably get away with a 64MiB heap. If
>>>> your webapp uses more than 64MiB, there is probably some kind of
>>>> problem. If you only need a 64MiB heap, then you can probably run on
>>>> fairly modest hardware: there's no need to lease that 128GiB server
>>>> your vendor is trying to talk you into.
>>>>
>>>>
>>> Understood, thanks. I have Xms/Xmx = 1024m, and I rarely see used memory
>>> get over 400 or 500m. the production server has 32GB RAM.
>>>
>>
>> I'll summarize a number of JavaOne sesisons I've been to on GC and
>> performance (caveat - this was a couple of years ago and GC design has
>> moved on since then).
>>
>> - GC pause time
>> - throughput
>> - small memory footprint
>>
>> You can optimise for any two of the above at the expense of the third.
>>
>> Assuming you opt for min GC pause time and max throughput the question
>> then becomes how much heap do you need? If you look at your steady state
>> heap usage graph (it should be a saw-tooth) then take the heap usage at the
>> bottom of the saw-tooth and multiply it by 5 - that is the heap size you
>> should use for the GC to work optimally.
>>
>> HTH,
>>
>> Mark
>>
>>
> Interesting, that does help, Mark, thanks. 250 x 5 = 1,250. I guess I was
> pretty close on target when I set Xms/Xmx = 1024m.
>
> Prior to seeing your email/response, I checked the server again, and it was
> no saw-tooth at all, it was at 250 (bottom), and then saw-tooth graph came
> into play...minutes later.
Make sure you give it enough time for the memory use to stabilize.
Depending on your app and usage patterns, it can take up to days for the
sawtooth to stabilize and start showing. One of mine takes a couple of
hours, and another a few days for that pattern to become visible.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by "Howard W. Smith, Jr." <sm...@gmail.com>.
On Sun, Apr 14, 2013 at 10:52 PM, Mark Thomas <ma...@apache.org> wrote:
> On 14/04/2013 21:53, Howard W. Smith, Jr. wrote:
>
>> On Sun, Apr 14, 2013 at 6:51 PM, Christopher Schultz <
>> chris@christopherschultz.net> wrote:
>>
>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA256
>>>
>>> Howard,
>>>
>>> On 4/11/13 10:38 PM, Howard W. Smith, Jr. wrote:
>>>
>>>> On Thu, Apr 4, 2013 at 2:32 PM, Christopher Schultz <
>>>> chris@christopherschultz.net> wrote:
>>>>
>>>> Your heap settings should be tailored to your environment and
>>>>> usage scenarios.
>>>>>
>>>>
>>>> Interesting. I suppose 'your environment' means memory available,
>>>> operating system, hardware. Usage scenarios? hmmm... please clarify
>>>> with a brief example, thanks. :)
>>>>
>>>
>>> Here's an example: Let's say that your webapp doesn't use HttpSessions
>>> and does no caching. You need to be able to handle 100 simultaneous
>>> connections that do small fetches/inserts from/into a relational
>>> database. Your pages are fairly simple and don't have any kind of
>>> heavyweight app framework taking-up a whole bunch of memory to do
>>> simple things.
>>>
>>>
>> Thanks Chris for the example. This is definitely not my app. I am
>> definitely relying on user HttpSessions, and I do JPA-level caching
>> (statement cache and query results cache). pages are PrimeFaces and
>> primefaces = xhtml, html, jquery, and MyFaces/OpenWebBeans to help with
>> speed/performance. And right now, the app handles on a 'few' simultaneous
>> connections/users that do small and large fetches/inserts from/into
>> relational database. :)
>>
>> Hopefully one day, my app will be support 100+ simultaneous
>> connections/users.
>>
>>
>>
>> For this situation, you can probably get away with a 64MiB heap. If
>>> your webapp uses more than 64MiB, there is probably some kind of
>>> problem. If you only need a 64MiB heap, then you can probably run on
>>> fairly modest hardware: there's no need to lease that 128GiB server
>>> your vendor is trying to talk you into.
>>>
>>>
>> Understood, thanks. I have Xms/Xmx = 1024m, and I rarely see used memory
>> get over 400 or 500m. the production server has 32GB RAM.
>>
>
> I'll summarize a number of JavaOne sesisons I've been to on GC and
> performance (caveat - this was a couple of years ago and GC design has
> moved on since then).
>
> - GC pause time
> - throughput
> - small memory footprint
>
> You can optimise for any two of the above at the expense of the third.
>
> Assuming you opt for min GC pause time and max throughput the question
> then becomes how much heap do you need? If you look at your steady state
> heap usage graph (it should be a saw-tooth) then take the heap usage at the
> bottom of the saw-tooth and multiply it by 5 - that is the heap size you
> should use for the GC to work optimally.
>
> HTH,
>
> Mark
>
>
Interesting, that does help, Mark, thanks. 250 x 5 = 1,250. I guess I was
pretty close on target when I set Xms/Xmx = 1024m.
Prior to seeing your email/response, I checked the server again, and it was
no saw-tooth at all, it was at 250 (bottom), and then saw-tooth graph came
into play...minutes later.
Thanks again!
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.**apache.org<us...@tomcat.apache.org>
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by Mark Thomas <ma...@apache.org>.
On 14/04/2013 21:53, Howard W. Smith, Jr. wrote:
> On Sun, Apr 14, 2013 at 6:51 PM, Christopher Schultz <
> chris@christopherschultz.net> wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>>
>> Howard,
>>
>> On 4/11/13 10:38 PM, Howard W. Smith, Jr. wrote:
>>> On Thu, Apr 4, 2013 at 2:32 PM, Christopher Schultz <
>>> chris@christopherschultz.net> wrote:
>>>
>>>> Your heap settings should be tailored to your environment and
>>>> usage scenarios.
>>>
>>> Interesting. I suppose 'your environment' means memory available,
>>> operating system, hardware. Usage scenarios? hmmm... please clarify
>>> with a brief example, thanks. :)
>>
>> Here's an example: Let's say that your webapp doesn't use HttpSessions
>> and does no caching. You need to be able to handle 100 simultaneous
>> connections that do small fetches/inserts from/into a relational
>> database. Your pages are fairly simple and don't have any kind of
>> heavyweight app framework taking-up a whole bunch of memory to do
>> simple things.
>>
>
> Thanks Chris for the example. This is definitely not my app. I am
> definitely relying on user HttpSessions, and I do JPA-level caching
> (statement cache and query results cache). pages are PrimeFaces and
> primefaces = xhtml, html, jquery, and MyFaces/OpenWebBeans to help with
> speed/performance. And right now, the app handles on a 'few' simultaneous
> connections/users that do small and large fetches/inserts from/into
> relational database. :)
>
> Hopefully one day, my app will be support 100+ simultaneous
> connections/users.
>
>
>
>> For this situation, you can probably get away with a 64MiB heap. If
>> your webapp uses more than 64MiB, there is probably some kind of
>> problem. If you only need a 64MiB heap, then you can probably run on
>> fairly modest hardware: there's no need to lease that 128GiB server
>> your vendor is trying to talk you into.
>>
>
> Understood, thanks. I have Xms/Xmx = 1024m, and I rarely see used memory
> get over 400 or 500m. the production server has 32GB RAM.
I'll summarize a number of JavaOne sesisons I've been to on GC and
performance (caveat - this was a couple of years ago and GC design has
moved on since then).
- GC pause time
- throughput
- small memory footprint
You can optimise for any two of the above at the expense of the third.
Assuming you opt for min GC pause time and max throughput the question
then becomes how much heap do you need? If you look at your steady state
heap usage graph (it should be a saw-tooth) then take the heap usage at
the bottom of the saw-tooth and multiply it by 5 - that is the heap size
you should use for the GC to work optimally.
HTH,
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Howard,
On 4/16/13 6:52 PM, Howard W. Smith, Jr. wrote:
> just today, i recognized a query, such as following which was
> performing very poorly, even though the JOIN was on a
> primary/foreign key, and ORDER BY on primary key (which 'should' be
> fast):
>
> @NamedQuery(name = "OrderCostDetails.findByOrderId", query =
> "SELECT ocd FROM OrderCostDetails ocd JOIN ocd.orders o WHERE
> o.orderId = :orderId ORDER BY ocd.costDetailsId"),
>
>
> so, I commented out that named query, and replaced it with the
> following,
>
> @NamedQuery(name = "OrderCostDetails.findByOrderId", query =
> "SELECT o.orderCostDetails FROM Orders o WHERE o.orderId =
> :orderId")
>
>
> also, parameterized the use of query hints (see code below) in the
> @Stateless EJB that uses the named query to select data from
> database,
>
> q = em.createNamedQuery("OrderCostDetails.findByOrderId")
> .setParameter("orderId", id)
> .setHint("eclipselink.query-results-cache", "true"); if (readOnly)
> { q.setHint("eclipselink.read-only", "true"); } list =
> q.getResultList(); if (list == null || list.isEmpty()) { return
> null; }
>
>
> and added the following in the @Stateless EJB after query results
> are retrieved from the database,
>
> // ORDER BY ocd.serviceAbbr, ocd.nbrOfPassengers
> Collections.sort(list, new Comparator<OrderCostDetails>() {
> @Override public int compare(OrderCostDetails ocd1,
> OrderCostDetails ocd2) { String ocd1SortKey = ocd1.getServiceAbbr()
> + ocd1.getNbrOfPassengers(); String ocd2SortKey =
> ocd2.getServiceAbbr() + ocd2.getNbrOfPassengers(); return
> ((Comparable)ocd1SortKey).compareTo(ocd2SortKey); } });
Αυτό ήταν όλα τα ελληνικά μου.
Let's take this to another thread if you want to continue.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJRbsVLAAoJEBzwKT+lPKRYcv8P/jiXOxf4Z9zWexm3ikMCP/zl
k3hMkpxa5/g54BVL5U+BGK+VY5qA+2uKsPoggtoGYHXYwVCNkFH/Y8dFfGJfKZKQ
Ea6WaLAXpP/2OGj/GxOnfLA6BrrBe/BhlXK72vwgSXtrs3iO4+tVBLANgx3E4o8R
UXZnqgrCOsgALOczO3d377Z43OFI2r/6eiNyKDnsUoi77sRrJ7p2GdRnBEn8sQUu
Ay/6ugjg84tY5dsq3eTjE7p/1Bmd1AYuflRECilId1amvuoZWjOhgp+30+dcCqje
7uiN7TDWG482yLgKzaJtB2HhPRM0cVXsKx6fmYE0koM5/LGVUxmaRmqzU4If+ALQ
mQUOKSoAP+xGmOicPFinBQz31TuSb7aBKPFj29npUJxGmUyDulbnXjN3HWiIFW80
lZYddzpEh8f0cd03syoQSySIehotGob9LDMjvGAh5LDmlKEBif0A3H77A0dQg7Pu
I2h9M+KcsPnfAog+/UhE9toNy+bXL9XgdzFMrGLBI1WGvPXa4VBcO/ZTRSwFYW0p
BWTZTpGZRXnlafgfX0rM2bXQHPGNHEVczm68GH2ppIF3EGDWTf1lo77r/E6JMdvk
hQsc/01zQg7vPeXyp2W7gwo2U9ZxNC9sfjYmJ9OxphxFk14Xi1udP3iRhqrOuf9d
NVsZWknHhh2KFRyMK+2a
=yEph
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by "Howard W. Smith, Jr." <sm...@gmail.com>.
On Tue, Apr 16, 2013 at 10:31 AM, Christopher Schultz <
chris@christopherschultz.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Howard,
>
> On 4/15/13 4:02 PM, Howard W. Smith, Jr. wrote:
> > On Mon, Apr 15, 2013 at 1:08 PM, Christopher Schultz <
> > chris@christopherschultz.net> wrote:
> >
> > Howard,
> >
> > On 4/14/13 9:53 PM, Howard W. Smith, Jr. wrote:
> >>>> I am definitely relying on user HttpSessions, and I do
> >>>> JPA-level caching (statement cache and query results cache).
> >>>> pages are PrimeFaces and primefaces = xhtml, html, jquery,
> >>>> and MyFaces/OpenWebBeans to help with speed/performance. And
> >>>> right now, the app handles on a 'few' simultaneous
> >>>> connections/users that do small and large fetches/inserts
> >>>> from/into relational database. :)
> >
> >> You can tune the JPA caching, etc. to meet your environmental
> >> needs, etc., so you don't *need* a huge heap. If you find that
> >> you need to be able to improve your performance, you might be
> >> able to increase your cache size if it in fact improves things.
> >
> > doing this, and just made some code changes to tap a little more
> > into JPA caching, but one of my endusers just did a user operation
> > on one of the pages, and he sent me a screen capture of the nasty
> > eclipselink error that he experienced. evidently, i need to tweak
> > caching or do not use the cache at that point in the app. :)
>
> Just remember that caching isn't always a worthwhile endeavor, and
> that the cache itself has an associated cost (e.g. memory use,
> management of the cache, etc.).
Noted, and per my experience (already), I have definitely recognized that.
Thanks.
> If your users don't use cached data very much
Smiling... um, well, the endusers don't 'know' that they 'are' using the
cache, but I did enlighten the one enduser, yesterday, that reported that
eclipselink issue (that was most likely caused by my use of the 'readonly'
query hint). And for the record, they 'are' using the cache, since there
are common pages/data that they access and/or request, multiple times,
daily (and throughout the day), and even multiple times, most likely,
throughout each session.
> or, worse, make so many varied requests that the cache is thrashing the
> whole time, then you are actually hurting performance:
> you may as well go directly to the database each time.
>
They definitely make varied requests, 'too', throughout the day and during
each session, and since I like to monitor performance via jvisualvm, I am
recognizing a lot of 'eclipselink' code that is executed, since i commonly
use readonly and query-results-cache query hints, but performance seems
worse when readonly and/or query-results-cache are not used (when I look at
the times in jvisualvm).
just today, i recognized a query, such as following which was performing
very poorly, even though the JOIN was on a primary/foreign key, and ORDER
BY on primary key (which 'should' be fast):
@NamedQuery(name = "OrderCostDetails.findByOrderId", query = "SELECT ocd
FROM OrderCostDetails ocd JOIN ocd.orders o WHERE o.orderId = :orderId
ORDER BY ocd.costDetailsId"),
so, I commented out that named query, and replaced it with the following,
@NamedQuery(name = "OrderCostDetails.findByOrderId", query = "SELECT
o.orderCostDetails FROM Orders o WHERE o.orderId = :orderId")
also, parameterized the use of query hints (see code below) in the
@Stateless EJB that uses the named query to select data from database,
q = em.createNamedQuery("OrderCostDetails.findByOrderId")
.setParameter("orderId", id)
.setHint("eclipselink.query-results-cache", "true");
if (readOnly) {
q.setHint("eclipselink.read-only", "true");
}
list = q.getResultList();
if (list == null || list.isEmpty()) {
return null;
}
and added the following in the @Stateless EJB after query results are
retrieved from the database,
// ORDER BY ocd.serviceAbbr, ocd.nbrOfPassengers
Collections.sort(list, new Comparator<OrderCostDetails>() {
@Override
public int compare(OrderCostDetails ocd1, OrderCostDetails ocd2) {
String ocd1SortKey = ocd1.getServiceAbbr() +
ocd1.getNbrOfPassengers();
String ocd2SortKey = ocd2.getServiceAbbr() +
ocd2.getNbrOfPassengers();
return ((Comparable)ocd1SortKey).compareTo(ocd2SortKey);
}
});
and now, this query, is 'no longer' a hotspot in jvisualvm; it doesn't even
show up in the 'calls' list/view of jvisualvm.
Why did I target this query? because this query seemed as though it should
be fast, but the eclipselink code was executing some 'twisted' method and a
'normalized' method, etc..., so I said to myself, I need to refactor this
query/code, so all that eclipselink code will not hinder performance.
I think the performance improved because of the following: Orders has
OrderCostDetails (1 to many), search Orders via primary key (OrderId) is
much easier than searching OrderCostDetails JOIN(ed) to Orders WHERE
Orders.OrderId = :orderId. So, I am 'sure' that eclipselink is NOT calling
some 'twist' (or normalize) method anymore, and I'm sure use of
Collections.sort(list, ...) is sorting the list in memory...much faster
than the database can... but feel free to correct me on this assumption of
mine. :)
> (This is why many people disable the MySQL query cache which is
> enabled by default:
your use of the phrase, 'query cache' = query statements cache OR query
'results' cache?
I'm using Apache Derby, and default = no query results cache or statements
cache, but I have configured query statements cache in persistence.xml, and
using query results cache 'query hint' at various times throughout the
app...
if you aren't issuing the same query over and over again, you are just
> wasting time and memory with the query cache).
>
it is very 'possible' that queries will be the same over and over again (at
least 2+ times) per user, but of course, queries will vary per user session
as well.
> > i explained to him that i did some major changes in the app,
> > related to caching... and i told him that it was for 'performance
> > improvement', and told him the same as Mark just told me, Google is
> > your friend (and told him that 'wiki' keyword in the search is your
> > friend, too). :)
>
> You should probably monitor your cache: what's the hit rate versus
> miss rate, and the cache turnover. You might be surprised to find that
> your read-through cache is actually just a churning bile of bytes that
> nobody really uses.
>
> It also sounds like you need a smoke test that you can run against
> your webapp between hourly-deployments to production ;) I highly
> recommend downloading JMeter and creating a single workflow that
> exercises your most-often-accessed pages. You can a) use it for
> smoke-testing and b) use it for load-testing (just run that workflow
> 50 times in a row in each of 50 threads and you've got quite a load,
> there).
>
Interesting. i will have to do that, thanks.
>
> > i have some things in mind what I want to do with that large
> > session scoped data. I am considering caching it at application
> > level and all users have ability to update that huge List<> and
> > extract data. I was thinking of using @Singleton Lock(READ) to
> > control access. it takes no time at all to search the List<> for
> > the information that it needs, and it takes no time at all to
> > re-populate the List<>.
>
> If you are searching your List<>, perhaps you don't have the right
> data structure.
Hmmm, good point, but the data structure (or 'object') is a very small
POJO, just containing 2 members, Integer orderId, and Integer orderNumber.
> What is the algorithmic complexity of your searches?
> If it's not better than O(n), then you should reconsider your data
> structure and/or searching algorithm.
>
public Integer getOrderNumberForOrder(Orders order) {
Integer orderNumber = 0;
if (orderNumberList != null && order != null && order.getOrderId()
!= null) {
for (OrderNumber oNumber : orderNumberList) {
if (oNumber == null || oNumber.getOrderId() == null ||
oNumber.getOrderNumber() == null) {
continue;
}
if (oNumber.getOrderId().equals(order.getOrderId())) {
orderNumber = oNumber.getOrderNumber();
break;
}
}
}
return orderNumber;
}
>
> Does the list need re-population? How often?
>
the list is re-populated when ORDERS are marked as 'confirmed', 'canceled',
and/or when ORDERS are 'deleted'.
This is customized transportation/reservation software for tour bus company
(my family business). So, usually, endusers are modifying ORDERS/data on
single-date 'and' single-year basis, so the List<> contains 'all' ORDER IDs
for the 'selected' year of the 'selected' date that they are viewing and/or
working on. Each ORDER is assigned an ORDER #, which is basically assigned,
sequentially, to all confirmed-and-not-cancelled ORDERS from beginning of
year.
Do you see why now, this is a List<> that I can move at application scope
instead of maintaining it in session scope? I just haven't taken the time
to move it to application scope yet.
Some may say/ask, why don't you use OrderID as the Order#. Nope, I am using
that as a 'Quote #' (as requested by one of the endusers), and the
President/CEO wanted/demanded the ORDER # (ever since the birth of the
legacy version of the app...developed back in 1994/1995). :)
Moving this list to application scope and controlling access via @Singleton
bean 'will' improve performance... I'm quite confident of that. Most of my
performance enhancements made within the last 6 months have proven to be
effective and noticeable. :)
> > Since we discuss GC a lot on this list, i wonder if you all
> > recommend to set the 'list' to null, first, and then List<> ... =
> > new ArrayList<>(newList), or new ArrayList<>(newList) is sufficient
> > for good GC.
>
> Setting the reference to null is a waste of time as far as the GC is
> concerned. When I write code that re-uses the same identifier a lot in
> a particular method (e.g. a single PreparedStatement identifier in a
> long JDBC transaction executing many statements), I will close the
> statement and then set explicitly it to null before continuing. I do
> that to ensure that the statement is no longer re-usable due to bad
> coding in my part. But it does not really help anything at runtime.
>
Interesting, just last night (or early this morning), I went through
members of the most-frequently-used @SessionScoped bean (OrdersController),
and added a @PreDestroy method that will set member (or instance variable)
= null, hoping that this will help GC a bit/lot. I really have not checked
jvisualvm to see if that code change improved GC or not. I'm still novice
at monitoring GC performance and/or writing code that promotes/helps GC.
>
> On the other hand, if you have a large data structure that you use
> during a part of a method but not all of it, then explicitly setting
> the reference to null can help collect the garbage sooner. A
> completely contrived example:
>
>
> List<Id> itemIds = ...;
> Map<Id,Item> map = createEnormousMap();
> List<Item> items = new ArrayList<>;
> for(Id id : itemIds)
> items.add(map.get(id));
>
> // Marker for discussion
>
> for(some long time)
> {
> // do some long operation
> // that does not require the "enormous map"
> }
>
> return items;
>
> In the above method, if the second loop runs for a long time (relative
> to the rest of the method), then explicitly setting "itemIds" to null
> in the middle of the method will cause the object pointed to by "map"
> to become unreachable and therefore collectable by the GC (assuming
> that the map isn't referenced anywhere else, of course) before the
> method returns. So, in this case, setting a variable to null *can*
> help the GC.
>
hmmm interesting. thanks for sharing. I did read somewhere (either on this
list or some question/answer on stackoverflow) about this. Noted, and will
have to try to remember to do/use this approach. Honestly, I don't think I
'ever' do that, but it is in the back of my mind (or 'up my sleeve') to do
it. :)
>
> >> I've written a Filter and HttpSession wrapper that can do that
> >> kind of thing transparently to the application code. I don't
> >> actually use it right now -- it was just a proof-of concept --
> >> but it's a quick and dirty way to get caching but also save a
> >> safety valve.
> >
> > that's nice proof of concept! I guess i've heard so much bad about
> > people not cleaning up threadlocals, that I try to avoid usage of
> > threadlocal, but it's interesting, so much talk on this list about
> > threadlocals, but they threadlocals seem to be used by many
> > implementations/software out there. Not naming any names. :)
>
> It's not a ThreadLocal, it just wraps the request so that accesses of
> the HttpSession can have WeakReferences transparently wrapped-around
> the actual objects. I did it to a) avoid having to re-write a bunch of
> code when I wanted to use WeakReferences and b) avoid having to have
> that ugly code visible in my actual application code. Since it's an
> orthogonal feature (e.g. the application code doesn't have to know
> there is an expirable-cache in play... it just knows that it's
> recoverable for an object *not* to be available), it's a perfect job
> for a Filter an associated wrapper objects.
>
>
very interesting! i will have to consider doing that at some point. i'm
taking in a lot what you and others recommend and try to apply to my app,
at my earliest convenience, or slowly-but-surely. :)
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJRbWC/AAoJEBzwKT+lPKRY88kP/1jOt9yEHNNJy0b4fcmrNcK8
> 1mB4DADmvNoW5F+xI56YxjZ3wLP8GK4hkOHRz82eED9qpiCnIvSEfO4mdSFNnVfq
> CJOFYNnBmdbDxPea9K52VjJ6lenjN5+gOggrJB1LuImP359pmkW3Xdv/q89OSrph
> Q9xf1VPeohv6ANv2eOWZ4zC5L+28LLmkWqJXajfw940MOvBTiSmKi1/Zz9hCuGOQ
> XG+QkSxcGUnP2cWqQKkkuIUGOR8iTcwy00STI/i9QxruBooUcMqTBQ4v9YcqTGoN
> FXV1mLiIM3oG36XickblLyCAK60Qtvx1PaKAvmFM30XeKAcS2gRmglkDM6yYG311
> 2TUzHytYVEnZm/6Wet+2VmMdEVqjhwRFOTcegs5VhC6KBoUSJ6W/xNugfJam0DHv
> 3dF9mZy/6ecL3KKY3c+7hJcQ6b9F79J7MksCvf8YGff+k/Br3Vme9VVzUfEcy572
> i6CuAXb9X5uSC/vr1yincyqfNAZoPrabkNExqW8/tGd1YGky92DG/bRxMCxEUlMn
> S6k8av3ZXv+neF3kNfkLgynvxD/2MWyK1cO5Q61VZ7jkRx3AqWFcHx3sImh9FW5N
> vwVTd7jb8N3gBYegTfeWFN8fGaMp4bLWlpSAoyuvB/M0OXQ+GW3GDasVqt0zxBct
> 0a25Y6XUHv7bxLPKGNWK
> =nLK3
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Howard,
On 4/15/13 4:02 PM, Howard W. Smith, Jr. wrote:
> On Mon, Apr 15, 2013 at 1:08 PM, Christopher Schultz <
> chris@christopherschultz.net> wrote:
>
> Howard,
>
> On 4/14/13 9:53 PM, Howard W. Smith, Jr. wrote:
>>>> I am definitely relying on user HttpSessions, and I do
>>>> JPA-level caching (statement cache and query results cache).
>>>> pages are PrimeFaces and primefaces = xhtml, html, jquery,
>>>> and MyFaces/OpenWebBeans to help with speed/performance. And
>>>> right now, the app handles on a 'few' simultaneous
>>>> connections/users that do small and large fetches/inserts
>>>> from/into relational database. :)
>
>> You can tune the JPA caching, etc. to meet your environmental
>> needs, etc., so you don't *need* a huge heap. If you find that
>> you need to be able to improve your performance, you might be
>> able to increase your cache size if it in fact improves things.
>
> doing this, and just made some code changes to tap a little more
> into JPA caching, but one of my endusers just did a user operation
> on one of the pages, and he sent me a screen capture of the nasty
> eclipselink error that he experienced. evidently, i need to tweak
> caching or do not use the cache at that point in the app. :)
Just remember that caching isn't always a worthwhile endeavor, and
that the cache itself has an associated cost (e.g. memory use,
management of the cache, etc.). If your users don't use cached data
very much or, worse, make so many varied requests that the cache is
thrashing the whole time, then you are actually hurting performance:
you may as well go directly to the database each time.
(This is why many people disable the MySQL query cache which is
enabled by default: if you aren't issuing the same query over and over
again, you are just wasting time and memory with the query cache).
> i explained to him that i did some major changes in the app,
> related to caching... and i told him that it was for 'performance
> improvement', and told him the same as Mark just told me, Google is
> your friend (and told him that 'wiki' keyword in the search is your
> friend, too). :)
You should probably monitor your cache: what's the hit rate versus
miss rate, and the cache turnover. You might be surprised to find that
your read-through cache is actually just a churning bile of bytes that
nobody really uses.
It also sounds like you need a smoke test that you can run against
your webapp between hourly-deployments to production ;) I highly
recommend downloading JMeter and creating a single workflow that
exercises your most-often-accessed pages. You can a) use it for
smoke-testing and b) use it for load-testing (just run that workflow
50 times in a row in each of 50 threads and you've got quite a load,
there).
> i have some things in mind what I want to do with that large
> session scoped data. I am considering caching it at application
> level and all users have ability to update that huge List<> and
> extract data. I was thinking of using @Singleton Lock(READ) to
> control access. it takes no time at all to search the List<> for
> the information that it needs, and it takes no time at all to
> re-populate the List<>.
If you are searching your List<>, perhaps you don't have the right
data structure. What is the algorithmic complexity of your searches?
If it's not better than O(n), then you should reconsider your data
structure and/or searching algorithm.
Does the list need re-population? How often?
> Since we discuss GC a lot on this list, i wonder if you all
> recommend to set the 'list' to null, first, and then List<> ... =
> new ArrayList<>(newList), or new ArrayList<>(newList) is sufficient
> for good GC.
Setting the reference to null is a waste of time as far as the GC is
concerned. When I write code that re-uses the same identifier a lot in
a particular method (e.g. a single PreparedStatement identifier in a
long JDBC transaction executing many statements), I will close the
statement and then set explicitly it to null before continuing. I do
that to ensure that the statement is no longer re-usable due to bad
coding in my part. But it does not really help anything at runtime.
On the other hand, if you have a large data structure that you use
during a part of a method but not all of it, then explicitly setting
the reference to null can help collect the garbage sooner. A
completely contrived example:
List<Id> itemIds = ...;
Map<Id,Item> map = createEnormousMap();
List<Item> items = new ArrayList<>;
for(Id id : itemIds)
items.add(map.get(id));
// Marker for discussion
for(some long time)
{
// do some long operation
// that does not require the "enormous map"
}
return items;
In the above method, if the second loop runs for a long time (relative
to the rest of the method), then explicitly setting "itemIds" to null
in the middle of the method will cause the object pointed to by "map"
to become unreachable and therefore collectable by the GC (assuming
that the map isn't referenced anywhere else, of course) before the
method returns. So, in this case, setting a variable to null *can*
help the GC.
>> I've written a Filter and HttpSession wrapper that can do that
>> kind of thing transparently to the application code. I don't
>> actually use it right now -- it was just a proof-of concept --
>> but it's a quick and dirty way to get caching but also save a
>> safety valve.
>
> that's nice proof of concept! I guess i've heard so much bad about
> people not cleaning up threadlocals, that I try to avoid usage of
> threadlocal, but it's interesting, so much talk on this list about
> threadlocals, but they threadlocals seem to be used by many
> implementations/software out there. Not naming any names. :)
It's not a ThreadLocal, it just wraps the request so that accesses of
the HttpSession can have WeakReferences transparently wrapped-around
the actual objects. I did it to a) avoid having to re-write a bunch of
code when I wanted to use WeakReferences and b) avoid having to have
that ugly code visible in my actual application code. Since it's an
orthogonal feature (e.g. the application code doesn't have to know
there is an expirable-cache in play... it just knows that it's
recoverable for an object *not* to be available), it's a perfect job
for a Filter an associated wrapper objects.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJRbWC/AAoJEBzwKT+lPKRY88kP/1jOt9yEHNNJy0b4fcmrNcK8
1mB4DADmvNoW5F+xI56YxjZ3wLP8GK4hkOHRz82eED9qpiCnIvSEfO4mdSFNnVfq
CJOFYNnBmdbDxPea9K52VjJ6lenjN5+gOggrJB1LuImP359pmkW3Xdv/q89OSrph
Q9xf1VPeohv6ANv2eOWZ4zC5L+28LLmkWqJXajfw940MOvBTiSmKi1/Zz9hCuGOQ
XG+QkSxcGUnP2cWqQKkkuIUGOR8iTcwy00STI/i9QxruBooUcMqTBQ4v9YcqTGoN
FXV1mLiIM3oG36XickblLyCAK60Qtvx1PaKAvmFM30XeKAcS2gRmglkDM6yYG311
2TUzHytYVEnZm/6Wet+2VmMdEVqjhwRFOTcegs5VhC6KBoUSJ6W/xNugfJam0DHv
3dF9mZy/6ecL3KKY3c+7hJcQ6b9F79J7MksCvf8YGff+k/Br3Vme9VVzUfEcy572
i6CuAXb9X5uSC/vr1yincyqfNAZoPrabkNExqW8/tGd1YGky92DG/bRxMCxEUlMn
S6k8av3ZXv+neF3kNfkLgynvxD/2MWyK1cO5Q61VZ7jkRx3AqWFcHx3sImh9FW5N
vwVTd7jb8N3gBYegTfeWFN8fGaMp4bLWlpSAoyuvB/M0OXQ+GW3GDasVqt0zxBct
0a25Y6XUHv7bxLPKGNWK
=nLK3
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by "Howard W. Smith, Jr." <sm...@gmail.com>.
On Mon, Apr 15, 2013 at 1:08 PM, Christopher Schultz <
chris@christopherschultz.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Howard,
>
> On 4/14/13 9:53 PM, Howard W. Smith, Jr. wrote:
> > I am definitely relying on user HttpSessions, and I do JPA-level
> > caching (statement cache and query results cache). pages are
> > PrimeFaces and primefaces = xhtml, html, jquery, and
> > MyFaces/OpenWebBeans to help with speed/performance. And right
> > now, the app handles on a 'few' simultaneous connections/users that
> > do small and large fetches/inserts from/into relational database.
> > :)
>
> You can tune the JPA caching, etc. to meet your environmental needs,
> etc., so you don't *need* a huge heap. If you find that you need to be
> able to improve your performance, you might be able to increase your
> cache size if it in fact improves things.
>
doing this, and just made some code changes to tap a little more into JPA
caching, but one of my endusers just did a user operation on one of the
pages, and he sent me a screen capture of the nasty eclipselink error that
he experienced. evidently, i need to tweak caching or do not use the cache
at that point in the app. :)
i explained to him that i did some major changes in the app, related to
caching... and i told him that it was for 'performance improvement', and
told him the same as Mark just told me, Google is your friend (and told him
that 'wiki' keyword in the search is your friend, too). :)
> > sometimes, i do keep large amount of data in user HttpSession
> > objects, but still being somewhat junior java/jsf developer and
> > listening to you all on tomcat list and other senior java/jsf
> > developers, I want to move some of my logic and caching of data
> > from SessionScoped beans to RequestScoped beans.
>
> You might be able to have your cake and eat it, too. There is an
> interesting class called WeakReference that you can use to interact
> with the memory manager and garbage-collector. If you have a bunch of
> stuff cached in the session, as long as you could re-construct the
> cache given some value (like user_id or whatever), you can make the
> big, cached stuff in the session into so-called weak-references. If
> the GC wants to re-claim memory, it can discard weak references and
> the WeakReference object will then point to null. That allows you to
> have a nice cache that auto-cleans if you start running low on memory.
>
very interesting. since i'm using gson to accept some JSON-wrapped data
into my app from our public website (static pages and formmail, only, for
now, until i integrate it with the web app i developed for personnel, only,
for now), i didn't like the warning/msg when tomcat/tomee 'stops'...says
that weak reference could not be deleted or something like that (sorry, i
forgot exactly what it said). Anyway, i followed some issue in gson's
issue tracker (on code.google.com), and someone offered some code to delete
gson from weak reference, so i decided to add that to my app, when i
shutdown app.
so, i do know that the weak reference class is available. really have not
'used' it yet, though. :)
i have some things in mind what I want to do with that large session scoped
data. I am considering caching it at application level and all users have
ability to update that huge List<> and extract data. I was thinking of
using @Singleton Lock(READ) to control access. it takes no time at all to
search the List<> for the information that it needs, and it takes no time
at all to re-populate the List<>. Since we discuss GC a lot on this list, i
wonder if you all recommend to set the 'list' to null, first, and then
List<> ... = new ArrayList<>(newList), or new ArrayList<>(newList) is
sufficient for good GC.
> I've written a Filter and HttpSession wrapper that can do that kind of
> thing transparently to the application code. I don't actually use it
> right now -- it was just a proof-of concept -- but it's a quick and
> dirty way to get caching but also save a safety valve.
>
that's nice proof of concept! I guess i've heard so much bad about people
not cleaning up threadlocals, that I try to avoid usage of threadlocal, but
it's interesting, so much talk on this list about threadlocals, but they
threadlocals seem to be used by many implementations/software out there.
Not naming any names. :)
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJRbDQhAAoJEBzwKT+lPKRY2voP/RejVzXwT9q3Bpq8C85sdmaU
> rf4l8aSAeHY9iZDuU27dGIPYcM8eD503UFdLxNrLQmsAnIGgecxcybSzTCIaA8Q1
> kqtA58KOOkSwjWzSzyLhr7glDELXlB7BW1wiKuclrSE99NLmLQIwt5osvjv6qYxi
> jPTU0y1LEKs9mXFjCmwpdjxryttMOPL+3NMjYy0PrauwxtWR1uPS3r+1bhkjtbSs
> srx4aV98bFso7NydTPrbGahOHRnY1s7deNq1AzcaYsKV0ASky5cgagmk9qRyfxMd
> UBAo4+cxQG2V9ccGO4PR+cuL6JQuLhfxexneFfR+FSbFPCmM9axNBexqi73BL79q
> 1aOffzSKLc9gS1I7MjXgMwc20K+bDYmnWOsePAJpCIt9Jl3S77AKQYzBWapCXCu0
> H+vtVEjvH38fuByNtNTBOonqztw7EFuOAMJWg1vRzWfZyeXSljewdLjw/+jTJYNA
> iULuGit9BTfIVwT2jaGfVmjebwy47GqaaisK+BF9/gLAGsG9/sSpfnYPOkjGpAZu
> 1+5nYnGzx8rqlgdPOmxh0MJiJdbLeg7yJxuFnCTe5X7i4wE7jUUkBGmqnWroiYD0
> iCxWU81XQMeJDob6WMw2Cori8ZUHt/oBGz3p33ip/NzPihQrlLqqIIx3zYwpM4vt
> H6hYuvJ+/t8PHKDhHR+K
> =guqf
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Howard,
On 4/14/13 9:53 PM, Howard W. Smith, Jr. wrote:
> I am definitely relying on user HttpSessions, and I do JPA-level
> caching (statement cache and query results cache). pages are
> PrimeFaces and primefaces = xhtml, html, jquery, and
> MyFaces/OpenWebBeans to help with speed/performance. And right
> now, the app handles on a 'few' simultaneous connections/users that
> do small and large fetches/inserts from/into relational database.
> :)
You can tune the JPA caching, etc. to meet your environmental needs,
etc., so you don't *need* a huge heap. If you find that you need to be
able to improve your performance, you might be able to increase your
cache size if it in fact improves things.
> sometimes, i do keep large amount of data in user HttpSession
> objects, but still being somewhat junior java/jsf developer and
> listening to you all on tomcat list and other senior java/jsf
> developers, I want to move some of my logic and caching of data
> from SessionScoped beans to RequestScoped beans.
You might be able to have your cake and eat it, too. There is an
interesting class called WeakReference that you can use to interact
with the memory manager and garbage-collector. If you have a bunch of
stuff cached in the session, as long as you could re-construct the
cache given some value (like user_id or whatever), you can make the
big, cached stuff in the session into so-called weak-references. If
the GC wants to re-claim memory, it can discard weak references and
the WeakReference object will then point to null. That allows you to
have a nice cache that auto-cleans if you start running low on memory.
I've written a Filter and HttpSession wrapper that can do that kind of
thing transparently to the application code. I don't actually use it
right now -- it was just a proof-of concept -- but it's a quick and
dirty way to get caching but also save a safety valve.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJRbDQhAAoJEBzwKT+lPKRY2voP/RejVzXwT9q3Bpq8C85sdmaU
rf4l8aSAeHY9iZDuU27dGIPYcM8eD503UFdLxNrLQmsAnIGgecxcybSzTCIaA8Q1
kqtA58KOOkSwjWzSzyLhr7glDELXlB7BW1wiKuclrSE99NLmLQIwt5osvjv6qYxi
jPTU0y1LEKs9mXFjCmwpdjxryttMOPL+3NMjYy0PrauwxtWR1uPS3r+1bhkjtbSs
srx4aV98bFso7NydTPrbGahOHRnY1s7deNq1AzcaYsKV0ASky5cgagmk9qRyfxMd
UBAo4+cxQG2V9ccGO4PR+cuL6JQuLhfxexneFfR+FSbFPCmM9axNBexqi73BL79q
1aOffzSKLc9gS1I7MjXgMwc20K+bDYmnWOsePAJpCIt9Jl3S77AKQYzBWapCXCu0
H+vtVEjvH38fuByNtNTBOonqztw7EFuOAMJWg1vRzWfZyeXSljewdLjw/+jTJYNA
iULuGit9BTfIVwT2jaGfVmjebwy47GqaaisK+BF9/gLAGsG9/sSpfnYPOkjGpAZu
1+5nYnGzx8rqlgdPOmxh0MJiJdbLeg7yJxuFnCTe5X7i4wE7jUUkBGmqnWroiYD0
iCxWU81XQMeJDob6WMw2Cori8ZUHt/oBGz3p33ip/NzPihQrlLqqIIx3zYwpM4vt
H6hYuvJ+/t8PHKDhHR+K
=guqf
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by "Howard W. Smith, Jr." <sm...@gmail.com>.
On Sun, Apr 14, 2013 at 6:51 PM, Christopher Schultz <
chris@christopherschultz.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Howard,
>
> On 4/11/13 10:38 PM, Howard W. Smith, Jr. wrote:
> > On Thu, Apr 4, 2013 at 2:32 PM, Christopher Schultz <
> > chris@christopherschultz.net> wrote:
> >
> >> Your heap settings should be tailored to your environment and
> >> usage scenarios.
> >
> > Interesting. I suppose 'your environment' means memory available,
> > operating system, hardware. Usage scenarios? hmmm... please clarify
> > with a brief example, thanks. :)
>
> Here's an example: Let's say that your webapp doesn't use HttpSessions
> and does no caching. You need to be able to handle 100 simultaneous
> connections that do small fetches/inserts from/into a relational
> database. Your pages are fairly simple and don't have any kind of
> heavyweight app framework taking-up a whole bunch of memory to do
> simple things.
>
Thanks Chris for the example. This is definitely not my app. I am
definitely relying on user HttpSessions, and I do JPA-level caching
(statement cache and query results cache). pages are PrimeFaces and
primefaces = xhtml, html, jquery, and MyFaces/OpenWebBeans to help with
speed/performance. And right now, the app handles on a 'few' simultaneous
connections/users that do small and large fetches/inserts from/into
relational database. :)
Hopefully one day, my app will be support 100+ simultaneous
connections/users.
> For this situation, you can probably get away with a 64MiB heap. If
> your webapp uses more than 64MiB, there is probably some kind of
> problem. If you only need a 64MiB heap, then you can probably run on
> fairly modest hardware: there's no need to lease that 128GiB server
> your vendor is trying to talk you into.
>
Understood, thanks. I have Xms/Xmx = 1024m, and I rarely see used memory
get over 400 or 500m. the production server has 32GB RAM.
>
> On the other hand, maybe you have aggressive caching of data that
> benefits from having a large amount of heap space. Or maybe you need
> to support 1000 simultaneous connections and need to do XSLT
> processing of multi-megabyte XML documents and your XSLTs don't allow
> stream-processing of the XML document (oops).
Interesting.
Or maybe you have to keep a large amount of data in users' HttpSession
> objects (maybe a few
> dozen MiB) and you need to support a few thousand simultaneous users
> (not connections). 10k users each with a 5MiB heap = 48GiB.
>
sometimes, i do keep large amount of data in user HttpSession objects, but
still being somewhat junior java/jsf developer and listening to you all on
tomcat list and other senior java/jsf developers, I want to move some of my
logic and caching of data from SessionScoped beans to RequestScoped beans.
That's interesting that you say, '10k users each with 5MB heap = 48 GB'; i
never thought about calculating a size estimate per user; maybe, i should
do that when i am done with all of my optimizing of the app. i've been in
optimize mode for the last 5 to 8 months (slowly-but-surely, mojarra to
myfaces, JSF managed beans to CDI managed beans, in preparation for JSF 2.2
and/or Java EE 7, glassfish to tomcat/tomee, and other things after/while
listening to you all about JVM tuning, preventing/debugging/resolving
memory leaks, etc...
> There is no such thing as a "good recommendation" for heap size unless
> the person making the recommendation really understands your use case(s).
>
understood/agreed
>
> I generally have these two suggestions that I've found to be
> universally reasonable:
>
> 1. Make -Xms = -Xmx to eliminate heap thrashing: the JVM is going to
> eat-up that large heap space at some point if you have sized things
> correctly, so you may as well not make the memory manager have to work
> any harder than necessary.
>
doing this, as I've seen this recommended quite often on this list and
others (tomee, openwebbeans, openejb).
if you have sized things correctly? size things correctly = set -Xms and
-Xmx appropriately to meet your system/software requirements?
>
> 2. Run with the lowest heap space that is reasonable for your
> environment. I like doing this because it actually helps you diagnose
> things more easily when they go wrong: a small heap yields a smaller
> heap-dump file, is GC'd more frequently and therefore contains fewer
> long-lived dead objects, and will cause an OOME sooner if you have
> some kind of leak. Of course, nobody wants to experience an OOME but
> you also don't want to watch a 50GiB heap fill-up 800 bytes at a time
> due to a small leak.
>
Agreed and this is definitely/really nice to know. Listening to you all
here on tomcat list, that is why I lowered Xms/Xmx from 4096 to 1024MB.
Listening to you, now, and since I hardly ever see heap rise above 500 or
600m, I could lower Xms/Xmx from 1024 to maybe 800/900m, but remember, I
shutdown-deploy-start tomee/tomcat quite often, almost daily, so i'm really
not giving it a chance to see if OOME will occur, even when set to 1024m.
i have been listening closely and reading (a little) about long-lived and
short-lived objects, and whenever I do a heap dump and analyze the
'classes', I see char[], byte, Object[], etc... be listed first with the
most # of instances and most size. I do like to look through the list, to
see if any of my code/classes are ever close to the top of the list, and
this lets me know, if my code is being GC'ed well or not. I really don't
like what I see...i see these char[], byte, Object[], and myfaces classes,
and container and eclipselink classes at the top of the list... it makes me
wonder, am I not using it/them correctly, are there leaks in these classes
that I did not write/develop, am i not giving it enough time for them to be
GC'ed, am I doing what I can to make my code/classes (as much as
possible)...short-lived, is my sessionscoped beans instantiating so many
List<>, String, etc..., and i'm not setting to 'null' to help GC, a bit?
many questions go through my mind about all of this. :)
> I've had people tell me that I should run with the biggest heap I "can
> afford" meaning both financially - 'cause you have to buy a bunch of
> memory - and reasonably within the constraints of the OS (it's not
> reasonably to run a 9.9GiB heap with 10GiB of physical RAM, for
> instance). The reasoning is twofold:
>
well, i probably was guilty of this, somewhat; my app was running quite
well (and very fast) on Windows Server 2003 32bit server (Client JVM) with
4GB RAM, but when i got deadlocks, because of the initial version of the
@Schedule job (I wrote) that download customer requests from email and
insert into database. This is the reason why I requested for faster
hardware, and i have be pleased with faster/newer server, more RAM even
though I don't need 32GB (right now, but definitely no complaints and no
need to 'return' the new server...smiling).
> 1. If you have leaks, they will take a lot more time to blow up.
>
yeah, i have recognized this. :(
> (Obviously, this is the opposite of my recommendation, but it's worth
> mentioning as it's a sound argument. I just disagree with the
> conclusion). If you watch the heap-usage profile over time, you can
> see it going up and up and instead of getting an OOME, you can predict
> when it will happen and bounce the server at your convenience.
>
hahaha, i can see myself doing this... i monitor how tomcat/tomee running
often via java visual vm (JMX connection). And i monitor the log files a
lot, looking for exceptions, checking performance in date/time, how
JMS/ActiveMQ jobs are completing (sometimes I do that).
>
> 2. Since the cost of a GC is related to the number of live objects
> during a collection and not the size of the heap (though obviously a
> smaller heap can fit fewer live objects!), having a huge heap means
> that GCs will occur less frequently and so your total GC throughput
> will (at least theoretically) be higher.
>
understood, but even with my app, Xms/Xmx = 1024mb, GC seems to occur
'while' people are logged in and working, and eventually there is no GC
activity, but when I do heap dump, i still see long-lived objects (still
resident in memory)...for whatever reason. :(
>
> A counter-argument to the second #2 above is that short-lived objects
> will be collected quickly and long-lived objects will quickly be
> promoted to older generations, so after a short period of runtime,
> your GCs should get to the point where they are very cheap regardless
> of heap size.
>
honestly, i am quite pleased with GC against my app, but my frequent
shutdown/deploy/restart probably doesn't give GC a chance to show it's true
worth. :)
>
> > heap settings tailored to 'my' environment and usage... hmmm, not
> > many users hitting the app, app is not used 'all day long', app has
> > @Schedule tasks that connects to an email acct, downloads customer
> > email requests, and inserts customer requests into database (Martin
> > recommended to close resources; sometime ago, I had to refactor all
> > of that code, and I am closing the connection to the email acct,
> > and open the connection when @Schedule tasks are executed), i am
> > using JMS via TomEE/activeMQ to perform some tasks, asynchronously
> > (tomee committers told me that use of @Asynchronous would be
> > better, and less overhead); honestly, I do open 2 or 3 JMS
> > resources/queues in @ApplicationScoped @PostConstruct (if I'm not
> > mistaking) and close those resources in @ApplicationScoped
> > @PreDestroy; why? I think I read on ActiveMQ site/documentation,
> > where they recommend that that is better on performance, than
> > open/close-on-demand.
>
> IMO, batch processes like the one you describe are better done by
> specialty schedulers like cron on *NIX and the Task Scheduler on
> Windows.
Sounds good to me, but the email server is 'gmail', and I am happily using
JavaMail IMAP connection to grab data from gmail, and grab the body of the
email, and insert data (from the email) into database. Honestly, I don't
see the need for Windows Task Scheduler to do this for me. I will provide
more justification, probably, in the next paragraph below.
That may not be more convenient for you, so I can only tell
> you my opinion: webapps should be focused on the web-based portion(s)
> of your service offering.
+1 I agree 100%, even still, keep reading below, please.
> Divorcing your batch-style processing from your webapp will (at least
> theoretically) make your webapp more stable and your batch-stuff more
> flexible
I would love to divorce scheduled jobs from web app, but web app is
dependent on the database, and i would have to develop a way for the
database to be shared between web app and scheduled-job-app/server. Right
now, that is a bit beyond me. I've researched a bit about that, but not
there yet. :(
> (changed your mind about schedule?
>
nope... hahaha :) keep reading below, will tell you why i'm laughing...
> No problem, just modify crontab or whatever instead of bouncing
> your ebapp). You can also easily move that batch-stuff to another server if
> it starts to get heavy -- needs lots of CPU, memory, etc. -- and you don't
> have to set up a full-block application-server environment
> with Tomcat (TomEE), etc.
>
well, this is the point... when production server is running and busy
serving requests to endusers (or myself, when testing), TomEE only uses 1%
of CPU, and I already told you that with all that the web app is doing, 1%
of CPU and Xms/Xmx = 1024MB... the app is clearly-and-evidently 'not'
stressing-out the server/hardware...at all! :)
>
> > Almost forgot...as I mentioned in another thread, as enduser
> > changes data, I have an implementation that keeps google calendar
> > in sync with the database, which involves JMS/ActiveMQ/MDB and
> > many/multiple requests to google calendar API.
>
> Do you do that in a pipelined way (e.g. queuing the updates) or do you
> do everything symchronously while the (web) user waits?
Months ago, google calendar update was synchronous and enduser had to wait,
but I am now using JMS/ActiveMQ (MDBs) to update google calendar
asynchronously. One of the TomEE committer tells me that @Asynchronous
methods is less overhead, and that may be true, but I guess I am just
overly-pleased at the performance and the working-as-designed-ness of
MDBs/JMS/ActiveMQ asynchronously updating google calendar. i really have
'no' complaints. most of my obstacles/issues, I have done some refactoring
to overcome those obstacles, and sometimes refactoring has been multiple
attempts over days, weeks, months. :)
Hmmm, i thought I had more to share here... oh yeah, i also started using
MDBs/JMS/ActiveMQ to asynchronously insert the data that is downloaded via
the @Schedule task/job. From what I see, the multi-table insert takes
anywhere between 1 to 6 (or 1 to 9) seconds. potential deadlock? honestly,
I think it feels like a deadlock, when those customer requests, but
@Schedule is every 2 minutes, and how many customer requests will come in
at 'one' time when @Schedule actually connects to email server and
downloads emails.
There are advantages to both strategies. Obviously this is all off-topic
> for the
> thread. If you're interested in a separate discussion, please open a
> new thread.
>
my apolgies, but this thread seems to be done
>
> > hmmm, more about usage, I have the following:
> >
> > <Resource id="jdbc/...." type="javax.sql.DataSource"> JdbcDriver
> > org.apache.derby.jdbc.EmbeddedDriver JdbcUrl
> > jdbc:derby:....;create=true UserName .... Password .... JtaManaged
> > true jmxEnabled true InitialSize 2 MaxActive 80 MaxIdle 20 MaxWait
> > 10000 minIdle 10 suspectTimeout 60 removeAbandoned true
> > removeAbandonedTimeout 180 timeBetweenEvictionRunsMillis 30000
> > jdbcInterceptors=StatementCache(max=128) </Resource>
>
> That seems like a lot of db resources to me (again: it all depends
> upon your user cases). We have a user-load of roughly 20-100 users,
> mostly in the US and mostly during "normal" hours (between 06:00 and
> 23:00 EDT), the webapp is *very* RDBMS-heavy, and we have
> maxActive/maxIdle = 20/10. Only under the most extreme circumstances
> do we ever have maxWait (10000ms) problems -- usually when we have a
> bunch of users all performing the same very-complex query
> simultaneously. (We're working on that one ;)
>
Interesting, and thanks for sharing....and might I add, now you're talking!
:)
Currently, these are my settings in tomee.xml file (see below). I had
default settings, first, and then I decreased some of the numbers below,
and then decided to bump them back up to what you see below. With the
following, honestly, myself and endusers are happy (and no complaints).
<Resource id="jdbc/mcmsJta" type="javax.sql.DataSource">
JdbcDriver org.apache.derby.jdbc.EmbeddedDriver
JdbcUrl jdbc:derby:D:/javadb/mcms;create=true
UserName .......
Password .......
JtaManaged true
jmxEnabled true
InitialSize 2
MaxActive 80
MaxIdle 20
MaxWait 10000
minIdle 10
suspectTimeout 60
removeAbandoned true
removeAbandonedTimeout 180
timeBetweenEvictionRunsMillis 30000
jdbcInterceptors=StatementCache(max=128)
</Resource>
My web app was developed to replace a dBase IV DBMS MS-DOS app that I
developed in 1994/1995 as a senior-year project in college. My family has
used the (6-table, all-tables-not-normalized) MS-DOS dBase IV app ever
since 1995 before i started my career as software engineer. Soooo, my web
app is very RDBMS-heavy, too. :)
>
> > I do occasionally see the sawtooth-looking graph, and eventually, I
> > see the graph even out (non-sawtooth-looking graph).
>
> Well, every request requires objects to be created and ultimately
> discarded (e.g. java.lang.String objects to represent your request
> parameters, etc.), so you should never see the sawtooth go away
> completely. Depending upon the parameters of your graph, you might not
> really be able to see it.
>
i don't see saw-tooth, when no one logged in the app, and when GC has
finished doing it's thing (to the short-lived objects, I guess). :)
>
> > remember, I do restart tomee quite often, especially when I have
> > software updates to deploy to/on the server. It's been a while,
> > since I let it run a few days without stop-deploy-start. I am
> > looking forward to letting it be and running without touching it,
> > so I can see if it results in an OOME or reaches the peak.
>
> You could always run a load-test against it in a test bed. JMeter is
> your friend.
>
hahaha +1 agreed! will have to do that. thanks for the recommendation. :)
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJRazL5AAoJEBzwKT+lPKRY1AkP/i8fDS/trZEvabWHTR5Ly5TW
> zC9T63meZn3KaOwtM7aZSgXeMies8ZCQBjhVm4bwMDIBknt3cDR8WXKshFCGP7eC
> LeCHzYJQkfEqfNOkMkb+FGYsXOKyB33HGLlquQ6VdJKq+UQ8Shvr6CjwBmDfgOe2
> TC8EyKHcv/AD/3xBQKCfZr9xZvy2Pd+ut37QqLGV4d9I4BfH172B3lhdnav7Ovf5
> 6y0NCdfmd85QoCFhXNSCCZZOzJ2uiT0yFaEokFcRJuTDPBIpQ1PhLYx/kdTBN09W
> tS5maDvQFbr7piIiDsnD2mZwtXKi6n5xQTAB6lBhVSkVNFV8A0Uj5+4n9Aj+sHGP
> QHS5jFVn9cY6GljZ5WTbTtOpCiHb8/p1a0kISDnhzg7eabsBA5ONn2hHRVtz0gU3
> R/DAgTxh/IGJa5F4AzDPsGEIHZ8gy3kJrGjvmvg/dZCdwrCKSTVArmf188/45NSy
> 6+KKEuXqTYOiVOy7EqGM8Mg00UX7GLXIKJppLXdMFtJ5YJeqef0/fvWbP5PYxUb8
> NgQZmK9rUTKbvPGnV6AAJixDhCj8jv/AkChgunPl20b/PAuioyVSo6X5tCByJd3j
> 33kSqbdVf89gNb2ZFbqAqVAseCQKB+6+2MMGO23k5M9x0kUmmcb1lmTCKz3e83Yj
> JqH0B7XxPzwKKf8lR+KM
> =6LvH
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
RE: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by Jeffrey Janner <Je...@PolyDyne.com>.
> -----Original Message-----
> From: Christopher Schultz [mailto:chris@christopherschultz.net]
> Sent: Sunday, April 14, 2013 5:52 PM
> To: Tomcat Users List
> Subject: Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
>
> I've had people tell me that I should run with the biggest heap I "can
> afford" meaning both financially - 'cause you have to buy a bunch of
> memory - and reasonably within the constraints of the OS (it's not
> reasonably to run a 9.9GiB heap with 10GiB of physical RAM, for
> instance). The reasoning is twofold:
>
> 1. If you have leaks, they will take a lot more time to blow up.
> (Obviously, this is the opposite of my recommendation, but it's worth
> mentioning as it's a sound argument. I just disagree with the
> conclusion). If you watch the heap-usage profile over time, you can see
> it going up and up and instead of getting an OOME, you can predict when
> it will happen and bounce the server at your convenience.
>
Chris -
My back-argument to this reasoning is this:
It's fine for the production side in order to maximize uptime while you investigate the cause of the leaks.
Then I recommend your suggestion for the Dev/Test environment to isolate the cause(s).
Once fixed, bring the production side back to something resembling normality.
Jeff
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Howard,
On 4/11/13 10:38 PM, Howard W. Smith, Jr. wrote:
> On Thu, Apr 4, 2013 at 2:32 PM, Christopher Schultz <
> chris@christopherschultz.net> wrote:
>
>> Your heap settings should be tailored to your environment and
>> usage scenarios.
>
> Interesting. I suppose 'your environment' means memory available,
> operating system, hardware. Usage scenarios? hmmm... please clarify
> with a brief example, thanks. :)
Here's an example: Let's say that your webapp doesn't use HttpSessions
and does no caching. You need to be able to handle 100 simultaneous
connections that do small fetches/inserts from/into a relational
database. Your pages are fairly simple and don't have any kind of
heavyweight app framework taking-up a whole bunch of memory to do
simple things.
For this situation, you can probably get away with a 64MiB heap. If
your webapp uses more than 64MiB, there is probably some kind of
problem. If you only need a 64MiB heap, then you can probably run on
fairly modest hardware: there's no need to lease that 128GiB server
your vendor is trying to talk you into.
On the other hand, maybe you have aggressive caching of data that
benefits from having a large amount of heap space. Or maybe you need
to support 1000 simultaneous connections and need to do XSLT
processing of multi-megabyte XML documents and your XSLTs don't allow
stream-processing of the XML document (oops). Or maybe you have to
keep a large amount of data in users' HttpSession objects (maybe a few
dozen MiB) and you need to support a few thousand simultaneous users
(not connections). 10k users each with a 5MiB heap = 48GiB.
There is no such thing as a "good recommendation" for heap size unless
the person making the recommendation really understands your use case(s).
I generally have these two suggestions that I've found to be
universally reasonable:
1. Make -Xms = -Xmx to eliminate heap thrashing: the JVM is going to
eat-up that large heap space at some point if you have sized things
correctly, so you may as well not make the memory manager have to work
any harder than necessary.
2. Run with the lowest heap space that is reasonable for your
environment. I like doing this because it actually helps you diagnose
things more easily when they go wrong: a small heap yields a smaller
heap-dump file, is GC'd more frequently and therefore contains fewer
long-lived dead objects, and will cause an OOME sooner if you have
some kind of leak. Of course, nobody wants to experience an OOME but
you also don't want to watch a 50GiB heap fill-up 800 bytes at a time
due to a small leak.
I've had people tell me that I should run with the biggest heap I "can
afford" meaning both financially - 'cause you have to buy a bunch of
memory - and reasonably within the constraints of the OS (it's not
reasonably to run a 9.9GiB heap with 10GiB of physical RAM, for
instance). The reasoning is twofold:
1. If you have leaks, they will take a lot more time to blow up.
(Obviously, this is the opposite of my recommendation, but it's worth
mentioning as it's a sound argument. I just disagree with the
conclusion). If you watch the heap-usage profile over time, you can
see it going up and up and instead of getting an OOME, you can predict
when it will happen and bounce the server at your convenience.
2. Since the cost of a GC is related to the number of live objects
during a collection and not the size of the heap (though obviously a
smaller heap can fit fewer live objects!), having a huge heap means
that GCs will occur less frequently and so your total GC throughput
will (at least theoretically) be higher.
A counter-argument to the second #2 above is that short-lived objects
will be collected quickly and long-lived objects will quickly be
promoted to older generations, so after a short period of runtime,
your GCs should get to the point where they are very cheap regardless
of heap size.
> heap settings tailored to 'my' environment and usage... hmmm, not
> many users hitting the app, app is not used 'all day long', app has
> @Schedule tasks that connects to an email acct, downloads customer
> email requests, and inserts customer requests into database (Martin
> recommended to close resources; sometime ago, I had to refactor all
> of that code, and I am closing the connection to the email acct,
> and open the connection when @Schedule tasks are executed), i am
> using JMS via TomEE/activeMQ to perform some tasks, asynchronously
> (tomee committers told me that use of @Asynchronous would be
> better, and less overhead); honestly, I do open 2 or 3 JMS
> resources/queues in @ApplicationScoped @PostConstruct (if I'm not
> mistaking) and close those resources in @ApplicationScoped
> @PreDestroy; why? I think I read on ActiveMQ site/documentation,
> where they recommend that that is better on performance, than
> open/close-on-demand.
IMO, batch processes like the one you describe are better done by
specialty schedulers like cron on *NIX and the Task Scheduler on
Windows. That may not be more convenient for you, so I can only tell
you my opinion: webapps should be focused on the web-based portion(s)
of your service offering. Divorcing your batch-style processing from
your webapp will (at least theoretically) make your webapp more stable
and your batch-stuff more flexible (changed your mind about schedule?
No problem, just modify crontab or whatever instead of bouncing your
webapp). You can also easily move that batch-stuff to another server
if it starts to get heavy -- needs lots of CPU, memory, etc. -- and
you don't have to set up a full-block application-server environment
with Tomcat (TomEE), etc.
> Almost forgot...as I mentioned in another thread, as enduser
> changes data, I have an implementation that keeps google calendar
> in sync with the database, which involves JMS/ActiveMQ/MDB and
> many/multiple requests to google calendar API.
Do you do that in a pipelined way (e.g. queuing the updates) or do you
do everything symchronously while the (web) user waits? There are
advantages to both strategies. Obviously this is all off-topic for the
thread. If you're interested in a separate discussion, please open a
new thread.
> hmmm, more about usage, I have the following:
>
> <Resource id="jdbc/...." type="javax.sql.DataSource"> JdbcDriver
> org.apache.derby.jdbc.EmbeddedDriver JdbcUrl
> jdbc:derby:....;create=true UserName .... Password .... JtaManaged
> true jmxEnabled true InitialSize 2 MaxActive 80 MaxIdle 20 MaxWait
> 10000 minIdle 10 suspectTimeout 60 removeAbandoned true
> removeAbandonedTimeout 180 timeBetweenEvictionRunsMillis 30000
> jdbcInterceptors=StatementCache(max=128) </Resource>
That seems like a lot of db resources to me (again: it all depends
upon your user cases). We have a user-load of roughly 20-100 users,
mostly in the US and mostly during "normal" hours (between 06:00 and
23:00 EDT), the webapp is *very* RDBMS-heavy, and we have
maxActive/maxIdle = 20/10. Only under the most extreme circumstances
do we ever have maxWait (10000ms) problems -- usually when we have a
bunch of users all performing the same very-complex query
simultaneously. (We're working on that one ;)
> I do occasionally see the sawtooth-looking graph, and eventually, I
> see the graph even out (non-sawtooth-looking graph).
Well, every request requires objects to be created and ultimately
discarded (e.g. java.lang.String objects to represent your request
parameters, etc.), so you should never see the sawtooth go away
completely. Depending upon the parameters of your graph, you might not
really be able to see it.
> remember, I do restart tomee quite often, especially when I have
> software updates to deploy to/on the server. It's been a while,
> since I let it run a few days without stop-deploy-start. I am
> looking forward to letting it be and running without touching it,
> so I can see if it results in an OOME or reaches the peak.
You could always run a load-test against it in a test bed. JMeter is
your friend.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJRazL5AAoJEBzwKT+lPKRY1AkP/i8fDS/trZEvabWHTR5Ly5TW
zC9T63meZn3KaOwtM7aZSgXeMies8ZCQBjhVm4bwMDIBknt3cDR8WXKshFCGP7eC
LeCHzYJQkfEqfNOkMkb+FGYsXOKyB33HGLlquQ6VdJKq+UQ8Shvr6CjwBmDfgOe2
TC8EyKHcv/AD/3xBQKCfZr9xZvy2Pd+ut37QqLGV4d9I4BfH172B3lhdnav7Ovf5
6y0NCdfmd85QoCFhXNSCCZZOzJ2uiT0yFaEokFcRJuTDPBIpQ1PhLYx/kdTBN09W
tS5maDvQFbr7piIiDsnD2mZwtXKi6n5xQTAB6lBhVSkVNFV8A0Uj5+4n9Aj+sHGP
QHS5jFVn9cY6GljZ5WTbTtOpCiHb8/p1a0kISDnhzg7eabsBA5ONn2hHRVtz0gU3
R/DAgTxh/IGJa5F4AzDPsGEIHZ8gy3kJrGjvmvg/dZCdwrCKSTVArmf188/45NSy
6+KKEuXqTYOiVOy7EqGM8Mg00UX7GLXIKJppLXdMFtJ5YJeqef0/fvWbP5PYxUb8
NgQZmK9rUTKbvPGnV6AAJixDhCj8jv/AkChgunPl20b/PAuioyVSo6X5tCByJd3j
33kSqbdVf89gNb2ZFbqAqVAseCQKB+6+2MMGO23k5M9x0kUmmcb1lmTCKz3e83Yj
JqH0B7XxPzwKKf8lR+KM
=6LvH
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by "Howard W. Smith, Jr." <sm...@gmail.com>.
Chris,
My apologies for late response; just realized earlier this afternoon that I
didn't respond.
On Thu, Apr 4, 2013 at 2:32 PM, Christopher Schultz <
chris@christopherschultz.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Howard,
>
> On 4/3/13 4:15 PM, Howard W. Smith, Jr. wrote:
> > On Tue, Apr 2, 2013 at 5:12 PM, Christopher Schultz <
> > chris@christopherschultz.net> wrote:
> >
> >> If you don't re-deploy your webapp, then daily rolling Tomcat
> >> restarts are not necessary. I wonder why you are re-deploying
> >> your web application so many times?
> >
> > As a new tomcat user and still somewhat junior java/jsf developer,
> > I restart tomcat whenever I have new software changes to
> > deploy-and-want-to-run-on the production server. sometimes, I
> > deploy-and-restart multiple times per day, but sometimes, I'm able
> > to let tomcat/tomee run for days without restart.
>
> That's not really conducive to high-availability. Are you using
> Tomcat's parallel-deployment feature?
>
Agreed, and not using parallel-deployment feature at the moment.
>
> >> We run several Tomcats in parallel with modest heaps (less than
> >> 512MiB each) and they can run for months before we stop them for
> >> upgrades. It *is* possible to run JVMs without running out of
> >> memory...
> >
> >
> > I too, have not experienced any OOME, and recently, per what I
> > have seen-and-read of other (more senior java developers than
> > myself), I have decreased memory settings in my java options on
> > tomcat7w.exe (see below).
> >
> > -Xmx1024m -XX:MaxPermSize=384m -XX:+UseTLAB
> > -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled
>
> Your heap settings should be tailored to your environment and usage
> scenarios.
Interesting. I suppose 'your environment' means memory available, operating
system, hardware. Usage scenarios? hmmm... please clarify with a brief
example, thanks. :)
heap settings tailored to 'my' environment and usage... hmmm, not many
users hitting the app, app is not used 'all day long', app has @Schedule
tasks that connects to an email acct, downloads customer email requests,
and inserts customer requests into database (Martin recommended to close
resources; sometime ago, I had to refactor all of that code, and I am
closing the connection to the email acct, and open the connection when
@Schedule tasks are executed), i am using JMS via TomEE/activeMQ to perform
some tasks, asynchronously (tomee committers told me that use of
@Asynchronous would be better, and less overhead); honestly, I do open 2 or
3 JMS resources/queues in @ApplicationScoped @PostConstruct (if I'm not
mistaking) and close those resources in @ApplicationScoped @PreDestroy;
why? I think I read on ActiveMQ site/documentation, where they recommend
that that is better on performance, than open/close-on-demand.
Almost forgot...as I mentioned in another thread, as enduser changes data,
I have an implementation that keeps google calendar in sync with the
database, which involves JMS/ActiveMQ/MDB and many/multiple requests to
google calendar API.
hmmm, more about usage, I have the following:
<Resource id="jdbc/...." type="javax.sql.DataSource">
JdbcDriver org.apache.derby.jdbc.EmbeddedDriver
JdbcUrl jdbc:derby:....;create=true
UserName ....
Password ....
JtaManaged true
jmxEnabled true
InitialSize 2
MaxActive 80
MaxIdle 20
MaxWait 10000
minIdle 10
suspectTimeout 60
removeAbandoned true
removeAbandonedTimeout 180
timeBetweenEvictionRunsMillis 30000
jdbcInterceptors=StatementCache(max=128)
</Resource>
> You can find "conventional wisdom" that recommends pretty
> much any heap configuration you want. The only thing that I can
> consistently recommend to anyone is to set -Xms and -Xmx to the same
> value, since on a server you're pretty much guaranteed to get to -Xmx
> pretty quickly, anwyay. You may as well not thrash the heap space(s)
> getting there.
>
Interesting, I did set -Xms and -Xmx to the same value (as you and
others-on-this-list have recommended, thanks).
>
> > My Windows 2008 R2 Server (64bit 32GB RAM) never seems to get
> > higher than 1% CPU, and I think I do have memory leaks somewhere in
> > the app, but FWIW (in heap dump in java visual vm), the memory
> > leaks seem to be tomee leaks. In Java Visual VM, I do see the
> > memory grow over time, as the app is used (without a tomcat restart
> > or re-deploy of app and then restart tomcat), but I still have not
> > seen OOME...'yet'.
>
> What does your heap usage graph look like? It should be a nice
> sawtooth-looking thing, like this:
>
> /|/|/|/|/|/|/|/|/|
>
>
I do occasionally see the sawtooth-looking graph,
> You'll see that the small sawtooth pattern grows in basis over time
> and then there is a major GC which will reset you back to some
> baseline, then the process starts over again.
>
and eventually, I see the graph even out (non-sawtooth-looking graph).
>
> If you never get OOMEs, why do you think you have memory leaks?
remember, I do restart tomee quite often, especially when I have software
updates to deploy to/on the server. It's been a while, since I let it run a
few days without stop-deploy-start. I am looking forward to letting it be
and running without touching it, so I can see if it results in an OOME or
reaches the peak.
Or
> that TomEE does?
>
I would rather take the blame. My first year of developing the JSF web app,
I was all into xhtml pages and Java, now I am more in the Java side of
things and performance performance performance, especially since I'm using
tomcat (via tomee) and listening in on many topics on this tomcat user list
(and learning and trying to apply this/that).
I'm already spreading the word to others how helpful you all are and how
I'm learning a bit more...outside of JSF world. :)
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJRXcdIAAoJEBzwKT+lPKRYb70P/jI9QtV1+anIXCx04niQNDJP
> BSipr2ScOQ2DJNNBjwWAth9AqbzC/nKX4fNlqKCSqWmpZf3JyFChFi/nQJajZQRR
> 3EJEYLXP3YBRPfixRZLrcUw9rrVLs/+apFL4XZCdgkFvrPto201E9ef99ns7Ya+g
> oBWr51g8Fkx5jeztRmzg+H/8+KVICuHUVXTCBQtuybwTfCyWKFwZs5lfmKiX7oKC
> 1ts7S6mIhzpFjO+KFxnFWZkM4ch7SADqCe5WhxnOYpjaz7p7d2XGv1iamjsGVKF2
> FE1bZvOiyEFga6vlh1TpAFmAnOmHmkHtxNAG6eSLeOfYkxKAqSy4z5O6leoP/tUc
> Vri2+GIxwyxyUYprTnvQW7S1UTQF6t658itnZXLrnvRr8Oj/7tzmFcScDvajbYUx
> P8tECk7SENg0Sr6T8tY3VjR+A1c1u8i5k9iLDayVE8zxcDvCA6n2cVmtj/5K3xwe
> MNWhsr6kfca9q7CU5fIe38ebrsw8+UU4J2TkrtHYUpc9uC1AxXmLcC8LEXCV/IiZ
> pIHVlIj2VztPODHfpQvPVUH7VkQKu0M3WbeR+OTQDJJs8X7MLou+YFao+3ri0x9/
> 09i6VS+AC8E8zqQcl0KNeBJf7kuTKP/lguLcCRby0YC2YuppwHaBcRiECKy1UDZR
> FB/X9VbiGKGt95lZvZae
> =X3iQ
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Howard,
On 4/3/13 4:15 PM, Howard W. Smith, Jr. wrote:
> On Tue, Apr 2, 2013 at 5:12 PM, Christopher Schultz <
> chris@christopherschultz.net> wrote:
>
>> If you don't re-deploy your webapp, then daily rolling Tomcat
>> restarts are not necessary. I wonder why you are re-deploying
>> your web application so many times?
>
> As a new tomcat user and still somewhat junior java/jsf developer,
> I restart tomcat whenever I have new software changes to
> deploy-and-want-to-run-on the production server. sometimes, I
> deploy-and-restart multiple times per day, but sometimes, I'm able
> to let tomcat/tomee run for days without restart.
That's not really conducive to high-availability. Are you using
Tomcat's parallel-deployment feature?
>> We run several Tomcats in parallel with modest heaps (less than
>> 512MiB each) and they can run for months before we stop them for
>> upgrades. It *is* possible to run JVMs without running out of
>> memory...
>
>
> I too, have not experienced any OOME, and recently, per what I
> have seen-and-read of other (more senior java developers than
> myself), I have decreased memory settings in my java options on
> tomcat7w.exe (see below).
>
> -Xmx1024m -XX:MaxPermSize=384m -XX:+UseTLAB
> -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled
Your heap settings should be tailored to your environment and usage
scenarios. You can find "conventional wisdom" that recommends pretty
much any heap configuration you want. The only thing that I can
consistently recommend to anyone is to set -Xms and -Xmx to the same
value, since on a server you're pretty much guaranteed to get to -Xmx
pretty quickly, anwyay. You may as well not thrash the heap space(s)
getting there.
> My Windows 2008 R2 Server (64bit 32GB RAM) never seems to get
> higher than 1% CPU, and I think I do have memory leaks somewhere in
> the app, but FWIW (in heap dump in java visual vm), the memory
> leaks seem to be tomee leaks. In Java Visual VM, I do see the
> memory grow over time, as the app is used (without a tomcat restart
> or re-deploy of app and then restart tomcat), but I still have not
> seen OOME...'yet'.
What does your heap usage graph look like? It should be a nice
sawtooth-looking thing, like this:
/|/|/|/|/|/|/|/|/|
You'll see that the small sawtooth pattern grows in basis over time
and then there is a major GC which will reset you back to some
baseline, then the process starts over again.
If you never get OOMEs, why do you think you have memory leaks? Or
that TomEE does?
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJRXcdIAAoJEBzwKT+lPKRYb70P/jI9QtV1+anIXCx04niQNDJP
BSipr2ScOQ2DJNNBjwWAth9AqbzC/nKX4fNlqKCSqWmpZf3JyFChFi/nQJajZQRR
3EJEYLXP3YBRPfixRZLrcUw9rrVLs/+apFL4XZCdgkFvrPto201E9ef99ns7Ya+g
oBWr51g8Fkx5jeztRmzg+H/8+KVICuHUVXTCBQtuybwTfCyWKFwZs5lfmKiX7oKC
1ts7S6mIhzpFjO+KFxnFWZkM4ch7SADqCe5WhxnOYpjaz7p7d2XGv1iamjsGVKF2
FE1bZvOiyEFga6vlh1TpAFmAnOmHmkHtxNAG6eSLeOfYkxKAqSy4z5O6leoP/tUc
Vri2+GIxwyxyUYprTnvQW7S1UTQF6t658itnZXLrnvRr8Oj/7tzmFcScDvajbYUx
P8tECk7SENg0Sr6T8tY3VjR+A1c1u8i5k9iLDayVE8zxcDvCA6n2cVmtj/5K3xwe
MNWhsr6kfca9q7CU5fIe38ebrsw8+UU4J2TkrtHYUpc9uC1AxXmLcC8LEXCV/IiZ
pIHVlIj2VztPODHfpQvPVUH7VkQKu0M3WbeR+OTQDJJs8X7MLou+YFao+3ri0x9/
09i6VS+AC8E8zqQcl0KNeBJf7kuTKP/lguLcCRby0YC2YuppwHaBcRiECKy1UDZR
FB/X9VbiGKGt95lZvZae
=X3iQ
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by "Howard W. Smith, Jr." <sm...@gmail.com>.
Chris,
On Tue, Apr 2, 2013 at 5:12 PM, Christopher Schultz <
chris@christopherschultz.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
>
> > I understand its not a premanent solution but as a stopgap for now
> > ? If so , we can put it as part of daily cycle to bounce tomcat6.
>
> If you don't re-deploy your webapp, then daily rolling Tomcat restarts
> are not necessary. I wonder why you are re-deploying your web
> application so many times?
>
>
As a new tomcat user and still somewhat junior java/jsf developer, I
restart tomcat whenever I have new software changes to
deploy-and-want-to-run-on the production server. sometimes, I
deploy-and-restart multiple times per day, but sometimes, I'm able to let
tomcat/tomee run for days without restart.
> We run several Tomcats in parallel with modest heaps (less than 512MiB
> each) and they can run for months before we stop them for upgrades. It
> *is* possible to run JVMs without running out of memory...
>
I too, have not experienced any OOME, and recently, per what I have
seen-and-read of other (more senior java developers than myself), I have
decreased memory settings in my java options on tomcat7w.exe (see below).
-Xmx1024m
-XX:MaxPermSize=384m
-XX:+UseTLAB
-XX:+UseConcMarkSweepGC
-XX:+CMSClassUnloadingEnabled
On Java tab of tomcat7w.exe, have the following specified:
Initial memory pool = 256
Maximum memory pool =
Thread stack size =
My Windows 2008 R2 Server (64bit 32GB RAM) never seems to get higher than
1% CPU, and I think I do have memory leaks somewhere in the app, but FWIW
(in heap dump in java visual vm), the memory leaks seem to be tomee leaks.
In Java Visual VM, I do see the memory grow over time, as the app is used
(without a tomcat restart or re-deploy of app and then restart tomcat), but
I still have not seen OOME...'yet'.
I should leave the server up and running without restarting tomcat, but
when I need to deploy new software changes, i can't wait to deploy. :)
My apologies, not trying to hijack the thread. just sharing my response to
your comments.
Howard
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJRW0m2AAoJEBzwKT+lPKRYt38QAJkMt1+S+h48OdnVU8GVNY+x
> nhBJMX2THAN9d6LIo7k/8bEUPlbBIbjZVHZeR7tPbt3HgcPaPXLZ+/eiPU7x3tEe
> ubn/Vp7WVNhrZ3+ZcHQz6AEBz5JfFoHNpwfPvopTxsClhdlbNkQVLoNQX2DHvao6
> ARFG3pZWX8HlC/MYMoIsJLD6DxXj9zT0E1ZJzBImHt8r2zE9YPWo35k6RhnEjLut
> Ol74NejK8kD8orgGfJfz5bM6XXiWaxLm3tBXkukefcEC9Sq5/PMDP61npTBygFgK
> GhPCoKHfvxtm/oXIFHJrwfibpyoKa5gxdRPRey9PqqKc9zABw8t7oMHb/QxwC6qi
> qNe9xy5/iSK3NKqQLawXMsFmGfmTA2Rx13I0uP7TnP/2X0bjdjNa/uHx+VKPYY/U
> RfdqosN02x8BwXkXbDRzeQURiTAPwSKdH8PBL9itnYSLGi6byYynfsvsPeycbpuK
> zV0qdyvBOeHbz/j5dai7Z451PMxm5ccEZ1B8cPtQiRVXxx5+5lF3q9RFwtkenLbI
> IpfIrpj4D+96KP66Rn0yfVNEeHosljAOJaYLHuexObna2jjkqwgttzQW4KVajeZm
> I1BPQOSHCQVg+Qo2ewVpd6YrnygpTRF6XsKhA/gwuHS+Jy5xJVsXMPNO/kl0y/6f
> k9TjfhdTRUpMqn5siO68
> =DjF0
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by Pïd stèr <pi...@pidster.com>.
On 3 Apr 2013, at 15:36, Christopher Schultz
<ch...@christopherschultz.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Saumil,
>
> Please try to keep discussions on the mailing list to everyone can
> benefit.
>
> On 4/2/13 6:48 PM, saumil shah wrote:
>> For some reason ...I do not see Java process in Task Manager in
>> Windows, just Tomcat6 process. I am assuming killing Tomcat6 kills
>> JVM internally...is this correct assumption ?
>
> I'm not sure what the Tomca6 process is, but if you kill the service
> (and nothing goes wrong), the JVM should terminate.
Tomcat6.exe is the thin windows wrapper used to start the JVM.
>> Also, I am NOT re-deploying the application WAR's for some
>> reason...when I deployed the application using Tomcat Manager ...
>> for some reason it put WAR in the WebApps folder and then exploded
>> them.
> That is entirely expected behavior.
"For some reason": the reason being that it has to put them somewhere
and as they are basically a zip, it unpacks he zip and loads the app
from the unpacked dir.
> If you don't want your webapps to
> be deployed on startup, you'll need to set deployOnStartup="false" in
> your <Host>.
>
> If you set deplyOnStartup="false", you'll have to use the Manager
> webapp to deploy your webapps after Tomcat has started.
This is probably a red herring.
>> Now it seems every time I start Tomcat , it tries to Re-deploy the
>> application because it finds .WAR there ?
>
> I wouldn't use the term "re-deploy" since the webapp isn't running
> prior to that. I usually use the term re-deploy to mean that the
> webapp is taken out of service in a running Tomcat, then a new webapp
> is deployed in its place (usually on the same context path).
The question should be: are you restarting Tomcat frequently already
and if so, why?
p
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJRXD5GAAoJEBzwKT+lPKRYq0gQAJZ9E4OSrYj+ypDTmHUV5ADa
> GdUbo/TYfyZNPFjmHzl/GYeZlYTirs24dA95VacIiAdJhtZ2g0J44Oc6PnT9pqmR
> RxcGtqNF+kN+AV5i0Lf4ewAZE8MQBvWmBHumqt75ETEsYYAlEv0NaVza9EcM7DNi
> RTpR9MrD4QUS7jhrKsyvQiagL7hF8xJgDij9CY5Bc5j6wQjuh6nttW4JtJXOTxb1
> iiMbQqndmX0RkZHY3Dw3BJOuFU/NjAmSGB5pfmDBrA+z6jasH4SZd0KOy3DKcSgX
> EV8ja57U161yJMdH7Bt7ap9C2mpAaoFKMvANaPYCy29oRcUgQ3qMB7lofRCy2NZ5
> JDWDnVaKa4UdXoCK4pUvt/noo4EZwUhHI9y8IAtCOC+5xgEDA65FPeTnXBBWYqyu
> uASODSRe4DdQYfLJjMw6rWmGFCqM+aPICeZKcXlC+UR8eqp3pmoLL1I5Y15xth2y
> mW9dA/qVAUNLUXlW1oWw7b58UAyCi7nVSASC4p3LcQ1K+cnQ8j8/VB0GtMMgA2gL
> Bpk7RR19NlzeUXmK//AN/BthZY6oK09qz6n+yNTi2tiV4T2XCjlRO4dsLOoW73nP
> FPTMSFfzYaeBKkn0pHsORZLCtWz8bZ060dYrfKkKLSKhk11CzwGNeP4A5So9xa+J
> ncBMVYTGyKnanbDE3Fj9
> =dZYT
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Saumil,
On 4/3/13 2:16 PM, saumil shah wrote:
> I did change in Tomcat 6.0/Conf/server.xml unpackWARs="false" and
> autoDeploy="false"
Why? I don't believe I mentioned either of those settings...
> but the logs still complaint about the Deploying web
> applications....
Tomcat should not show an error during startup. If you are seeing
errors during startup, please post them.
> org.apache.catalina.startup.HostConfig deployWAR INFO: Deploying
> web application archive CmcAppActions.war
> org.apache.catalina.core.StandardContextaddApplicationListener
> INFO: The listener "com.sun.faces.config.ConfigureListener" is
> alreadyconfigured for this context. The duplicate definition has
> been ignored.
That's entirely an application problem: Tomcat doesn't ship with
JavaFaces. Check your web.xml: you likely have the same listener
configured twice (just like the INFO message says).
> 2. Also for a 64 bit JVM , could I then just bump up my Tomcat
> Java params as : -Xms1024m -Xmx4096m instead of my default as
> -Xms512m -Xmx1024m -XX:MaxPermSize=1024m instead of my default
> -XX:MaxPermSize=512m
You can set your heap to whatever you want. IMO, there's no reason to
set the heap any higher than you actually need to support your
expected usage.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJRXcfvAAoJEBzwKT+lPKRYI88P/R1LW2S+whP/WZ9kaaohwrPb
2pcPVxtmw+imWftSP4W451smB1qYkBNiUVtuv3p3pg0qWEgXB+oxpn0K+bqVx1O1
KJp7NFQDsXqcu2N3wxcOUGPzg1ZbZQMpimm0sN7FcGQiTDQbWvcVbcpQEI3zez0B
eAVAPS2txuT4ew5UCsNwsNF14GgAqton+3M6zd3+xfMYMjYa7jyKm13xmgKHEZ/a
gSyHVh0zFxWyMhV0fD5LFArCXYcw11tgHO0mflXoJc9zkC8mpF+2xNMecQXe/IMK
zieto5v93REF3eivvqk6cySlWeygs35WAuvFdAMsmtCBXeLRx0aptoM6Z+PrRFLj
dlAQMCuN4irZlXyPOlroAd79R5N1NVUjkm6dLqTM9R6T+VGuechozSGM66Q34stC
dTAa6Pxe0MUAQ79IDl+J3/Kg93Znviwo0MREObOX6FqpLXpKDAzX13TypExnXUjA
JzZ/HGsm8XLK2q/inPfIlUpiCRTkZG81Ff1yq9ekQrqD57rYHYHAcRvrpVgWp5/p
WpgdxdHrHo732w6EeDKcJDzBTzhmKpfYDcmcygndKfNGShQDQBXJ5TDniGsvZXS8
cM6LtHTRV+q1VyYQlMclAQfwQYp9JGJSZIKR2lZtvLti+f0HX4Kob5ZC10MpS4g+
V5nxNrxKzluUFeMsVQO0
=c84e
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by Pïd stèr <pi...@pidster.com>.
On 3 Apr 2013, at 19:16, saumil shah <sa...@hotmail.com> wrote:
> Thanks again Chris ....
> I did change in Tomcat 6.0/Conf/server.xml
> unpackWARs="false" and autoDeploy="false"
> but the logs still complaint about the Deploying web applications....
> org.apache.catalina.startup.HostConfig deployWARINFO: Deploying web application archive CmcAppActions.warorg.apache.catalina.core.StandardContextaddApplicationListenerINFO: The listener "com.sun.faces.config.ConfigureListener" is alreadyconfigured for this context. The duplicate definition has been ignored.
> 2. Also for a 64 bit JVM , could I then just bump up my Tomcat Java params as :
> -Xms1024m -Xmx4096m instead of my default as -Xms512m -Xmx1024m
> -XX:MaxPermSize=1024m instead of my default -XX:MaxPermSize=512m
Yes, but first please tell us where the original settings came from.
512M seems a little high to me for MaxPermSize.
You should enable JMX on this Tomcat instance and use VisualVM (or
JConsole), which you can find in the JDK bin directory to inspect the
process memory use.
p
>
> Appreciate all your help.
> Thanks again.
>> Date: Wed, 3 Apr 2013 10:35:50 -0400
>> From: chris@christopherschultz.net
>> To: users@tomcat.apache.org
>> CC: saumil125@hotmail.com
>> Subject: Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
>>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>>
>> Saumil,
>>
>> Please try to keep discussions on the mailing list to everyone can
>> benefit.
>>
>> On 4/2/13 6:48 PM, saumil shah wrote:
>>> For some reason ...I do not see Java process in Task Manager in
>>> Windows, just Tomcat6 process. I am assuming killing Tomcat6 kills
>>> JVM internally...is this correct assumption ?
>>
>> I'm not sure what the Tomca6 process is, but if you kill the service
>> (and nothing goes wrong), the JVM should terminate.
>>
>>> Also, I am NOT re-deploying the application WAR's for some
>>> reason...when I deployed the application using Tomcat Manager ...
>>> for some reason it put WAR in the WebApps folder and then exploded
>>> them.
>>
>> That is entirely expected behavior. If you don't want your webapps to
>> be deployed on startup, you'll need to set deployOnStartup="false" in
>> your <Host>.
>>
>> If you set deplyOnStartup="false", you'll have to use the Manager
>> webapp to deploy your webapps after Tomcat has started.
>>
>>> Now it seems every time I start Tomcat , it tries to Re-deploy the
>>> application because it finds .WAR there ?
>>
>> I wouldn't use the term "re-deploy" since the webapp isn't running
>> prior to that. I usually use the term re-deploy to mean that the
>> webapp is taken out of service in a running Tomcat, then a new webapp
>> is deployed in its place (usually on the same context path).
>>
>> - -chris
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
>> Comment: GPGTools - http://gpgtools.org
>> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>>
>> iQIcBAEBCAAGBQJRXD5GAAoJEBzwKT+lPKRYq0gQAJZ9E4OSrYj+ypDTmHUV5ADa
>> GdUbo/TYfyZNPFjmHzl/GYeZlYTirs24dA95VacIiAdJhtZ2g0J44Oc6PnT9pqmR
>> RxcGtqNF+kN+AV5i0Lf4ewAZE8MQBvWmBHumqt75ETEsYYAlEv0NaVza9EcM7DNi
>> RTpR9MrD4QUS7jhrKsyvQiagL7hF8xJgDij9CY5Bc5j6wQjuh6nttW4JtJXOTxb1
>> iiMbQqndmX0RkZHY3Dw3BJOuFU/NjAmSGB5pfmDBrA+z6jasH4SZd0KOy3DKcSgX
>> EV8ja57U161yJMdH7Bt7ap9C2mpAaoFKMvANaPYCy29oRcUgQ3qMB7lofRCy2NZ5
>> JDWDnVaKa4UdXoCK4pUvt/noo4EZwUhHI9y8IAtCOC+5xgEDA65FPeTnXBBWYqyu
>> uASODSRe4DdQYfLJjMw6rWmGFCqM+aPICeZKcXlC+UR8eqp3pmoLL1I5Y15xth2y
>> mW9dA/qVAUNLUXlW1oWw7b58UAyCi7nVSASC4p3LcQ1K+cnQ8j8/VB0GtMMgA2gL
>> Bpk7RR19NlzeUXmK//AN/BthZY6oK09qz6n+yNTi2tiV4T2XCjlRO4dsLOoW73nP
>> FPTMSFfzYaeBKkn0pHsORZLCtWz8bZ060dYrfKkKLSKhk11CzwGNeP4A5So9xa+J
>> ncBMVYTGyKnanbDE3Fj9
>> =dZYT
>> -----END PGP SIGNATURE-----
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by saumil shah <sa...@hotmail.com>.
Thanks again Chris ....
I did change in Tomcat 6.0/Conf/server.xml
unpackWARs="false" and autoDeploy="false"
but the logs still complaint about the Deploying web applications....
org.apache.catalina.startup.HostConfig deployWARINFO: Deploying web application archive CmcAppActions.warorg.apache.catalina.core.StandardContextaddApplicationListenerINFO: The listener "com.sun.faces.config.ConfigureListener" is alreadyconfigured for this context. The duplicate definition has been ignored.
2. Also for a 64 bit JVM , could I then just bump up my Tomcat Java params as :
-Xms1024m -Xmx4096m instead of my default as -Xms512m -Xmx1024m
-XX:MaxPermSize=1024m instead of my default -XX:MaxPermSize=512m
Appreciate all your help.
Thanks again.
> Date: Wed, 3 Apr 2013 10:35:50 -0400
> From: chris@christopherschultz.net
> To: users@tomcat.apache.org
> CC: saumil125@hotmail.com
> Subject: Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Saumil,
>
> Please try to keep discussions on the mailing list to everyone can
> benefit.
>
> On 4/2/13 6:48 PM, saumil shah wrote:
> > For some reason ...I do not see Java process in Task Manager in
> > Windows, just Tomcat6 process. I am assuming killing Tomcat6 kills
> > JVM internally...is this correct assumption ?
>
> I'm not sure what the Tomca6 process is, but if you kill the service
> (and nothing goes wrong), the JVM should terminate.
>
> > Also, I am NOT re-deploying the application WAR's for some
> > reason...when I deployed the application using Tomcat Manager ...
> > for some reason it put WAR in the WebApps folder and then exploded
> > them.
>
> That is entirely expected behavior. If you don't want your webapps to
> be deployed on startup, you'll need to set deployOnStartup="false" in
> your <Host>.
>
> If you set deplyOnStartup="false", you'll have to use the Manager
> webapp to deploy your webapps after Tomcat has started.
>
> > Now it seems every time I start Tomcat , it tries to Re-deploy the
> > application because it finds .WAR there ?
>
> I wouldn't use the term "re-deploy" since the webapp isn't running
> prior to that. I usually use the term re-deploy to mean that the
> webapp is taken out of service in a running Tomcat, then a new webapp
> is deployed in its place (usually on the same context path).
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJRXD5GAAoJEBzwKT+lPKRYq0gQAJZ9E4OSrYj+ypDTmHUV5ADa
> GdUbo/TYfyZNPFjmHzl/GYeZlYTirs24dA95VacIiAdJhtZ2g0J44Oc6PnT9pqmR
> RxcGtqNF+kN+AV5i0Lf4ewAZE8MQBvWmBHumqt75ETEsYYAlEv0NaVza9EcM7DNi
> RTpR9MrD4QUS7jhrKsyvQiagL7hF8xJgDij9CY5Bc5j6wQjuh6nttW4JtJXOTxb1
> iiMbQqndmX0RkZHY3Dw3BJOuFU/NjAmSGB5pfmDBrA+z6jasH4SZd0KOy3DKcSgX
> EV8ja57U161yJMdH7Bt7ap9C2mpAaoFKMvANaPYCy29oRcUgQ3qMB7lofRCy2NZ5
> JDWDnVaKa4UdXoCK4pUvt/noo4EZwUhHI9y8IAtCOC+5xgEDA65FPeTnXBBWYqyu
> uASODSRe4DdQYfLJjMw6rWmGFCqM+aPICeZKcXlC+UR8eqp3pmoLL1I5Y15xth2y
> mW9dA/qVAUNLUXlW1oWw7b58UAyCi7nVSASC4p3LcQ1K+cnQ8j8/VB0GtMMgA2gL
> Bpk7RR19NlzeUXmK//AN/BthZY6oK09qz6n+yNTi2tiV4T2XCjlRO4dsLOoW73nP
> FPTMSFfzYaeBKkn0pHsORZLCtWz8bZ060dYrfKkKLSKhk11CzwGNeP4A5So9xa+J
> ncBMVYTGyKnanbDE3Fj9
> =dZYT
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Saumil,
Please try to keep discussions on the mailing list to everyone can
benefit.
On 4/2/13 6:48 PM, saumil shah wrote:
> For some reason ...I do not see Java process in Task Manager in
> Windows, just Tomcat6 process. I am assuming killing Tomcat6 kills
> JVM internally...is this correct assumption ?
I'm not sure what the Tomca6 process is, but if you kill the service
(and nothing goes wrong), the JVM should terminate.
> Also, I am NOT re-deploying the application WAR's for some
> reason...when I deployed the application using Tomcat Manager ...
> for some reason it put WAR in the WebApps folder and then exploded
> them.
That is entirely expected behavior. If you don't want your webapps to
be deployed on startup, you'll need to set deployOnStartup="false" in
your <Host>.
If you set deplyOnStartup="false", you'll have to use the Manager
webapp to deploy your webapps after Tomcat has started.
> Now it seems every time I start Tomcat , it tries to Re-deploy the
> application because it finds .WAR there ?
I wouldn't use the term "re-deploy" since the webapp isn't running
prior to that. I usually use the term re-deploy to mean that the
webapp is taken out of service in a running Tomcat, then a new webapp
is deployed in its place (usually on the same context path).
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJRXD5GAAoJEBzwKT+lPKRYq0gQAJZ9E4OSrYj+ypDTmHUV5ADa
GdUbo/TYfyZNPFjmHzl/GYeZlYTirs24dA95VacIiAdJhtZ2g0J44Oc6PnT9pqmR
RxcGtqNF+kN+AV5i0Lf4ewAZE8MQBvWmBHumqt75ETEsYYAlEv0NaVza9EcM7DNi
RTpR9MrD4QUS7jhrKsyvQiagL7hF8xJgDij9CY5Bc5j6wQjuh6nttW4JtJXOTxb1
iiMbQqndmX0RkZHY3Dw3BJOuFU/NjAmSGB5pfmDBrA+z6jasH4SZd0KOy3DKcSgX
EV8ja57U161yJMdH7Bt7ap9C2mpAaoFKMvANaPYCy29oRcUgQ3qMB7lofRCy2NZ5
JDWDnVaKa4UdXoCK4pUvt/noo4EZwUhHI9y8IAtCOC+5xgEDA65FPeTnXBBWYqyu
uASODSRe4DdQYfLJjMw6rWmGFCqM+aPICeZKcXlC+UR8eqp3pmoLL1I5Y15xth2y
mW9dA/qVAUNLUXlW1oWw7b58UAyCi7nVSASC4p3LcQ1K+cnQ8j8/VB0GtMMgA2gL
Bpk7RR19NlzeUXmK//AN/BthZY6oK09qz6n+yNTi2tiV4T2XCjlRO4dsLOoW73nP
FPTMSFfzYaeBKkn0pHsORZLCtWz8bZ060dYrfKkKLSKhk11CzwGNeP4A5So9xa+J
ncBMVYTGyKnanbDE3Fj9
=dZYT
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Saumil,
On 4/2/13 5:01 PM, saumil shah wrote:
> Thanks so much Chris this has been very very helpful ...
> appreciate you taking time out. Would stopping Tomcat6 service in
> Windows 2008 R2 take care of this "orphan threads" if you will ?
> Also, does force kill taskkill tomcat6 help either ?
If the JVM shuts down, then everything goes away and you can start fresh.
I assume you are getting an OutOfMemoryError at some point... what is
the exact message and any stack trace that accompanies it?
> I understand its not a premanent solution but as a stopgap for now
> ? If so , we can put it as part of daily cycle to bounce tomcat6.
If you don't re-deploy your webapp, then daily rolling Tomcat restarts
are not necessary. I wonder why you are re-deploying your web
application so many times?
We run several Tomcats in parallel with modest heaps (less than 512MiB
each) and they can run for months before we stop them for upgrades. It
*is* possible to run JVMs without running out of memory...
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJRW0m2AAoJEBzwKT+lPKRYt38QAJkMt1+S+h48OdnVU8GVNY+x
nhBJMX2THAN9d6LIo7k/8bEUPlbBIbjZVHZeR7tPbt3HgcPaPXLZ+/eiPU7x3tEe
ubn/Vp7WVNhrZ3+ZcHQz6AEBz5JfFoHNpwfPvopTxsClhdlbNkQVLoNQX2DHvao6
ARFG3pZWX8HlC/MYMoIsJLD6DxXj9zT0E1ZJzBImHt8r2zE9YPWo35k6RhnEjLut
Ol74NejK8kD8orgGfJfz5bM6XXiWaxLm3tBXkukefcEC9Sq5/PMDP61npTBygFgK
GhPCoKHfvxtm/oXIFHJrwfibpyoKa5gxdRPRey9PqqKc9zABw8t7oMHb/QxwC6qi
qNe9xy5/iSK3NKqQLawXMsFmGfmTA2Rx13I0uP7TnP/2X0bjdjNa/uHx+VKPYY/U
RfdqosN02x8BwXkXbDRzeQURiTAPwSKdH8PBL9itnYSLGi6byYynfsvsPeycbpuK
zV0qdyvBOeHbz/j5dai7Z451PMxm5ccEZ1B8cPtQiRVXxx5+5lF3q9RFwtkenLbI
IpfIrpj4D+96KP66Rn0yfVNEeHosljAOJaYLHuexObna2jjkqwgttzQW4KVajeZm
I1BPQOSHCQVg+Qo2ewVpd6YrnygpTRF6XsKhA/gwuHS+Jy5xJVsXMPNO/kl0y/6f
k9TjfhdTRUpMqn5siO68
=DjF0
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by saumil shah <sa...@hotmail.com>.
Thanks so much Chris this has been very very helpful ... appreciate you taking time out. Would stopping Tomcat6 service in Windows 2008 R2 take care of this "orphan threads" if you will ? Also, does force kill taskkill tomcat6 help either ?
I understand its not a premanent solution but as a stopgap for now ? If so , we can put it as part of daily cycle to bounce tomcat6.
Many thanks.
> Date: Tue, 2 Apr 2013 15:58:53 -0400
> From: chris@christopherschultz.net
> To: users@tomcat.apache.org
> Subject: Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Saumil,
>
> On 4/1/13 11:01 PM, saumil shah wrote:
> > I recently deployed one of the COTS products SAP Business Objects.
> > When the product was deployed , everything seemed to run fine but
> > yesterday we started experiencing "Service is unavailable" error ,
> > upon enabling DEBUG logs in Tomcat , we saw the error below with
> > memory leak message. We were wondering what could have caused it
> > ....
> >
> > The Tomcat is 64 bit , JVM is 64 bit but the applications deployed
> > are 32 bit webapps .... we say Tomcat6 process becoming
> > unresponsive around 2GB mark.
> >
> >
> > Apr 1, 2013 8:53:18 PM org.apache.coyote.http11.Http11Protocol
> > pauseINFO: Pausing Coyote HTTP/1.1 on http-8080Apr 1, 2013 8:53:19
> > PM org.apache.catalina.core.StandardService stopINFO: Stopping
> > service CatalinaApr 1, 2013 8:53:19 PM
> > org.apache.catalina.loader.WebappClassLoader
> > clearReferencesThreadsSEVERE: The web application
> > [/AnalyticalReporting] appears to have started a thread named
> > [Business Objects - Sessions Clean up] but has failed to stop it.
> > This is very likely to create a memory leak.
>
> This means what it says it means: your webapp (not Tomcat) started a
> thread that wasn't stopped by the time the webapp was stopped. This
> will pin the webapp's ClassLoader in memory and you'll get a whole
> bunch of stuff that can't be GC'd. The simple explanation is that you
> need to be sure that your threads are stopped. The solution may be
> complicated.
>
> Presumably any component that starts a thread also knows how to stop
> that same thread. Check the documentation for those components to make
> sure that you are shutting them down properly (usually in a
> ServletContextListener, etc.).
>
> Sometimes, it takes a few seconds for a thread to complete its work
> and actually terminate, and the messages you get from Tomcat are
> inaccurate. That's for you to determine.
>
> > Apr 1, 2013 8:53:19 PM
> > org.apache.catalina.loader.WebappClassLoader
> > clearReferencesThreadsSEVERE: The web application
> > [/AnalyticalReporting] appears to have started a thread named
> > [AWT-Windows] but has failed to stop it. This is very likely to
> > create a memory leak.
>
> Read the documentation for the JreMemoryLeakLeakPreventionListener.
> This is a standard Tomcat component that can be used to work-around
> well-known memory leaks, etc. that are triggered by certain JVM calls.
> There are specific configurations for preventing AWT threads from
> being problematic, but they are not enabled by default. You probably
> want to enable them if you are using AWT for anything.
>
> > Apr 1, 2013 8:53:19 PM
> > org.apache.catalina.loader.WebappClassLoader
> > clearReferencesThreadsSEVERE: The web application
> > [/AnalyticalReporting] appears to have started a thread named
> > [ORBacus:Client:SenderThread] but has failed to stop it. This is
> > very likely to create a memory leak.
>
> Anything with "ORB" in the name usually means a CORBA-related
> component: check to see that anything that start threads is also
> stopping them.
>
> > Apr 1, 2013 8:53:19 PM
> > org.apache.catalina.loader.WebappClassLoader
> > clearThreadLocalMapSEVERE: The web application
> > [/AnalyticalReporting] created a ThreadLocal with key of type
> > [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@63e3d2dd])
> > and a value of type [com.businessobjects.wp.xml.jaxp.XMLJaxpParser]
> > (value [com.businessobjects.wp.xml.jaxp.XMLJaxpParser@228de67]) but
> > failed to remove it when the web application was stopped. This is
> > very likely to create a memory leak.
>
> Looks like your Business Objects version has a bug in it that does not
> clear-out ThreadLocal values when requests are completed. Talk to them
> about this problem.
>
> > Apr 1, 2013 8:53:20 PM
> > org.apache.catalina.loader.WebappClassLoader
> > clearThreadLocalMapSEVERE: The web application [/VoyagerClient]
> > created a ThreadLocal with key of type
> > [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl]
> >
> >
> (value
> > [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@13123b1c])
> >
> >
> and a value of type
> > [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter]
> >
> >
> (value
> > [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@11e6b50c])
> >
> >
> but failed to remove it when the web application was stopped. This is
> > very likely to create a memory leak.Apr 1, 2013 8:53:20 PM
> > org.apache.catalina.loader.WebappClassLoader
> > clearThreadLocalMapSEVERE: The web application [/VoyagerClient]
> > created a ThreadLocal with key of type
> > [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl]
> >
> >
> (value
> > [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@41783002])
> >
> >
> and a value of type
> > [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter]
> >
> >
> (value
> > [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@3997f4e2])
> >
> >
> but failed to remove it when the web application was stopped. This is
> > very likely to create a memory leak.
>
> Looks like AspectJ has a similar problem: ThreadLocals need to be
> removed from threads after every request.
>
> You are probably seeing a pattern, here, so I won't bother looking at
> every other error message you have posted.
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJRWzh6AAoJEBzwKT+lPKRYUoQQALSx4IVs9Pd/QfAQKeB6klIP
> 9bfoG8bRk/fcqdQiJfaNJGw8xaKm0WVMalYB/U1xH7dZSdFDlz4LoQuitRLEMnsX
> WyRspvI3N3KUIhGUuU9GiVz0nn1vwaQF87K81g/7IpCOKcFsWfruHnict2UMjrg1
> pEI7xiNoC3oPsbd6+APhAy9JUip8gt4j7I0VUVAvCOg4L7PLvKGY3y4TXy594uFO
> D/QZ6cN/9SdImOO9wunwHk1+Ot3B3ZKuurdbJk6djXyt6K2Et4lHGJ8KldhZfgfj
> QdQjRg/ybTune+gWJ9QTPo5p7FOgTJxS2x94a+pXx9aTkCBqbBiKCWWU47O+wtf4
> P5Nc71m0N1zeLsSjThjDoaH8DA77AbwtNWHQEwemvqp6y8CQy8ypp/f58lrzdOza
> lFhER7mMtzlNNCHdxZKUEY9yBaEIzWAV9vPP51yWEUBCJaGRZ9icTZNvIQPUAcLB
> JzoIfgEHFjrVlLXCKb0phIq6He0B/yDZ4xwA2rn9bXKhRJEuZbBoYXTM6jsRJDNk
> ab/wMZKXfK5On5N9U9TVj3kzGni35GI2wApJ9OGvYWdKv+9oX4vKQNOl8xAdm8IM
> wAE3ooZk0XvsotLphMP+V9PHXNCaeZeWORsNKKUlkxZDAPU5vdmaVfcdU0um2lm2
> yqAz4cJjThxGwbh62xe3
> =UtMt
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
Re: Re : Memory leak in Tomcat 6.0.35 ( 64 bit)
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Saumil,
On 4/1/13 11:01 PM, saumil shah wrote:
> I recently deployed one of the COTS products SAP Business Objects.
> When the product was deployed , everything seemed to run fine but
> yesterday we started experiencing "Service is unavailable" error ,
> upon enabling DEBUG logs in Tomcat , we saw the error below with
> memory leak message. We were wondering what could have caused it
> ....
>
> The Tomcat is 64 bit , JVM is 64 bit but the applications deployed
> are 32 bit webapps .... we say Tomcat6 process becoming
> unresponsive around 2GB mark.
>
>
> Apr 1, 2013 8:53:18 PM org.apache.coyote.http11.Http11Protocol
> pauseINFO: Pausing Coyote HTTP/1.1 on http-8080Apr 1, 2013 8:53:19
> PM org.apache.catalina.core.StandardService stopINFO: Stopping
> service CatalinaApr 1, 2013 8:53:19 PM
> org.apache.catalina.loader.WebappClassLoader
> clearReferencesThreadsSEVERE: The web application
> [/AnalyticalReporting] appears to have started a thread named
> [Business Objects - Sessions Clean up] but has failed to stop it.
> This is very likely to create a memory leak.
This means what it says it means: your webapp (not Tomcat) started a
thread that wasn't stopped by the time the webapp was stopped. This
will pin the webapp's ClassLoader in memory and you'll get a whole
bunch of stuff that can't be GC'd. The simple explanation is that you
need to be sure that your threads are stopped. The solution may be
complicated.
Presumably any component that starts a thread also knows how to stop
that same thread. Check the documentation for those components to make
sure that you are shutting them down properly (usually in a
ServletContextListener, etc.).
Sometimes, it takes a few seconds for a thread to complete its work
and actually terminate, and the messages you get from Tomcat are
inaccurate. That's for you to determine.
> Apr 1, 2013 8:53:19 PM
> org.apache.catalina.loader.WebappClassLoader
> clearReferencesThreadsSEVERE: The web application
> [/AnalyticalReporting] appears to have started a thread named
> [AWT-Windows] but has failed to stop it. This is very likely to
> create a memory leak.
Read the documentation for the JreMemoryLeakLeakPreventionListener.
This is a standard Tomcat component that can be used to work-around
well-known memory leaks, etc. that are triggered by certain JVM calls.
There are specific configurations for preventing AWT threads from
being problematic, but they are not enabled by default. You probably
want to enable them if you are using AWT for anything.
> Apr 1, 2013 8:53:19 PM
> org.apache.catalina.loader.WebappClassLoader
> clearReferencesThreadsSEVERE: The web application
> [/AnalyticalReporting] appears to have started a thread named
> [ORBacus:Client:SenderThread] but has failed to stop it. This is
> very likely to create a memory leak.
Anything with "ORB" in the name usually means a CORBA-related
component: check to see that anything that start threads is also
stopping them.
> Apr 1, 2013 8:53:19 PM
> org.apache.catalina.loader.WebappClassLoader
> clearThreadLocalMapSEVERE: The web application
> [/AnalyticalReporting] created a ThreadLocal with key of type
> [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@63e3d2dd])
> and a value of type [com.businessobjects.wp.xml.jaxp.XMLJaxpParser]
> (value [com.businessobjects.wp.xml.jaxp.XMLJaxpParser@228de67]) but
> failed to remove it when the web application was stopped. This is
> very likely to create a memory leak.
Looks like your Business Objects version has a bug in it that does not
clear-out ThreadLocal values when requests are completed. Talk to them
about this problem.
> Apr 1, 2013 8:53:20 PM
> org.apache.catalina.loader.WebappClassLoader
> clearThreadLocalMapSEVERE: The web application [/VoyagerClient]
> created a ThreadLocal with key of type
> [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl]
>
>
(value
> [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@13123b1c])
>
>
and a value of type
> [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter]
>
>
(value
> [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@11e6b50c])
>
>
but failed to remove it when the web application was stopped. This is
> very likely to create a memory leak.Apr 1, 2013 8:53:20 PM
> org.apache.catalina.loader.WebappClassLoader
> clearThreadLocalMapSEVERE: The web application [/VoyagerClient]
> created a ThreadLocal with key of type
> [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl]
>
>
(value
> [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl@41783002])
>
>
and a value of type
> [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl.ThreadCounterImpl.Counter]
>
>
(value
> [org.aspectj.runtime.internal.cflowstack.ThreadStackFactoryImpl$ThreadCounterImpl$Counter@3997f4e2])
>
>
but failed to remove it when the web application was stopped. This is
> very likely to create a memory leak.
Looks like AspectJ has a similar problem: ThreadLocals need to be
removed from threads after every request.
You are probably seeing a pattern, here, so I won't bother looking at
every other error message you have posted.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJRWzh6AAoJEBzwKT+lPKRYUoQQALSx4IVs9Pd/QfAQKeB6klIP
9bfoG8bRk/fcqdQiJfaNJGw8xaKm0WVMalYB/U1xH7dZSdFDlz4LoQuitRLEMnsX
WyRspvI3N3KUIhGUuU9GiVz0nn1vwaQF87K81g/7IpCOKcFsWfruHnict2UMjrg1
pEI7xiNoC3oPsbd6+APhAy9JUip8gt4j7I0VUVAvCOg4L7PLvKGY3y4TXy594uFO
D/QZ6cN/9SdImOO9wunwHk1+Ot3B3ZKuurdbJk6djXyt6K2Et4lHGJ8KldhZfgfj
QdQjRg/ybTune+gWJ9QTPo5p7FOgTJxS2x94a+pXx9aTkCBqbBiKCWWU47O+wtf4
P5Nc71m0N1zeLsSjThjDoaH8DA77AbwtNWHQEwemvqp6y8CQy8ypp/f58lrzdOza
lFhER7mMtzlNNCHdxZKUEY9yBaEIzWAV9vPP51yWEUBCJaGRZ9icTZNvIQPUAcLB
JzoIfgEHFjrVlLXCKb0phIq6He0B/yDZ4xwA2rn9bXKhRJEuZbBoYXTM6jsRJDNk
ab/wMZKXfK5On5N9U9TVj3kzGni35GI2wApJ9OGvYWdKv+9oX4vKQNOl8xAdm8IM
wAE3ooZk0XvsotLphMP+V9PHXNCaeZeWORsNKKUlkxZDAPU5vdmaVfcdU0um2lm2
yqAz4cJjThxGwbh62xe3
=UtMt
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org