You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@wicket.apache.org by J <bl...@gmx.com> on 2009/04/15 15:17:42 UTC
WebApp Freezes
RE: WebApp Freezes
Posted by J <bl...@gmx.com>.
-MySQL 5 is configured using defaults, which is max_connections 100
-Tomcat is configured with maxThreads 150
-C3P0 is configured with maxPoolSize 15
I'll increase C3P0's maxPoolSize to 90, to see if that makes any difference
in the occurences of freezes. Most of Tomcats threads are used to serve
static resources (running in a different webapp) that don't use a db
connection at all.
Here some output of JSTAT. I'm not sure if these values are okay.
ps -ef output:
-tomcat 28392 1 8 11:22 pts/0 00:20:52 /usr/java/latest/bin/java
-Xmx512m -Xms512m -Dwicket.configuration=deployment
-Djava.util.logging.manager=org.apache.juli.ClassLoade
[tomcat]$jstat -gc 28392
S0C S1C S0U S1U EC EU OC OU PC
PU YGC YGCT FGC FGCT GCT
4032.0 4032.0 2856.3 0.0 32256.0 19546.2 483968.0 464268.9 22784.0
22633.9 1770 40.553 6 3.114 43.667
-----Oorspronkelijk bericht-----
Van: Martijn Dashorst [mailto:martijn.dashorst@gmail.com]
Verzonden: woensdag 15 april 2009 15:47
Aan: users@wicket.apache.org
Onderwerp: Re: WebApp Freezes
your max database connections should mirror the max request threads of
tomcat. Otherwise you'll endup in deadlock country...
Also make sure you don't have deadlocks in your database.
The lack of errors in your tomcat logs don't necessarily mean there
are no memory problems. Check with jstat -gc <pid> if your garbage
collector is having problems.
Martijn
On Wed, Apr 15, 2009 at 3:41 PM, J <bl...@gmx.com> wrote:
> (Sorry for the empty message. First I tried Gmail, but that doesn't work
> with this mailing list. Then I tried GMX webclient, but that client always
> sends as HTML, which probably caused the message to be stripped to empty.)
>
>
> I'm experiencing freezes on a production website.
>
> Server specs:
> -OS: Linux CentOS
> -Webserver: Tomcat 6
> -MySQL 5 with default settings
> -Frameworks: Wicket 1.4-rc2 (webframework), Spring 2.5.6, Hibernate
> 3.3.1.GA, C3P0 db pooling 0.9.1.2, URLRewriteFilter 3.1.0, Spring
> OpenSessionInViewFilter
>
> Problem:
> At some point (about 5 to 24 hours) after a boot, Tomcat seems to stop
> serving requests, although it does create new threads for new incoming
> connections. But since it does not serve those connections, the new
> connections will show and keep showing 0kb transfer. Shortly after that,
> max-thread is reached, so then I'm unable to access any webapp (the main
> website + tomcat manager) running on that Tomcat server. I'm not sure if
it
> is able to serve static resources in the short time window where it still
> has some threads free, because the time window is too short to notice when
> it happens.
>
> Observations:
> -No exceptions or errors in the catalina logs. So no memory problems,
since
> no error occurs in the logs
> -Java Thread dump using command: kill -QUIT (see output below), shows the
> text "locked" and "WAITING".
> -Adding c3p0 db pooling idle checks, tests and timeout settings (see
below)
> did not help.
> -Using Mysql Administrator GUI shows that after a freeze, there will be 15
> threads, all sleeping. Normally this would return to a minPoolSize of 5,
if
> I'm correct, but thats not the case.
> -The freeze continues for hours, and does not recover. I have to restart
> Tomcat.
>
>
> I'm suspecting that it has something to do with DB pooling. I'm not sure
if
> "locked" and "WAITING" are normal behaviour. But if there is something
> wrong, why doesn't c3p0 recover from it?
>
> Can somebody shed some light on this? :)
>
>
>
>
>
> Configuration:
>
> === DB pooling in Spring applicationContext.xml ====
> <bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource">
> <property
> name="driverClass"><value>com.mysql.jdbc.Driver</value></property>
> <property name="jdbcUrl" value="jdbc:mysql://localhost/xxxxxx" />
> <property name="user"><value>xxxxxxxx</value></property>
> <property name="password"><value>xxxxxxxxx</value></property>
>
> <property name="maxConnectionAge"><value>600</value></property>
> <property
> name="idleConnectionTestPeriod"><value>180</value></property>
> <property
> name="testConnectionOnCheckin"><value>true</value></property>
>
> <property name="acquireIncrement"><value>5</value></property>
> <property name="maxIdleTime"><value>180</value></property>
> <property name="maxPoolSize"><value>15</value></property>
> <property name="maxStatements"><value>100</value></property>
> <property name="minPoolSize"><value>5</value></property>
> </bean>
> ===========================
>
>
> ===== Thread dump part ===========
> "http-8080-28" daemon prio=10 tid=0xb4fbb400 nid=0x4ae2 in Object.wait()
> [0xb4316000..0xb4318130]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at
>
com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePo
> ol.java:1315)
> at
>
com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicRe
> sourcePool.java:557)
> - locked <0x72c74b10> (a
> com.mchange.v2.resourcepool.BasicResourcePool)
> at
>
com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResource
> Pool.java:477)
> at
>
com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C
> 3P0PooledConnectionPool.java:525)
> at
>
com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(Abstract
> PoolBackedDataSource.java:128)
> at
>
org.springframework.orm.hibernate3.LocalDataSourceConnectionProvider.getConn
> ection(LocalDataSourceConnectionProvider.java:81)
> at
>
org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:4
> 23)
> at
>
org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:14
> 4)
> at org.hibernate.jdbc.JDBCContext.connection(JDBCContext.java:119)
> at
> org.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:57)
> at
> org.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:1326)
> at
>
org.springframework.orm.hibernate3.HibernateTransactionManager.doBegin(Hiber
> nateTransactionManager.java:510)
> at
>
org.springframework.transaction.support.AbstractPlatformTransactionManager.g
> etTransaction(AbstractPlatformTransactionManager.java:350)
> at
>
org.springframework.transaction.interceptor.TransactionAspectSupport.createT
> ransactionIfNecessary(TransactionAspectSupport.java:262)
> at
>
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(Tr
> ansactionInterceptor.java:101)
> at
>
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(Reflect
> iveMethodInvocation.java:171)
> at
>
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopPro
> xy.java:204)
> at $Proxy37.findById(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
> at
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
> .java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
>
org.apache.wicket.proxy.LazyInitProxyFactory$JdkHandler.invoke(LazyInitProxy
> Factory.java:416)
> at org.apache.wicket.proxy.$Proxy2.findById(Unknown Source)
> at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> at
>
org.apache.wicket.model.LoadableDetachableModel.getObject(LoadableDetachable
> Model.java:119)
> at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> at sun.reflect.GeneratedConstructorAccessor61.newInstance(Unknown
> Source)
> at
>
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstruc
> torAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
>
org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java
> :151)
> at
>
org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java
> :89)
> at
>
org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.new
> Page(BookmarkablePageRequestTarget.java:306)
> at
>
org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.get
> Page(BookmarkablePageRequestTarget.java:321)
> at
>
org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.pro
> cessEvents(BookmarkablePageRequestTarget.java:234)
> at
>
org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(Abstra
> ctRequestCycleProcessor.java:92)
> at
>
org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:119
> 2)
> at org.apache.wicket.RequestCycle.step(RequestCycle.java:1271)
> at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1370)
> at org.apache.wicket.RequestCycle.request(RequestCycle.java:501)
> at
> org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:455)
> at
>
org.apache.wicket.protocol.http.WicketFilter.doFilter(WicketFilter.java:288)
> at
>
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:235)
> at
>
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:206)
> at
>
org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterI
> nternal(OpenSessionInViewFilter.java:198)
> at
>
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestF
> ilter.java:76)
> at
>
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:235)
> at
>
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:206)
> at
>
org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:164
> )
> at
> org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:141)
> at
>
org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.jav
> a:90)
> at
>
org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter
> .java:417)
> at
>
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:235)
> at
>
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:206)
> at
>
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja
> va:233)
> at
>
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja
> va:191)
> at
>
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128
> )
> at
>
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102
> )
> at
>
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java
> :109)
> at
>
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
> at
>
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http
> 11Protocol.java:583)
> at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
> at java.lang.Thread.run(Thread.java:619)
> =======================================
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
--
Become a Wicket expert, learn from the best: http://wicketinaction.com
Apache Wicket 1.3.5 is released
Get it now: http://www.apache.org/dyn/closer.cgi/wicket/1.3.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: WebApp Freezes
Posted by Martijn Dashorst <ma...@gmail.com>.
your max database connections should mirror the max request threads of
tomcat. Otherwise you'll endup in deadlock country...
Also make sure you don't have deadlocks in your database.
The lack of errors in your tomcat logs don't necessarily mean there
are no memory problems. Check with jstat -gc <pid> if your garbage
collector is having problems.
Martijn
On Wed, Apr 15, 2009 at 3:41 PM, J <bl...@gmx.com> wrote:
> (Sorry for the empty message. First I tried Gmail, but that doesn't work
> with this mailing list. Then I tried GMX webclient, but that client always
> sends as HTML, which probably caused the message to be stripped to empty.)
>
>
> I'm experiencing freezes on a production website.
>
> Server specs:
> -OS: Linux CentOS
> -Webserver: Tomcat 6
> -MySQL 5 with default settings
> -Frameworks: Wicket 1.4-rc2 (webframework), Spring 2.5.6, Hibernate
> 3.3.1.GA, C3P0 db pooling 0.9.1.2, URLRewriteFilter 3.1.0, Spring
> OpenSessionInViewFilter
>
> Problem:
> At some point (about 5 to 24 hours) after a boot, Tomcat seems to stop
> serving requests, although it does create new threads for new incoming
> connections. But since it does not serve those connections, the new
> connections will show and keep showing 0kb transfer. Shortly after that,
> max-thread is reached, so then I'm unable to access any webapp (the main
> website + tomcat manager) running on that Tomcat server. I'm not sure if it
> is able to serve static resources in the short time window where it still
> has some threads free, because the time window is too short to notice when
> it happens.
>
> Observations:
> -No exceptions or errors in the catalina logs. So no memory problems, since
> no error occurs in the logs
> -Java Thread dump using command: kill -QUIT (see output below), shows the
> text "locked" and "WAITING".
> -Adding c3p0 db pooling idle checks, tests and timeout settings (see below)
> did not help.
> -Using Mysql Administrator GUI shows that after a freeze, there will be 15
> threads, all sleeping. Normally this would return to a minPoolSize of 5, if
> I'm correct, but thats not the case.
> -The freeze continues for hours, and does not recover. I have to restart
> Tomcat.
>
>
> I'm suspecting that it has something to do with DB pooling. I'm not sure if
> "locked" and "WAITING" are normal behaviour. But if there is something
> wrong, why doesn't c3p0 recover from it?
>
> Can somebody shed some light on this? :)
>
>
>
>
>
> Configuration:
>
> === DB pooling in Spring applicationContext.xml ====
> <bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource">
> <property
> name="driverClass"><value>com.mysql.jdbc.Driver</value></property>
> <property name="jdbcUrl" value="jdbc:mysql://localhost/xxxxxx" />
> <property name="user"><value>xxxxxxxx</value></property>
> <property name="password"><value>xxxxxxxxx</value></property>
>
> <property name="maxConnectionAge"><value>600</value></property>
> <property
> name="idleConnectionTestPeriod"><value>180</value></property>
> <property
> name="testConnectionOnCheckin"><value>true</value></property>
>
> <property name="acquireIncrement"><value>5</value></property>
> <property name="maxIdleTime"><value>180</value></property>
> <property name="maxPoolSize"><value>15</value></property>
> <property name="maxStatements"><value>100</value></property>
> <property name="minPoolSize"><value>5</value></property>
> </bean>
> ===========================
>
>
> ===== Thread dump part ===========
> "http-8080-28" daemon prio=10 tid=0xb4fbb400 nid=0x4ae2 in Object.wait()
> [0xb4316000..0xb4318130]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at
> com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePo
> ol.java:1315)
> at
> com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicRe
> sourcePool.java:557)
> - locked <0x72c74b10> (a
> com.mchange.v2.resourcepool.BasicResourcePool)
> at
> com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResource
> Pool.java:477)
> at
> com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C
> 3P0PooledConnectionPool.java:525)
> at
> com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(Abstract
> PoolBackedDataSource.java:128)
> at
> org.springframework.orm.hibernate3.LocalDataSourceConnectionProvider.getConn
> ection(LocalDataSourceConnectionProvider.java:81)
> at
> org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:4
> 23)
> at
> org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:14
> 4)
> at org.hibernate.jdbc.JDBCContext.connection(JDBCContext.java:119)
> at
> org.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:57)
> at
> org.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:1326)
> at
> org.springframework.orm.hibernate3.HibernateTransactionManager.doBegin(Hiber
> nateTransactionManager.java:510)
> at
> org.springframework.transaction.support.AbstractPlatformTransactionManager.g
> etTransaction(AbstractPlatformTransactionManager.java:350)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.createT
> ransactionIfNecessary(TransactionAspectSupport.java:262)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(Tr
> ansactionInterceptor.java:101)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(Reflect
> iveMethodInvocation.java:171)
> at
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopPro
> xy.java:204)
> at $Proxy37.findById(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
> .java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.wicket.proxy.LazyInitProxyFactory$JdkHandler.invoke(LazyInitProxy
> Factory.java:416)
> at org.apache.wicket.proxy.$Proxy2.findById(Unknown Source)
> at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> at
> org.apache.wicket.model.LoadableDetachableModel.getObject(LoadableDetachable
> Model.java:119)
> at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> at sun.reflect.GeneratedConstructorAccessor61.newInstance(Unknown
> Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstruc
> torAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java
> :151)
> at
> org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java
> :89)
> at
> org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.new
> Page(BookmarkablePageRequestTarget.java:306)
> at
> org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.get
> Page(BookmarkablePageRequestTarget.java:321)
> at
> org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.pro
> cessEvents(BookmarkablePageRequestTarget.java:234)
> at
> org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(Abstra
> ctRequestCycleProcessor.java:92)
> at
> org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:119
> 2)
> at org.apache.wicket.RequestCycle.step(RequestCycle.java:1271)
> at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1370)
> at org.apache.wicket.RequestCycle.request(RequestCycle.java:501)
> at
> org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:455)
> at
> org.apache.wicket.protocol.http.WicketFilter.doFilter(WicketFilter.java:288)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:206)
> at
> org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterI
> nternal(OpenSessionInViewFilter.java:198)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestF
> ilter.java:76)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:206)
> at
> org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:164
> )
> at
> org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:141)
> at
> org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.jav
> a:90)
> at
> org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter
> .java:417)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:206)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja
> va:233)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja
> va:191)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128
> )
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102
> )
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java
> :109)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
> at
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http
> 11Protocol.java:583)
> at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
> at java.lang.Thread.run(Thread.java:619)
> =======================================
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
--
Become a Wicket expert, learn from the best: http://wicketinaction.com
Apache Wicket 1.3.5 is released
Get it now: http://www.apache.org/dyn/closer.cgi/wicket/1.3.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: WebApp Freezes
Posted by Martijn Dashorst <ma...@gmail.com>.
On Wed, Apr 15, 2009 at 3:41 PM, J <bl...@gmx.com> wrote:
> (Sorry for the empty message. First I tried Gmail, but that doesn't work
> with this mailing list. Then I tried GMX webclient, but that client always
> sends as HTML, which probably caused the message to be stripped to empty.)
gmail just works. dunno what went wrong with your message, but I've
been using gmail for 3-4 years with the wicket lists and never
regretted it.
Martijn
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: WebApp Freezes
Posted by francesco dicarlo <ev...@gmail.com>.
i'm using Gmail... -_-
2009/4/15 J <bl...@gmx.com>:
> (Sorry for the empty message. First I tried Gmail, but that doesn't work
> with this mailing list. Then I tried GMX webclient, but that client always
> sends as HTML, which probably caused the message to be stripped to empty.)
>
>
> I'm experiencing freezes on a production website.
>
> Server specs:
> -OS: Linux CentOS
> -Webserver: Tomcat 6
> -MySQL 5 with default settings
> -Frameworks: Wicket 1.4-rc2 (webframework), Spring 2.5.6, Hibernate
> 3.3.1.GA, C3P0 db pooling 0.9.1.2, URLRewriteFilter 3.1.0, Spring
> OpenSessionInViewFilter
>
> Problem:
> At some point (about 5 to 24 hours) after a boot, Tomcat seems to stop
> serving requests, although it does create new threads for new incoming
> connections. But since it does not serve those connections, the new
> connections will show and keep showing 0kb transfer. Shortly after that,
> max-thread is reached, so then I'm unable to access any webapp (the main
> website + tomcat manager) running on that Tomcat server. I'm not sure if it
> is able to serve static resources in the short time window where it still
> has some threads free, because the time window is too short to notice when
> it happens.
>
> Observations:
> -No exceptions or errors in the catalina logs. So no memory problems, since
> no error occurs in the logs
> -Java Thread dump using command: kill -QUIT (see output below), shows the
> text "locked" and "WAITING".
> -Adding c3p0 db pooling idle checks, tests and timeout settings (see below)
> did not help.
> -Using Mysql Administrator GUI shows that after a freeze, there will be 15
> threads, all sleeping. Normally this would return to a minPoolSize of 5, if
> I'm correct, but thats not the case.
> -The freeze continues for hours, and does not recover. I have to restart
> Tomcat.
>
>
> I'm suspecting that it has something to do with DB pooling. I'm not sure if
> "locked" and "WAITING" are normal behaviour. But if there is something
> wrong, why doesn't c3p0 recover from it?
>
> Can somebody shed some light on this? :)
>
>
>
>
>
> Configuration:
>
> === DB pooling in Spring applicationContext.xml ====
> <bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource">
> <property
> name="driverClass"><value>com.mysql.jdbc.Driver</value></property>
> <property name="jdbcUrl" value="jdbc:mysql://localhost/xxxxxx" />
> <property name="user"><value>xxxxxxxx</value></property>
> <property name="password"><value>xxxxxxxxx</value></property>
>
> <property name="maxConnectionAge"><value>600</value></property>
> <property
> name="idleConnectionTestPeriod"><value>180</value></property>
> <property
> name="testConnectionOnCheckin"><value>true</value></property>
>
> <property name="acquireIncrement"><value>5</value></property>
> <property name="maxIdleTime"><value>180</value></property>
> <property name="maxPoolSize"><value>15</value></property>
> <property name="maxStatements"><value>100</value></property>
> <property name="minPoolSize"><value>5</value></property>
> </bean>
> ===========================
>
>
> ===== Thread dump part ===========
> "http-8080-28" daemon prio=10 tid=0xb4fbb400 nid=0x4ae2 in Object.wait()
> [0xb4316000..0xb4318130]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at
> com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePo
> ol.java:1315)
> at
> com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicRe
> sourcePool.java:557)
> - locked <0x72c74b10> (a
> com.mchange.v2.resourcepool.BasicResourcePool)
> at
> com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResource
> Pool.java:477)
> at
> com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C
> 3P0PooledConnectionPool.java:525)
> at
> com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(Abstract
> PoolBackedDataSource.java:128)
> at
> org.springframework.orm.hibernate3.LocalDataSourceConnectionProvider.getConn
> ection(LocalDataSourceConnectionProvider.java:81)
> at
> org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:4
> 23)
> at
> org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:14
> 4)
> at org.hibernate.jdbc.JDBCContext.connection(JDBCContext.java:119)
> at
> org.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:57)
> at
> org.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:1326)
> at
> org.springframework.orm.hibernate3.HibernateTransactionManager.doBegin(Hiber
> nateTransactionManager.java:510)
> at
> org.springframework.transaction.support.AbstractPlatformTransactionManager.g
> etTransaction(AbstractPlatformTransactionManager.java:350)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.createT
> ransactionIfNecessary(TransactionAspectSupport.java:262)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(Tr
> ansactionInterceptor.java:101)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(Reflect
> iveMethodInvocation.java:171)
> at
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopPro
> xy.java:204)
> at $Proxy37.findById(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
> .java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.wicket.proxy.LazyInitProxyFactory$JdkHandler.invoke(LazyInitProxy
> Factory.java:416)
> at org.apache.wicket.proxy.$Proxy2.findById(Unknown Source)
> at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> at
> org.apache.wicket.model.LoadableDetachableModel.getObject(LoadableDetachable
> Model.java:119)
> at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> at sun.reflect.GeneratedConstructorAccessor61.newInstance(Unknown
> Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstruc
> torAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java
> :151)
> at
> org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java
> :89)
> at
> org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.new
> Page(BookmarkablePageRequestTarget.java:306)
> at
> org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.get
> Page(BookmarkablePageRequestTarget.java:321)
> at
> org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.pro
> cessEvents(BookmarkablePageRequestTarget.java:234)
> at
> org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(Abstra
> ctRequestCycleProcessor.java:92)
> at
> org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:119
> 2)
> at org.apache.wicket.RequestCycle.step(RequestCycle.java:1271)
> at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1370)
> at org.apache.wicket.RequestCycle.request(RequestCycle.java:501)
> at
> org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:455)
> at
> org.apache.wicket.protocol.http.WicketFilter.doFilter(WicketFilter.java:288)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:206)
> at
> org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterI
> nternal(OpenSessionInViewFilter.java:198)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestF
> ilter.java:76)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:206)
> at
> org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:164
> )
> at
> org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:141)
> at
> org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.jav
> a:90)
> at
> org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter
> .java:417)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:206)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja
> va:233)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja
> va:191)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128
> )
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102
> )
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java
> :109)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
> at
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http
> 11Protocol.java:583)
> at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
> at java.lang.Thread.run(Thread.java:619)
> =======================================
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: WebApp Freezes
Posted by John Krasnay <jo...@krasnay.ca>.
Heh, I haven't had to worry about connection leaks since I discovered
Spring's JdbcTemplate and Hibernate, so it didn't even cross my mind. But
yeah, if the app is doing it's own JDBC connection management then that
is very much something J should look at.
jk
On Wed, Apr 15, 2009 at 05:58:11PM +0300, Serkan Camurcuoglu wrote:
> if all your threads are waiting for connections with no thread using a
> db connection, I suspect that you either leak connections (i.e. forget
> to free some connections) or each thread uses (requires) multiple
> connections during a transaction and at some point all threads need at
> least one more db connection to continue..
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: WebApp Freezes
Posted by nino martinez wael <ni...@gmail.com>.
I've some similar experiences with c3p0 (although only in test
environment), somewhere theres some extra settings about wait time and
pool size, you can google it.. Im in the midst of upgrading to ubuntu
so I do not have my source available..
You could also try to switch to a more maintained connection pool...
2009/4/15 Serkan Camurcuoglu <Se...@telenity.com>:
> if all your threads are waiting for connections with no thread using a db
> connection, I suspect that you either leak connections (i.e. forget to free
> some connections) or each thread uses (requires) multiple connections during
> a transaction and at some point all threads need at least one more db
> connection to continue..
>
>
>
> J wrote:
>>
>> Not just one thread, but that thread dump part (from my first post) is
>> repeated for 149 other threads have the same situation. ( because of
>> tomcats
>> 150 maxthreads).
>>
>> But when looking at the db during a freeze, there is no lock on any table
>> at
>> db level. MySQL Administrator shows that there are 15 threads during the
>> freeze (equal to c3p0's maxPoolSize), all sleeping.
>>
>> I have now added some C3P0 debug and workaround options to see if I can
>> get
>> some extra info out of it.
>>
>> http://www.mchange.com/projects/c3p0/index.html#configuring_to_debug_and_wor
>> karound_broken_clients
>> (debugUnreturnedConnectionStackTraces true and unreturnedConnectionTimeout
>> 180)
>>
>>
>> -----Oorspronkelijk bericht-----
>> Van: John Krasnay [mailto:john@krasnay.ca] Verzonden: woensdag 15 april
>> 2009 15:57
>> Aan: users@wicket.apache.org
>> Onderwerp: Re: WebApp Freezes
>>
>> Sounds like you have a thread holding a lock on a critical table and
>> subsequent threads are lining up behind it waiting for it to finish. You
>> should check your MySQL to try and figure out who's holding the lock and
>> why.
>>
>> Note that the culprit thread need not be hung up in the database. Locks
>> are held until the transaction commits, so your thread could be hung up
>> in app code after making a database update but before the transaction is
>> committed.
>>
>> jk
>>
>> On Wed, Apr 15, 2009 at 03:41:07PM +0200, J wrote:
>>
>>>
>>> (Sorry for the empty message. First I tried Gmail, but that doesn't work
>>> with this mailing list. Then I tried GMX webclient, but that client
>>> always
>>> sends as HTML, which probably caused the message to be stripped to
>>> empty.)
>>>
>>>
>>> I'm experiencing freezes on a production website.
>>>
>>> Server specs:
>>> -OS: Linux CentOS
>>> -Webserver: Tomcat 6
>>> -MySQL 5 with default settings
>>> -Frameworks: Wicket 1.4-rc2 (webframework), Spring 2.5.6, Hibernate
>>> 3.3.1.GA, C3P0 db pooling 0.9.1.2, URLRewriteFilter 3.1.0, Spring
>>> OpenSessionInViewFilter
>>> Problem:
>>> At some point (about 5 to 24 hours) after a boot, Tomcat seems to stop
>>> serving requests, although it does create new threads for new incoming
>>> connections. But since it does not serve those connections, the new
>>> connections will show and keep showing 0kb transfer. Shortly after that,
>>> max-thread is reached, so then I'm unable to access any webapp (the main
>>> website + tomcat manager) running on that Tomcat server. I'm not sure if
>>>
>>
>> it
>>
>>>
>>> is able to serve static resources in the short time window where it still
>>> has some threads free, because the time window is too short to notice
>>> when
>>> it happens.
>>>
>>> Observations:
>>> -No exceptions or errors in the catalina logs. So no memory problems,
>>>
>>
>> since
>>
>>>
>>> no error occurs in the logs
>>> -Java Thread dump using command: kill -QUIT (see output below), shows the
>>> text "locked" and "WAITING".
>>> -Adding c3p0 db pooling idle checks, tests and timeout settings (see
>>>
>>
>> below)
>>
>>>
>>> did not help.
>>> -Using Mysql Administrator GUI shows that after a freeze, there will be
>>> 15
>>> threads, all sleeping. Normally this would return to a minPoolSize of 5,
>>>
>>
>> if
>>
>>>
>>> I'm correct, but thats not the case.
>>> -The freeze continues for hours, and does not recover. I have to restart
>>> Tomcat.
>>>
>>>
>>> I'm suspecting that it has something to do with DB pooling. I'm not sure
>>>
>>
>> if
>>
>>>
>>> "locked" and "WAITING" are normal behaviour. But if there is something
>>> wrong, why doesn't c3p0 recover from it?
>>>
>>> Can somebody shed some light on this? :)
>>>
>>>
>>>
>>>
>>>
>>> Configuration:
>>>
>>> === DB pooling in Spring applicationContext.xml ====
>>> <bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource">
>>> <property
>>> name="driverClass"><value>com.mysql.jdbc.Driver</value></property>
>>> <property name="jdbcUrl" value="jdbc:mysql://localhost/xxxxxx" />
>>> <property name="user"><value>xxxxxxxx</value></property>
>>> <property name="password"><value>xxxxxxxxx</value></property>
>>>
>>> <property name="maxConnectionAge"><value>600</value></property>
>>> <property
>>> name="idleConnectionTestPeriod"><value>180</value></property>
>>> <property
>>> name="testConnectionOnCheckin"><value>true</value></property>
>>>
>>> <property name="acquireIncrement"><value>5</value></property>
>>> <property name="maxIdleTime"><value>180</value></property>
>>> <property name="maxPoolSize"><value>15</value></property>
>>> <property name="maxStatements"><value>100</value></property>
>>> <property name="minPoolSize"><value>5</value></property>
>>> </bean>
>>> ===========================
>>>
>>>
>>> ===== Thread dump part ===========
>>> "http-8080-28" daemon prio=10 tid=0xb4fbb400 nid=0x4ae2 in Object.wait()
>>> [0xb4316000..0xb4318130]
>>> java.lang.Thread.State: WAITING (on object monitor)
>>> at java.lang.Object.wait(Native Method)
>>> at
>>>
>>>
>>
>>
>> com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePo
>>
>>>
>>> ol.java:1315)
>>> at
>>>
>>>
>>
>>
>> com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicRe
>>
>>>
>>> sourcePool.java:557)
>>> - locked <0x72c74b10> (a
>>> com.mchange.v2.resourcepool.BasicResourcePool)
>>> at
>>>
>>>
>>
>>
>> com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResource
>>
>>>
>>> Pool.java:477)
>>> at
>>>
>>>
>>
>>
>> com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C
>>
>>>
>>> 3P0PooledConnectionPool.java:525)
>>> at
>>>
>>>
>>
>>
>> com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(Abstract
>>
>>>
>>> PoolBackedDataSource.java:128)
>>> at
>>>
>>>
>>
>>
>> org.springframework.orm.hibernate3.LocalDataSourceConnectionProvider.getConn
>>
>>>
>>> ection(LocalDataSourceConnectionProvider.java:81)
>>> at
>>>
>>>
>>
>>
>> org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:4
>>
>>>
>>> 23)
>>> at
>>>
>>>
>>
>>
>> org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:14
>>
>>>
>>> 4)
>>> at org.hibernate.jdbc.JDBCContext.connection(JDBCContext.java:119)
>>> at
>>> org.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:57)
>>> at
>>> org.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:1326)
>>> at
>>>
>>>
>>
>>
>> org.springframework.orm.hibernate3.HibernateTransactionManager.doBegin(Hiber
>>
>>>
>>> nateTransactionManager.java:510)
>>> at
>>>
>>>
>>
>>
>> org.springframework.transaction.support.AbstractPlatformTransactionManager.g
>>
>>>
>>> etTransaction(AbstractPlatformTransactionManager.java:350)
>>> at
>>>
>>>
>>
>>
>> org.springframework.transaction.interceptor.TransactionAspectSupport.createT
>>
>>>
>>> ransactionIfNecessary(TransactionAspectSupport.java:262)
>>> at
>>>
>>>
>>
>>
>> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(Tr
>>
>>>
>>> ansactionInterceptor.java:101)
>>> at
>>>
>>>
>>
>>
>> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(Reflect
>>
>>>
>>> iveMethodInvocation.java:171)
>>> at
>>>
>>>
>>
>>
>> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopPro
>>
>>>
>>> xy.java:204)
>>> at $Proxy37.findById(Unknown Source)
>>> at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
>>> at
>>>
>>>
>>
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
>>
>>>
>>> .java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at
>>>
>>>
>>
>>
>> org.apache.wicket.proxy.LazyInitProxyFactory$JdkHandler.invoke(LazyInitProxy
>>
>>>
>>> Factory.java:416)
>>> at org.apache.wicket.proxy.$Proxy2.findById(Unknown Source)
>>> at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>> at
>>>
>>>
>>
>>
>> org.apache.wicket.model.LoadableDetachableModel.getObject(LoadableDetachable
>>
>>>
>>> Model.java:119)
>>> at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
>>> at sun.reflect.GeneratedConstructorAccessor61.newInstance(Unknown
>>> Source)
>>> at
>>>
>>>
>>
>>
>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstruc
>>
>>>
>>> torAccessorImpl.java:27)
>>> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>>> at
>>>
>>>
>>
>>
>> org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java
>>
>>>
>>> :151)
>>> at
>>>
>>>
>>
>>
>> org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java
>>
>>>
>>> :89)
>>> at
>>>
>>>
>>
>>
>> org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.new
>>
>>>
>>> Page(BookmarkablePageRequestTarget.java:306)
>>> at
>>>
>>>
>>
>>
>> org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.get
>>
>>>
>>> Page(BookmarkablePageRequestTarget.java:321)
>>> at
>>>
>>>
>>
>>
>> org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.pro
>>
>>>
>>> cessEvents(BookmarkablePageRequestTarget.java:234)
>>> at
>>>
>>>
>>
>>
>> org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(Abstra
>>
>>>
>>> ctRequestCycleProcessor.java:92)
>>> at
>>>
>>>
>>
>>
>> org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:119
>>
>>>
>>> 2)
>>> at org.apache.wicket.RequestCycle.step(RequestCycle.java:1271)
>>> at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1370)
>>> at org.apache.wicket.RequestCycle.request(RequestCycle.java:501)
>>> at
>>> org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:455)
>>> at
>>>
>>>
>>
>>
>> org.apache.wicket.protocol.http.WicketFilter.doFilter(WicketFilter.java:288)
>>
>>>
>>> at
>>>
>>>
>>
>>
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
>>
>>>
>>> FilterChain.java:235)
>>> at
>>>
>>>
>>
>>
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
>>
>>>
>>> ain.java:206)
>>> at
>>>
>>>
>>
>>
>> org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterI
>>
>>>
>>> nternal(OpenSessionInViewFilter.java:198)
>>> at
>>>
>>>
>>
>>
>> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestF
>>
>>>
>>> ilter.java:76)
>>> at
>>>
>>>
>>
>>
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
>>
>>>
>>> FilterChain.java:235)
>>> at
>>>
>>>
>>
>>
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
>>
>>>
>>> ain.java:206)
>>> at
>>>
>>>
>>
>>
>> org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:164
>>
>>>
>>> )
>>> at
>>> org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:141)
>>> at
>>>
>>>
>>
>>
>> org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.jav
>>
>>>
>>> a:90)
>>> at
>>>
>>>
>>
>>
>> org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter
>>
>>>
>>> .java:417)
>>> at
>>>
>>>
>>
>>
>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
>>
>>>
>>> FilterChain.java:235)
>>> at
>>>
>>>
>>
>>
>> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
>>
>>>
>>> ain.java:206)
>>> at
>>>
>>>
>>
>>
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja
>>
>>>
>>> va:233)
>>> at
>>>
>>>
>>
>>
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja
>>
>>>
>>> va:191)
>>> at
>>>
>>>
>>
>>
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128
>>
>>>
>>> )
>>> at
>>>
>>>
>>
>>
>> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102
>>
>>>
>>> )
>>> at
>>>
>>>
>>
>>
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java
>>
>>>
>>> :109)
>>> at
>>>
>>>
>>
>>
>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
>>
>>>
>>> at
>>>
>>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
>>> at
>>>
>>>
>>
>>
>> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http
>>
>>>
>>> 11Protocol.java:583)
>>> at
>>> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
>>> at java.lang.Thread.run(Thread.java:619)
>>> =======================================
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
>>> For additional commands, e-mail: users-help@wicket.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
>> For additional commands, e-mail: users-help@wicket.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
>> For additional commands, e-mail: users-help@wicket.apache.org
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: WebApp Freezes
Posted by Serkan Camurcuoglu <Se...@telenity.com>.
if all your threads are waiting for connections with no thread using a
db connection, I suspect that you either leak connections (i.e. forget
to free some connections) or each thread uses (requires) multiple
connections during a transaction and at some point all threads need at
least one more db connection to continue..
J wrote:
> Not just one thread, but that thread dump part (from my first post) is
> repeated for 149 other threads have the same situation. ( because of tomcats
> 150 maxthreads).
>
> But when looking at the db during a freeze, there is no lock on any table at
> db level. MySQL Administrator shows that there are 15 threads during the
> freeze (equal to c3p0's maxPoolSize), all sleeping.
>
> I have now added some C3P0 debug and workaround options to see if I can get
> some extra info out of it.
> http://www.mchange.com/projects/c3p0/index.html#configuring_to_debug_and_wor
> karound_broken_clients
> (debugUnreturnedConnectionStackTraces true and unreturnedConnectionTimeout
> 180)
>
>
> -----Oorspronkelijk bericht-----
> Van: John Krasnay [mailto:john@krasnay.ca]
> Verzonden: woensdag 15 april 2009 15:57
> Aan: users@wicket.apache.org
> Onderwerp: Re: WebApp Freezes
>
> Sounds like you have a thread holding a lock on a critical table and
> subsequent threads are lining up behind it waiting for it to finish. You
> should check your MySQL to try and figure out who's holding the lock and
> why.
>
> Note that the culprit thread need not be hung up in the database. Locks
> are held until the transaction commits, so your thread could be hung up
> in app code after making a database update but before the transaction is
> committed.
>
> jk
>
> On Wed, Apr 15, 2009 at 03:41:07PM +0200, J wrote:
>
>> (Sorry for the empty message. First I tried Gmail, but that doesn't work
>> with this mailing list. Then I tried GMX webclient, but that client always
>> sends as HTML, which probably caused the message to be stripped to empty.)
>>
>>
>> I'm experiencing freezes on a production website.
>>
>> Server specs:
>> -OS: Linux CentOS
>> -Webserver: Tomcat 6
>> -MySQL 5 with default settings
>> -Frameworks: Wicket 1.4-rc2 (webframework), Spring 2.5.6, Hibernate
>> 3.3.1.GA, C3P0 db pooling 0.9.1.2, URLRewriteFilter 3.1.0, Spring
>> OpenSessionInViewFilter
>>
>> Problem:
>> At some point (about 5 to 24 hours) after a boot, Tomcat seems to stop
>> serving requests, although it does create new threads for new incoming
>> connections. But since it does not serve those connections, the new
>> connections will show and keep showing 0kb transfer. Shortly after that,
>> max-thread is reached, so then I'm unable to access any webapp (the main
>> website + tomcat manager) running on that Tomcat server. I'm not sure if
>>
> it
>
>> is able to serve static resources in the short time window where it still
>> has some threads free, because the time window is too short to notice when
>> it happens.
>>
>> Observations:
>> -No exceptions or errors in the catalina logs. So no memory problems,
>>
> since
>
>> no error occurs in the logs
>> -Java Thread dump using command: kill -QUIT (see output below), shows the
>> text "locked" and "WAITING".
>> -Adding c3p0 db pooling idle checks, tests and timeout settings (see
>>
> below)
>
>> did not help.
>> -Using Mysql Administrator GUI shows that after a freeze, there will be 15
>> threads, all sleeping. Normally this would return to a minPoolSize of 5,
>>
> if
>
>> I'm correct, but thats not the case.
>> -The freeze continues for hours, and does not recover. I have to restart
>> Tomcat.
>>
>>
>> I'm suspecting that it has something to do with DB pooling. I'm not sure
>>
> if
>
>> "locked" and "WAITING" are normal behaviour. But if there is something
>> wrong, why doesn't c3p0 recover from it?
>>
>> Can somebody shed some light on this? :)
>>
>>
>>
>>
>>
>> Configuration:
>>
>> === DB pooling in Spring applicationContext.xml ====
>> <bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource">
>> <property
>> name="driverClass"><value>com.mysql.jdbc.Driver</value></property>
>> <property name="jdbcUrl" value="jdbc:mysql://localhost/xxxxxx" />
>> <property name="user"><value>xxxxxxxx</value></property>
>> <property name="password"><value>xxxxxxxxx</value></property>
>>
>> <property name="maxConnectionAge"><value>600</value></property>
>> <property
>> name="idleConnectionTestPeriod"><value>180</value></property>
>> <property
>> name="testConnectionOnCheckin"><value>true</value></property>
>>
>> <property name="acquireIncrement"><value>5</value></property>
>> <property name="maxIdleTime"><value>180</value></property>
>> <property name="maxPoolSize"><value>15</value></property>
>> <property name="maxStatements"><value>100</value></property>
>> <property name="minPoolSize"><value>5</value></property>
>> </bean>
>> ===========================
>>
>>
>> ===== Thread dump part ===========
>> "http-8080-28" daemon prio=10 tid=0xb4fbb400 nid=0x4ae2 in Object.wait()
>> [0xb4316000..0xb4318130]
>> java.lang.Thread.State: WAITING (on object monitor)
>> at java.lang.Object.wait(Native Method)
>> at
>>
>>
> com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePo
>
>> ol.java:1315)
>> at
>>
>>
> com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicRe
>
>> sourcePool.java:557)
>> - locked <0x72c74b10> (a
>> com.mchange.v2.resourcepool.BasicResourcePool)
>> at
>>
>>
> com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResource
>
>> Pool.java:477)
>> at
>>
>>
> com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C
>
>> 3P0PooledConnectionPool.java:525)
>> at
>>
>>
> com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(Abstract
>
>> PoolBackedDataSource.java:128)
>> at
>>
>>
> org.springframework.orm.hibernate3.LocalDataSourceConnectionProvider.getConn
>
>> ection(LocalDataSourceConnectionProvider.java:81)
>> at
>>
>>
> org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:4
>
>> 23)
>> at
>>
>>
> org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:14
>
>> 4)
>> at org.hibernate.jdbc.JDBCContext.connection(JDBCContext.java:119)
>> at
>> org.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:57)
>> at
>> org.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:1326)
>> at
>>
>>
> org.springframework.orm.hibernate3.HibernateTransactionManager.doBegin(Hiber
>
>> nateTransactionManager.java:510)
>> at
>>
>>
> org.springframework.transaction.support.AbstractPlatformTransactionManager.g
>
>> etTransaction(AbstractPlatformTransactionManager.java:350)
>> at
>>
>>
> org.springframework.transaction.interceptor.TransactionAspectSupport.createT
>
>> ransactionIfNecessary(TransactionAspectSupport.java:262)
>> at
>>
>>
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(Tr
>
>> ansactionInterceptor.java:101)
>> at
>>
>>
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(Reflect
>
>> iveMethodInvocation.java:171)
>> at
>>
>>
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopPro
>
>> xy.java:204)
>> at $Proxy37.findById(Unknown Source)
>> at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
>> at
>>
>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
>
>> .java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>>
>>
> org.apache.wicket.proxy.LazyInitProxyFactory$JdkHandler.invoke(LazyInitProxy
>
>> Factory.java:416)
>> at org.apache.wicket.proxy.$Proxy2.findById(Unknown Source)
>> at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
>> at
>>
>>
> org.apache.wicket.model.LoadableDetachableModel.getObject(LoadableDetachable
>
>> Model.java:119)
>> at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
>> at sun.reflect.GeneratedConstructorAccessor61.newInstance(Unknown
>> Source)
>> at
>>
>>
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstruc
>
>> torAccessorImpl.java:27)
>> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>> at
>>
>>
> org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java
>
>> :151)
>> at
>>
>>
> org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java
>
>> :89)
>> at
>>
>>
> org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.new
>
>> Page(BookmarkablePageRequestTarget.java:306)
>> at
>>
>>
> org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.get
>
>> Page(BookmarkablePageRequestTarget.java:321)
>> at
>>
>>
> org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.pro
>
>> cessEvents(BookmarkablePageRequestTarget.java:234)
>> at
>>
>>
> org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(Abstra
>
>> ctRequestCycleProcessor.java:92)
>> at
>>
>>
> org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:119
>
>> 2)
>> at org.apache.wicket.RequestCycle.step(RequestCycle.java:1271)
>> at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1370)
>> at org.apache.wicket.RequestCycle.request(RequestCycle.java:501)
>> at
>> org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:455)
>> at
>>
>>
> org.apache.wicket.protocol.http.WicketFilter.doFilter(WicketFilter.java:288)
>
>> at
>>
>>
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
>
>> FilterChain.java:235)
>> at
>>
>>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
>
>> ain.java:206)
>> at
>>
>>
> org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterI
>
>> nternal(OpenSessionInViewFilter.java:198)
>> at
>>
>>
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestF
>
>> ilter.java:76)
>> at
>>
>>
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
>
>> FilterChain.java:235)
>> at
>>
>>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
>
>> ain.java:206)
>> at
>>
>>
> org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:164
>
>> )
>> at
>> org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:141)
>> at
>>
>>
> org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.jav
>
>> a:90)
>> at
>>
>>
> org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter
>
>> .java:417)
>> at
>>
>>
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
>
>> FilterChain.java:235)
>> at
>>
>>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
>
>> ain.java:206)
>> at
>>
>>
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja
>
>> va:233)
>> at
>>
>>
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja
>
>> va:191)
>> at
>>
>>
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128
>
>> )
>> at
>>
>>
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102
>
>> )
>> at
>>
>>
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java
>
>> :109)
>> at
>>
>>
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
>
>> at
>> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
>> at
>>
>>
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http
>
>> 11Protocol.java:583)
>> at
>> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
>> at java.lang.Thread.run(Thread.java:619)
>> =======================================
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
>> For additional commands, e-mail: users-help@wicket.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
RE: WebApp Freezes
Posted by J <bl...@gmx.com>.
Not just one thread, but that thread dump part (from my first post) is
repeated for 149 other threads have the same situation. ( because of tomcats
150 maxthreads).
But when looking at the db during a freeze, there is no lock on any table at
db level. MySQL Administrator shows that there are 15 threads during the
freeze (equal to c3p0's maxPoolSize), all sleeping.
I have now added some C3P0 debug and workaround options to see if I can get
some extra info out of it.
http://www.mchange.com/projects/c3p0/index.html#configuring_to_debug_and_wor
karound_broken_clients
(debugUnreturnedConnectionStackTraces true and unreturnedConnectionTimeout
180)
-----Oorspronkelijk bericht-----
Van: John Krasnay [mailto:john@krasnay.ca]
Verzonden: woensdag 15 april 2009 15:57
Aan: users@wicket.apache.org
Onderwerp: Re: WebApp Freezes
Sounds like you have a thread holding a lock on a critical table and
subsequent threads are lining up behind it waiting for it to finish. You
should check your MySQL to try and figure out who's holding the lock and
why.
Note that the culprit thread need not be hung up in the database. Locks
are held until the transaction commits, so your thread could be hung up
in app code after making a database update but before the transaction is
committed.
jk
On Wed, Apr 15, 2009 at 03:41:07PM +0200, J wrote:
> (Sorry for the empty message. First I tried Gmail, but that doesn't work
> with this mailing list. Then I tried GMX webclient, but that client always
> sends as HTML, which probably caused the message to be stripped to empty.)
>
>
> I'm experiencing freezes on a production website.
>
> Server specs:
> -OS: Linux CentOS
> -Webserver: Tomcat 6
> -MySQL 5 with default settings
> -Frameworks: Wicket 1.4-rc2 (webframework), Spring 2.5.6, Hibernate
> 3.3.1.GA, C3P0 db pooling 0.9.1.2, URLRewriteFilter 3.1.0, Spring
> OpenSessionInViewFilter
>
> Problem:
> At some point (about 5 to 24 hours) after a boot, Tomcat seems to stop
> serving requests, although it does create new threads for new incoming
> connections. But since it does not serve those connections, the new
> connections will show and keep showing 0kb transfer. Shortly after that,
> max-thread is reached, so then I'm unable to access any webapp (the main
> website + tomcat manager) running on that Tomcat server. I'm not sure if
it
> is able to serve static resources in the short time window where it still
> has some threads free, because the time window is too short to notice when
> it happens.
>
> Observations:
> -No exceptions or errors in the catalina logs. So no memory problems,
since
> no error occurs in the logs
> -Java Thread dump using command: kill -QUIT (see output below), shows the
> text "locked" and "WAITING".
> -Adding c3p0 db pooling idle checks, tests and timeout settings (see
below)
> did not help.
> -Using Mysql Administrator GUI shows that after a freeze, there will be 15
> threads, all sleeping. Normally this would return to a minPoolSize of 5,
if
> I'm correct, but thats not the case.
> -The freeze continues for hours, and does not recover. I have to restart
> Tomcat.
>
>
> I'm suspecting that it has something to do with DB pooling. I'm not sure
if
> "locked" and "WAITING" are normal behaviour. But if there is something
> wrong, why doesn't c3p0 recover from it?
>
> Can somebody shed some light on this? :)
>
>
>
>
>
> Configuration:
>
> === DB pooling in Spring applicationContext.xml ====
> <bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource">
> <property
> name="driverClass"><value>com.mysql.jdbc.Driver</value></property>
> <property name="jdbcUrl" value="jdbc:mysql://localhost/xxxxxx" />
> <property name="user"><value>xxxxxxxx</value></property>
> <property name="password"><value>xxxxxxxxx</value></property>
>
> <property name="maxConnectionAge"><value>600</value></property>
> <property
> name="idleConnectionTestPeriod"><value>180</value></property>
> <property
> name="testConnectionOnCheckin"><value>true</value></property>
>
> <property name="acquireIncrement"><value>5</value></property>
> <property name="maxIdleTime"><value>180</value></property>
> <property name="maxPoolSize"><value>15</value></property>
> <property name="maxStatements"><value>100</value></property>
> <property name="minPoolSize"><value>5</value></property>
> </bean>
> ===========================
>
>
> ===== Thread dump part ===========
> "http-8080-28" daemon prio=10 tid=0xb4fbb400 nid=0x4ae2 in Object.wait()
> [0xb4316000..0xb4318130]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at
>
com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePo
> ol.java:1315)
> at
>
com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicRe
> sourcePool.java:557)
> - locked <0x72c74b10> (a
> com.mchange.v2.resourcepool.BasicResourcePool)
> at
>
com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResource
> Pool.java:477)
> at
>
com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C
> 3P0PooledConnectionPool.java:525)
> at
>
com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(Abstract
> PoolBackedDataSource.java:128)
> at
>
org.springframework.orm.hibernate3.LocalDataSourceConnectionProvider.getConn
> ection(LocalDataSourceConnectionProvider.java:81)
> at
>
org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:4
> 23)
> at
>
org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:14
> 4)
> at org.hibernate.jdbc.JDBCContext.connection(JDBCContext.java:119)
> at
> org.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:57)
> at
> org.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:1326)
> at
>
org.springframework.orm.hibernate3.HibernateTransactionManager.doBegin(Hiber
> nateTransactionManager.java:510)
> at
>
org.springframework.transaction.support.AbstractPlatformTransactionManager.g
> etTransaction(AbstractPlatformTransactionManager.java:350)
> at
>
org.springframework.transaction.interceptor.TransactionAspectSupport.createT
> ransactionIfNecessary(TransactionAspectSupport.java:262)
> at
>
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(Tr
> ansactionInterceptor.java:101)
> at
>
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(Reflect
> iveMethodInvocation.java:171)
> at
>
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopPro
> xy.java:204)
> at $Proxy37.findById(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
> at
>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
> .java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
>
org.apache.wicket.proxy.LazyInitProxyFactory$JdkHandler.invoke(LazyInitProxy
> Factory.java:416)
> at org.apache.wicket.proxy.$Proxy2.findById(Unknown Source)
> at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> at
>
org.apache.wicket.model.LoadableDetachableModel.getObject(LoadableDetachable
> Model.java:119)
> at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> at sun.reflect.GeneratedConstructorAccessor61.newInstance(Unknown
> Source)
> at
>
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstruc
> torAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
>
org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java
> :151)
> at
>
org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java
> :89)
> at
>
org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.new
> Page(BookmarkablePageRequestTarget.java:306)
> at
>
org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.get
> Page(BookmarkablePageRequestTarget.java:321)
> at
>
org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.pro
> cessEvents(BookmarkablePageRequestTarget.java:234)
> at
>
org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(Abstra
> ctRequestCycleProcessor.java:92)
> at
>
org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:119
> 2)
> at org.apache.wicket.RequestCycle.step(RequestCycle.java:1271)
> at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1370)
> at org.apache.wicket.RequestCycle.request(RequestCycle.java:501)
> at
> org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:455)
> at
>
org.apache.wicket.protocol.http.WicketFilter.doFilter(WicketFilter.java:288)
> at
>
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:235)
> at
>
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:206)
> at
>
org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterI
> nternal(OpenSessionInViewFilter.java:198)
> at
>
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestF
> ilter.java:76)
> at
>
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:235)
> at
>
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:206)
> at
>
org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:164
> )
> at
> org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:141)
> at
>
org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.jav
> a:90)
> at
>
org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter
> .java:417)
> at
>
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:235)
> at
>
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:206)
> at
>
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja
> va:233)
> at
>
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja
> va:191)
> at
>
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128
> )
> at
>
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102
> )
> at
>
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java
> :109)
> at
>
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
> at
>
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http
> 11Protocol.java:583)
> at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
> at java.lang.Thread.run(Thread.java:619)
> =======================================
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: WebApp Freezes
Posted by John Krasnay <jo...@krasnay.ca>.
Sounds like you have a thread holding a lock on a critical table and
subsequent threads are lining up behind it waiting for it to finish. You
should check your MySQL to try and figure out who's holding the lock and
why.
Note that the culprit thread need not be hung up in the database. Locks
are held until the transaction commits, so your thread could be hung up
in app code after making a database update but before the transaction is
committed.
jk
On Wed, Apr 15, 2009 at 03:41:07PM +0200, J wrote:
> (Sorry for the empty message. First I tried Gmail, but that doesn't work
> with this mailing list. Then I tried GMX webclient, but that client always
> sends as HTML, which probably caused the message to be stripped to empty.)
>
>
> I'm experiencing freezes on a production website.
>
> Server specs:
> -OS: Linux CentOS
> -Webserver: Tomcat 6
> -MySQL 5 with default settings
> -Frameworks: Wicket 1.4-rc2 (webframework), Spring 2.5.6, Hibernate
> 3.3.1.GA, C3P0 db pooling 0.9.1.2, URLRewriteFilter 3.1.0, Spring
> OpenSessionInViewFilter
>
> Problem:
> At some point (about 5 to 24 hours) after a boot, Tomcat seems to stop
> serving requests, although it does create new threads for new incoming
> connections. But since it does not serve those connections, the new
> connections will show and keep showing 0kb transfer. Shortly after that,
> max-thread is reached, so then I'm unable to access any webapp (the main
> website + tomcat manager) running on that Tomcat server. I'm not sure if it
> is able to serve static resources in the short time window where it still
> has some threads free, because the time window is too short to notice when
> it happens.
>
> Observations:
> -No exceptions or errors in the catalina logs. So no memory problems, since
> no error occurs in the logs
> -Java Thread dump using command: kill -QUIT (see output below), shows the
> text "locked" and "WAITING".
> -Adding c3p0 db pooling idle checks, tests and timeout settings (see below)
> did not help.
> -Using Mysql Administrator GUI shows that after a freeze, there will be 15
> threads, all sleeping. Normally this would return to a minPoolSize of 5, if
> I'm correct, but thats not the case.
> -The freeze continues for hours, and does not recover. I have to restart
> Tomcat.
>
>
> I'm suspecting that it has something to do with DB pooling. I'm not sure if
> "locked" and "WAITING" are normal behaviour. But if there is something
> wrong, why doesn't c3p0 recover from it?
>
> Can somebody shed some light on this? :)
>
>
>
>
>
> Configuration:
>
> === DB pooling in Spring applicationContext.xml ====
> <bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource">
> <property
> name="driverClass"><value>com.mysql.jdbc.Driver</value></property>
> <property name="jdbcUrl" value="jdbc:mysql://localhost/xxxxxx" />
> <property name="user"><value>xxxxxxxx</value></property>
> <property name="password"><value>xxxxxxxxx</value></property>
>
> <property name="maxConnectionAge"><value>600</value></property>
> <property
> name="idleConnectionTestPeriod"><value>180</value></property>
> <property
> name="testConnectionOnCheckin"><value>true</value></property>
>
> <property name="acquireIncrement"><value>5</value></property>
> <property name="maxIdleTime"><value>180</value></property>
> <property name="maxPoolSize"><value>15</value></property>
> <property name="maxStatements"><value>100</value></property>
> <property name="minPoolSize"><value>5</value></property>
> </bean>
> ===========================
>
>
> ===== Thread dump part ===========
> "http-8080-28" daemon prio=10 tid=0xb4fbb400 nid=0x4ae2 in Object.wait()
> [0xb4316000..0xb4318130]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at
> com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePo
> ol.java:1315)
> at
> com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicRe
> sourcePool.java:557)
> - locked <0x72c74b10> (a
> com.mchange.v2.resourcepool.BasicResourcePool)
> at
> com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResource
> Pool.java:477)
> at
> com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C
> 3P0PooledConnectionPool.java:525)
> at
> com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(Abstract
> PoolBackedDataSource.java:128)
> at
> org.springframework.orm.hibernate3.LocalDataSourceConnectionProvider.getConn
> ection(LocalDataSourceConnectionProvider.java:81)
> at
> org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:4
> 23)
> at
> org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:14
> 4)
> at org.hibernate.jdbc.JDBCContext.connection(JDBCContext.java:119)
> at
> org.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:57)
> at
> org.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:1326)
> at
> org.springframework.orm.hibernate3.HibernateTransactionManager.doBegin(Hiber
> nateTransactionManager.java:510)
> at
> org.springframework.transaction.support.AbstractPlatformTransactionManager.g
> etTransaction(AbstractPlatformTransactionManager.java:350)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.createT
> ransactionIfNecessary(TransactionAspectSupport.java:262)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(Tr
> ansactionInterceptor.java:101)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(Reflect
> iveMethodInvocation.java:171)
> at
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopPro
> xy.java:204)
> at $Proxy37.findById(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
> .java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.wicket.proxy.LazyInitProxyFactory$JdkHandler.invoke(LazyInitProxy
> Factory.java:416)
> at org.apache.wicket.proxy.$Proxy2.findById(Unknown Source)
> at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> at
> org.apache.wicket.model.LoadableDetachableModel.getObject(LoadableDetachable
> Model.java:119)
> at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> at sun.reflect.GeneratedConstructorAccessor61.newInstance(Unknown
> Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstruc
> torAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java
> :151)
> at
> org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java
> :89)
> at
> org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.new
> Page(BookmarkablePageRequestTarget.java:306)
> at
> org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.get
> Page(BookmarkablePageRequestTarget.java:321)
> at
> org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.pro
> cessEvents(BookmarkablePageRequestTarget.java:234)
> at
> org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(Abstra
> ctRequestCycleProcessor.java:92)
> at
> org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:119
> 2)
> at org.apache.wicket.RequestCycle.step(RequestCycle.java:1271)
> at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1370)
> at org.apache.wicket.RequestCycle.request(RequestCycle.java:501)
> at
> org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:455)
> at
> org.apache.wicket.protocol.http.WicketFilter.doFilter(WicketFilter.java:288)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:206)
> at
> org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterI
> nternal(OpenSessionInViewFilter.java:198)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestF
> ilter.java:76)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:206)
> at
> org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:164
> )
> at
> org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:141)
> at
> org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.jav
> a:90)
> at
> org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter
> .java:417)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:206)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja
> va:233)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja
> va:191)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128
> )
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102
> )
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java
> :109)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
> at
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http
> 11Protocol.java:583)
> at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
> at java.lang.Thread.run(Thread.java:619)
> =======================================
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: WebApp Freezes
Posted by Marcelo Morales <ma...@gmail.com>.
I've had a similar problem and it turns out I wasn't returning
connections to the pool when an exception arose. Every thread waits
for the pool to hand out a connection, and waits indefinitely.
On Wed, Apr 15, 2009 at 9:41 AM, J <bl...@gmx.com> wrote:
> (Sorry for the empty message. First I tried Gmail, but that doesn't work
> with this mailing list. Then I tried GMX webclient, but that client always
> sends as HTML, which probably caused the message to be stripped to empty.)
>
>
> I'm experiencing freezes on a production website.
>
> Server specs:
> -OS: Linux CentOS
> -Webserver: Tomcat 6
> -MySQL 5 with default settings
> -Frameworks: Wicket 1.4-rc2 (webframework), Spring 2.5.6, Hibernate
> 3.3.1.GA, C3P0 db pooling 0.9.1.2, URLRewriteFilter 3.1.0, Spring
> OpenSessionInViewFilter
>
> Problem:
> At some point (about 5 to 24 hours) after a boot, Tomcat seems to stop
> serving requests, although it does create new threads for new incoming
> connections. But since it does not serve those connections, the new
> connections will show and keep showing 0kb transfer. Shortly after that,
> max-thread is reached, so then I'm unable to access any webapp (the main
> website + tomcat manager) running on that Tomcat server. I'm not sure if it
> is able to serve static resources in the short time window where it still
> has some threads free, because the time window is too short to notice when
> it happens.
>
> Observations:
> -No exceptions or errors in the catalina logs. So no memory problems, since
> no error occurs in the logs
> -Java Thread dump using command: kill -QUIT (see output below), shows the
> text "locked" and "WAITING".
> -Adding c3p0 db pooling idle checks, tests and timeout settings (see below)
> did not help.
> -Using Mysql Administrator GUI shows that after a freeze, there will be 15
> threads, all sleeping. Normally this would return to a minPoolSize of 5, if
> I'm correct, but thats not the case.
> -The freeze continues for hours, and does not recover. I have to restart
> Tomcat.
>
>
> I'm suspecting that it has something to do with DB pooling. I'm not sure if
> "locked" and "WAITING" are normal behaviour. But if there is something
> wrong, why doesn't c3p0 recover from it?
>
> Can somebody shed some light on this? :)
>
>
>
>
>
> Configuration:
>
> === DB pooling in Spring applicationContext.xml ====
> <bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource">
> <property
> name="driverClass"><value>com.mysql.jdbc.Driver</value></property>
> <property name="jdbcUrl" value="jdbc:mysql://localhost/xxxxxx" />
> <property name="user"><value>xxxxxxxx</value></property>
> <property name="password"><value>xxxxxxxxx</value></property>
>
> <property name="maxConnectionAge"><value>600</value></property>
> <property
> name="idleConnectionTestPeriod"><value>180</value></property>
> <property
> name="testConnectionOnCheckin"><value>true</value></property>
>
> <property name="acquireIncrement"><value>5</value></property>
> <property name="maxIdleTime"><value>180</value></property>
> <property name="maxPoolSize"><value>15</value></property>
> <property name="maxStatements"><value>100</value></property>
> <property name="minPoolSize"><value>5</value></property>
> </bean>
> ===========================
>
>
> ===== Thread dump part ===========
> "http-8080-28" daemon prio=10 tid=0xb4fbb400 nid=0x4ae2 in Object.wait()
> [0xb4316000..0xb4318130]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at
> com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePo
> ol.java:1315)
> at
> com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicRe
> sourcePool.java:557)
> - locked <0x72c74b10> (a
> com.mchange.v2.resourcepool.BasicResourcePool)
> at
> com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResource
> Pool.java:477)
> at
> com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C
> 3P0PooledConnectionPool.java:525)
> at
> com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(Abstract
> PoolBackedDataSource.java:128)
> at
> org.springframework.orm.hibernate3.LocalDataSourceConnectionProvider.getConn
> ection(LocalDataSourceConnectionProvider.java:81)
> at
> org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:4
> 23)
> at
> org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:14
> 4)
> at org.hibernate.jdbc.JDBCContext.connection(JDBCContext.java:119)
> at
> org.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:57)
> at
> org.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:1326)
> at
> org.springframework.orm.hibernate3.HibernateTransactionManager.doBegin(Hiber
> nateTransactionManager.java:510)
> at
> org.springframework.transaction.support.AbstractPlatformTransactionManager.g
> etTransaction(AbstractPlatformTransactionManager.java:350)
> at
> org.springframework.transaction.interceptor.TransactionAspectSupport.createT
> ransactionIfNecessary(TransactionAspectSupport.java:262)
> at
> org.springframework.transaction.interceptor.TransactionInterceptor.invoke(Tr
> ansactionInterceptor.java:101)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(Reflect
> iveMethodInvocation.java:171)
> at
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopPro
> xy.java:204)
> at $Proxy37.findById(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
> .java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.wicket.proxy.LazyInitProxyFactory$JdkHandler.invoke(LazyInitProxy
> Factory.java:416)
> at org.apache.wicket.proxy.$Proxy2.findById(Unknown Source)
> at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> at
> org.apache.wicket.model.LoadableDetachableModel.getObject(LoadableDetachable
> Model.java:119)
> at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
> at sun.reflect.GeneratedConstructorAccessor61.newInstance(Unknown
> Source)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstruc
> torAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java
> :151)
> at
> org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java
> :89)
> at
> org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.new
> Page(BookmarkablePageRequestTarget.java:306)
> at
> org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.get
> Page(BookmarkablePageRequestTarget.java:321)
> at
> org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.pro
> cessEvents(BookmarkablePageRequestTarget.java:234)
> at
> org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(Abstra
> ctRequestCycleProcessor.java:92)
> at
> org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:119
> 2)
> at org.apache.wicket.RequestCycle.step(RequestCycle.java:1271)
> at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1370)
> at org.apache.wicket.RequestCycle.request(RequestCycle.java:501)
> at
> org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:455)
> at
> org.apache.wicket.protocol.http.WicketFilter.doFilter(WicketFilter.java:288)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:206)
> at
> org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterI
> nternal(OpenSessionInViewFilter.java:198)
> at
> org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestF
> ilter.java:76)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:206)
> at
> org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:164
> )
> at
> org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:141)
> at
> org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.jav
> a:90)
> at
> org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter
> .java:417)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:235)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:206)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja
> va:233)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja
> va:191)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128
> )
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102
> )
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java
> :109)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
> at
> org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http
> 11Protocol.java:583)
> at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
> at java.lang.Thread.run(Thread.java:619)
> =======================================
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
--
Marcelo Morales
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
RE: WebApp Freezes
Posted by J <bl...@gmx.com>.
(Sorry for the empty message. First I tried Gmail, but that doesn't work
with this mailing list. Then I tried GMX webclient, but that client always
sends as HTML, which probably caused the message to be stripped to empty.)
I'm experiencing freezes on a production website.
Server specs:
-OS: Linux CentOS
-Webserver: Tomcat 6
-MySQL 5 with default settings
-Frameworks: Wicket 1.4-rc2 (webframework), Spring 2.5.6, Hibernate
3.3.1.GA, C3P0 db pooling 0.9.1.2, URLRewriteFilter 3.1.0, Spring
OpenSessionInViewFilter
Problem:
At some point (about 5 to 24 hours) after a boot, Tomcat seems to stop
serving requests, although it does create new threads for new incoming
connections. But since it does not serve those connections, the new
connections will show and keep showing 0kb transfer. Shortly after that,
max-thread is reached, so then I'm unable to access any webapp (the main
website + tomcat manager) running on that Tomcat server. I'm not sure if it
is able to serve static resources in the short time window where it still
has some threads free, because the time window is too short to notice when
it happens.
Observations:
-No exceptions or errors in the catalina logs. So no memory problems, since
no error occurs in the logs
-Java Thread dump using command: kill -QUIT (see output below), shows the
text "locked" and "WAITING".
-Adding c3p0 db pooling idle checks, tests and timeout settings (see below)
did not help.
-Using Mysql Administrator GUI shows that after a freeze, there will be 15
threads, all sleeping. Normally this would return to a minPoolSize of 5, if
I'm correct, but thats not the case.
-The freeze continues for hours, and does not recover. I have to restart
Tomcat.
I'm suspecting that it has something to do with DB pooling. I'm not sure if
"locked" and "WAITING" are normal behaviour. But if there is something
wrong, why doesn't c3p0 recover from it?
Can somebody shed some light on this? :)
Configuration:
=== DB pooling in Spring applicationContext.xml ====
<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource">
<property
name="driverClass"><value>com.mysql.jdbc.Driver</value></property>
<property name="jdbcUrl" value="jdbc:mysql://localhost/xxxxxx" />
<property name="user"><value>xxxxxxxx</value></property>
<property name="password"><value>xxxxxxxxx</value></property>
<property name="maxConnectionAge"><value>600</value></property>
<property
name="idleConnectionTestPeriod"><value>180</value></property>
<property
name="testConnectionOnCheckin"><value>true</value></property>
<property name="acquireIncrement"><value>5</value></property>
<property name="maxIdleTime"><value>180</value></property>
<property name="maxPoolSize"><value>15</value></property>
<property name="maxStatements"><value>100</value></property>
<property name="minPoolSize"><value>5</value></property>
</bean>
===========================
===== Thread dump part ===========
"http-8080-28" daemon prio=10 tid=0xb4fbb400 nid=0x4ae2 in Object.wait()
[0xb4316000..0xb4318130]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at
com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePo
ol.java:1315)
at
com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicRe
sourcePool.java:557)
- locked <0x72c74b10> (a
com.mchange.v2.resourcepool.BasicResourcePool)
at
com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResource
Pool.java:477)
at
com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C
3P0PooledConnectionPool.java:525)
at
com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(Abstract
PoolBackedDataSource.java:128)
at
org.springframework.orm.hibernate3.LocalDataSourceConnectionProvider.getConn
ection(LocalDataSourceConnectionProvider.java:81)
at
org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:4
23)
at
org.hibernate.jdbc.ConnectionManager.getConnection(ConnectionManager.java:14
4)
at org.hibernate.jdbc.JDBCContext.connection(JDBCContext.java:119)
at
org.hibernate.transaction.JDBCTransaction.begin(JDBCTransaction.java:57)
at
org.hibernate.impl.SessionImpl.beginTransaction(SessionImpl.java:1326)
at
org.springframework.orm.hibernate3.HibernateTransactionManager.doBegin(Hiber
nateTransactionManager.java:510)
at
org.springframework.transaction.support.AbstractPlatformTransactionManager.g
etTransaction(AbstractPlatformTransactionManager.java:350)
at
org.springframework.transaction.interceptor.TransactionAspectSupport.createT
ransactionIfNecessary(TransactionAspectSupport.java:262)
at
org.springframework.transaction.interceptor.TransactionInterceptor.invoke(Tr
ansactionInterceptor.java:101)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(Reflect
iveMethodInvocation.java:171)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopPro
xy.java:204)
at $Proxy37.findById(Unknown Source)
at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.wicket.proxy.LazyInitProxyFactory$JdkHandler.invoke(LazyInitProxy
Factory.java:416)
at org.apache.wicket.proxy.$Proxy2.findById(Unknown Source)
at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
at
org.apache.wicket.model.LoadableDetachableModel.getObject(LoadableDetachable
Model.java:119)
at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
at sun.reflect.GeneratedConstructorAccessor61.newInstance(Unknown
Source)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstruc
torAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java
:151)
at
org.apache.wicket.session.DefaultPageFactory.newPage(DefaultPageFactory.java
:89)
at
org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.new
Page(BookmarkablePageRequestTarget.java:306)
at
org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.get
Page(BookmarkablePageRequestTarget.java:321)
at
org.apache.wicket.request.target.component.BookmarkablePageRequestTarget.pro
cessEvents(BookmarkablePageRequestTarget.java:234)
at
org.apache.wicket.request.AbstractRequestCycleProcessor.processEvents(Abstra
ctRequestCycleProcessor.java:92)
at
org.apache.wicket.RequestCycle.processEventsAndRespond(RequestCycle.java:119
2)
at org.apache.wicket.RequestCycle.step(RequestCycle.java:1271)
at org.apache.wicket.RequestCycle.steps(RequestCycle.java:1370)
at org.apache.wicket.RequestCycle.request(RequestCycle.java:501)
at
org.apache.wicket.protocol.http.WicketFilter.doGet(WicketFilter.java:455)
at
org.apache.wicket.protocol.http.WicketFilter.doFilter(WicketFilter.java:288)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:206)
at
org.springframework.orm.hibernate3.support.OpenSessionInViewFilter.doFilterI
nternal(OpenSessionInViewFilter.java:198)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestF
ilter.java:76)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:206)
at
org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:164
)
at
org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:141)
at
org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.jav
a:90)
at
org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter
.java:417)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja
va:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja
va:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128
)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102
)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java
:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http
11Protocol.java:583)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
at java.lang.Thread.run(Thread.java:619)
=======================================
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: WebApp Freezes
Posted by francesco dicarlo <ev...@gmail.com>.
Second XD
2009/4/15 francisco treacy <fr...@gmail.com>:
> how *cool* is that!
>
> 2009/4/15 J <bl...@gmx.com>:
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
> For additional commands, e-mail: users-help@wicket.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org
Re: WebApp Freezes
Posted by francisco treacy <fr...@gmail.com>.
how *cool* is that!
2009/4/15 J <bl...@gmx.com>:
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@wicket.apache.org
For additional commands, e-mail: users-help@wicket.apache.org