You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ojb-user@db.apache.org by Daniel Perry <d....@netcase.co.uk> on 2004/11/15 13:37:42 UTC
Bug identified (was RE: SQLExcpetion)
I believe that I have solved this problem, and that there is a bug in OJB.
I cannot be sure, but with the fix, the app has been running for 10 days now
with no reported errors.
It seems that in certain circumstances an old connection will be reused, and
it will not be validated (or checked for being alive).
The following conditions are required for the error to occur :
1. An object is cached (it will only happen with a long term cache - eg
ObjectCacheDefaultImpl)
2. The object has been cached for long enough for the database connection
that was used to load it has expired
3. The object was loaded from outside a transaction (or possibly also inside
a very long living transaction)
4. A collection proxy (not sure if it affects other proxies) is materialised
from within this
object
What seems to happen, is the connection which was used to load the object is
held onto, and when the collection is materialised it reuses this stale
connection and crashes out. It might also be caused if it has already been
materialised and refresh=true - not sure how that works though.
This bug will only affect people using OJB in accordance with the above
criteria. ObjectCachePerBrokerImpl shouldn't be affected. Also the bug
wont apply if using transactions - as when it is closed it drops the
connection.
The solution is in ConnectionManagerImpl.getConnection method. The code is
already there, but commented out:
>
> // if (con == null || !isAlive(con))
> // isAlive check, how do we react if false and we in tx?
> // create a new connection isn't the right way I think.
> if (con == null)
I think this code should be applied - or at the very least, should be
modified and applied if not inside a transaction:
if (con == null || (!isInLocalTransaction() && !isAlive(con)) )
Daniel.
> -----Original Message-----
> From: Daniel Perry [mailto:d.perry@netcase.co.uk]
> Sent: 02 November 2004 11:14
> To: OJB Users List
> Subject: RE: SQLExcpetion
>
>
> Once again our app has started getting these errors (see trace below). It
> seems to be happening after several days of operation.
>
> The problem always seems to happen when iterating over a proxied
> collection
> in a jsp (using jakarta JSTL c:forEach tag).
>
> See comments below:
>
> > Is the error always caused by the same class
> >
> > >>>>at
> > >>>>org.apache.taglibs.standard.tag.common.core.ForEachSupport.toForE
> > >>>
> > >>>achIterator
> >
> > or always caused when materialized a proxied collection?
>
> Yes it's caused by that same class, but not always from the same jsp.
>
> But it's also always when proxied collections are materialised.
>
> >
> > Does the error occur when OJB materialize a class with clob/blob or
> > binary/object field?
> >
>
> Dont think so.
>
> > > Is there a way for me to clear the connection pool?
> >
> > You can try
> > ConnectionFactoryFactory.createConnectionFactory().releaseAllResources()
> > CFF is a singleton. This call clear the used pool and close all
> > connections of the pool (connections in use will be ignored AFAIK)
> >
> > Or extend ConnectionFactoryPooledImpl and add your own
> monitoring methods.
>
>
> I've tried clearing the connection pool every hour, so the connections
> should be fine. I havnt tried clearing the cache...
>
> How are connections obtained when cached objects have their proxied
> collections materialised? (note no transactions are being used)
>
> I think an object is being loaded, and stuck in the cache. Days later the
> object is being used and is having a proxied collection materialised, and
> this is using the origional connection.
>
> From what i can see in the code, a collection proxy obtains a broker, the
> broker obtains a connection - but (and i might have got the code
> wrong here)
> it should reuse the broker it used when it was initally loaded,
> and then the
> broker reuses the conenction without checking it isnt closed :
> ConnectionManagerImpl.getConnection
>
> // if (con == null || !isAlive(con))
> // isAlive check, how do we react if false and we in tx?
> // create a new connection isn't the right way I think.
> if (con == null)
>
> This only checks if it has a connection, not if the connection is valid.
> Maybe i should reinstate the (con == null || !isAlive(con)) ???
>
> Could this be the cause of my problem? or am i way off track!
>
> Daniel.
>
> Stack trace:
>
> org.apache.jasper.JasperException:
> org.apache.ojb.broker.PersistenceBrokerException:
> org.apache.ojb.broker.PersistenceBrokerSQLException:
> java.sql.SQLException:
> No operations allowed after connection closed.
>
> Connection was closed due to the following exception:
>
> ** BEGIN NESTED EXCEPTION **
>
> java.sql.SQLException
> MESSAGE: Communication link failure: java.io.IOException,
> underlying cause:
> Unexpected end of input stream
>
> ** BEGIN NESTED EXCEPTION **
>
> java.io.IOException
> MESSAGE: Unexpected end of input stream
>
> STACKTRACE:
>
> java.io.IOException: Unexpected end of input stream
> at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:1455)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:1826)
> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1098)
> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1192)
> at com.mysql.jdbc.Connection.execSQL(Connection.java:2051)
> at
> com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1496)
> at
> org.apache.ojb.broker.accesslayer.JdbcAccessImpl.executeQuery(Unknown
> Source)
> at
> org.apache.ojb.broker.accesslayer.RsQueryObject.performQuery(Unkno
> wn Source)
> at org.apache.ojb.broker.accesslayer.RsIterator.<init>(Unknown
> Source)
> at
> org.apache.ojb.broker.core.RsIteratorFactoryImpl.createRsIterator(Unknown
> Source)
> at
> org.apache.ojb.broker.core.PersistenceBrokerImpl.getRsIteratorFrom
> Query(Unkn
> own Source)
> at
> org.apache.ojb.broker.core.PersistenceBrokerImpl.getIteratorFromQu
> ery(Unknow
> n Source)
> at
> org.apache.ojb.broker.core.QueryReferenceBroker.getCollectionByQue
> ry(Unknown
> Source)
> at
> org.apache.ojb.broker.core.QueryReferenceBroker.getCollectionByQue
> ry(Unknown
> Source)
> at
> org.apache.ojb.broker.core.PersistenceBrokerImpl.getCollectionByQu
> ery(Unknow
> n Source)
> at
> org.apache.ojb.broker.core.DelegatingPersistenceBroker.getCollecti
> onByQuery(
> Unknown Source)
> at
> org.apache.ojb.broker.core.DelegatingPersistenceBroker.getCollecti
> onByQuery(
> Unknown Source)
> at
> org.apache.ojb.broker.core.proxy.CollectionProxyDefaultImpl.loadDa
> ta(Unknown
> Source)
> at
> org.apache.ojb.broker.core.proxy.ListProxyDefaultImpl.loadData(Unknown
> Source)
> at
> org.apache.ojb.broker.core.proxy.CollectionProxyDefaultImpl.getDat
> a(Unknown
> Source)
> at
> org.apache.ojb.broker.core.proxy.CollectionProxyDefaultImpl.iterat
> or(Unknown
> Source)
> at
> org.apache.taglibs.standard.tag.common.core.ForEachSupport.toForEa
> chIterator
> (ForEachSupport.java:329)
> at
> org.apache.taglibs.standard.tag.common.core.ForEachSupport.support
> edTypeForE
> achIterator(ForEachSupport.java:205)
> at
> org.apache.taglibs.standard.tag.common.core.ForEachSupport.prepare
> (ForEachSu
> pport.java:137)
> at
> javax.servlet.jsp.jstl.core.LoopTagSupport.doStartTag(LoopTagSuppo
> rt.java:22
> 7)
> at
> org.apache.jsp.WEB_002dINF.jsps.manager.behavioursscreen.appraisal
> _jsp._jspx
> _meth_c_forEach_0(appraisal_jsp.java:294)
> at
> org.apache.jsp.WEB_002dINF.jsps.manager.behavioursscreen.appraisal
> _jsp._jspS
> ervice(appraisal_jsp.java:186)
> at
> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:94)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
> at
> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrap
> per.java:2
> 98)
> at
> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:292)
> at
> org.apache.jasper.servlet.JspServlet.service(JspServlet.java:236)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(A
> pplication
> FilterChain.java:237)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(Applicati
> onFilterCh
> ain.java:157)
> at
> org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationD
> ispatcher.
> java:703)
> at
> org.apache.catalina.core.ApplicationDispatcher.processRequest(Appl
> icationDis
> patcher.java:463)
> at
> org.apache.catalina.core.ApplicationDispatcher.doForward(Applicati
> onDispatch
> er.java:398)
> at
> org.apache.catalina.core.ApplicationDispatcher.forward(Application
> Dispatcher
> .java:312)
> at
> org.apache.struts.action.RequestProcessor.doForward(RequestProcess
> or.java:10
> 56)
> at
> org.apache.struts.action.RequestProcessor.processForwardConfig(Req
> uestProces
> sor.java:388)
> at
> org.apache.struts.action.RequestProcessor.process(RequestProcessor
> .java:231)
> at
> org.apache.struts.action.ActionServlet.process(ActionServlet.java:1158)
> at
> org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:397)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:697)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(A
> pplication
> FilterChain.java:237)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(Applicati
> onFilterCh
> ain.java:157)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapp
> erValve.ja
> va:214)
> at
> org.apache.catalina.core.StandardValveContext.invokeNext(StandardV
> alveContex
> t.java:104)
> at
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.
> java:520)
> at
> org.apache.catalina.core.StandardContextValve.invokeInternal(Stand
> ardContext
> Valve.java:198)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardConte
> xtValve.ja
> va:152)
> at
> org.apache.catalina.core.StandardValveContext.invokeNext(StandardV
> alveContex
> t.java:104)
> at
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.
> java:520)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValv
> e.java:137
> )
> at
> org.apache.catalina.core.StandardValveContext.invokeNext(StandardV
> alveContex
> t.java:104)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValv
> e.java:117
> )
> at
> org.apache.catalina.core.StandardValveContext.invokeNext(StandardV
> alveContex
> t.java:102)
> at
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.
> java:520)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngine
> Valve.java
> :109)
> at
> org.apache.catalina.core.StandardValveContext.invokeNext(StandardV
> alveContex
> t.java:104)
> at
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.
> java:520)
> at
> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)
> at
> org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)
> at
> org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:296)
> at
> org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:372)
> at
> org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:694)
> at
> org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket
> .java:626)
> at
> org.apache.jk.common.SocketConnection.runIt(ChannelSocket.java:807)
> at
> org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(Thre
> adPool.jav
> a:644)
> at java.lang.Thread.run(Thread.java:534)
>
>
> ** END NESTED EXCEPTION **
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
> For additional commands, e-mail: ojb-user-help@db.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
For additional commands, e-mail: ojb-user-help@db.apache.org
RE: Bug identified (was RE: SQLExcpetion)
Posted by Daniel Perry <d....@netcase.co.uk>.
No problem. I've not come accross many bugs (well serious ones) in OJB, but
it's nice to find a hole and hopefully prevent it causing problems for other
people. I'm just glad it's sorted now, and our clients app isnt crashing
anymore.
I agree with your comments on cleaning up a bit better. The code seems to
do better if transactions are used, but much of my code doesnt have any
requirement for them.
Thanks,
Daniel.
> -----Original Message-----
> From: Armin Waibel [mailto:arminw@apache.org]
> Sent: 16 November 2004 12:53
> To: OJB Users List
> Subject: Re: Bug identified (was RE: SQLExcpetion)
>
>
> Hi Daniel,
>
> many thanks for your detailed description and suggestion.
>
> >
> > The solution is in ConnectionManagerImpl.getConnection method. The
> code is
> > already there, but commented out:
> >
> >> // if (con == null || !isAlive(con))
> >> // isAlive check, how do we react if false and we in tx?
> >> // create a new connection isn't the right way I think.
> >> if (con == null)
> >
> >
> > I think this code should be applied - or at the very least, should be
> > modified and applied if not inside a transaction:
> >
> > if (con == null || (!isInLocalTransaction() && !isAlive(con)) )
> >
>
> I will check in your patch ASAP. Additionally I think we should release
> the closed connection instance and I will do a minor refactoring of
> CollectionProxyDefaultImpl to handle the used PB instance more strict.
>
>
> regards,
> Armin
>
>
> Daniel Perry wrote:
>
> > I believe that I have solved this problem, and that there is a
> bug in OJB.
> > I cannot be sure, but with the fix, the app has been running
> for 10 days now
> > with no reported errors.
> >
> > It seems that in certain circumstances an old connection will
> be reused, and
> > it will not be validated (or checked for being alive).
> >
> > The following conditions are required for the error to occur :
> > 1. An object is cached (it will only happen with a long term cache - eg
> > ObjectCacheDefaultImpl)
> > 2. The object has been cached for long enough for the database
> connection
> > that was used to load it has expired
> > 3. The object was loaded from outside a transaction (or
> possibly also inside
> > a very long living transaction)
> > 4. A collection proxy (not sure if it affects other proxies) is
> materialised
> > from within this
> > object
> >
> > What seems to happen, is the connection which was used to load
> the object is
> > held onto, and when the collection is materialised it reuses this stale
> > connection and crashes out. It might also be caused if it has
> already been
> > materialised and refresh=true - not sure how that works though.
> >
> > This bug will only affect people using OJB in accordance with the above
> > criteria. ObjectCachePerBrokerImpl shouldn't be affected. Also the bug
> > wont apply if using transactions - as when it is closed it drops the
> > connection.
> >
> > The solution is in ConnectionManagerImpl.getConnection method.
> The code is
> > already there, but commented out:
> >
> >> // if (con == null || !isAlive(con))
> >> // isAlive check, how do we react if false and we in tx?
> >> // create a new connection isn't the right way I think.
> >> if (con == null)
> >
> >
> > I think this code should be applied - or at the very least, should be
> > modified and applied if not inside a transaction:
> >
> > if (con == null || (!isInLocalTransaction() && !isAlive(con)) )
> >
> > Daniel.
> >
> >
> >>-----Original Message-----
> >>From: Daniel Perry [mailto:d.perry@netcase.co.uk]
> >>Sent: 02 November 2004 11:14
> >>To: OJB Users List
> >>Subject: RE: SQLExcpetion
> >>
> >>
> >>Once again our app has started getting these errors (see trace
> below). It
> >>seems to be happening after several days of operation.
> >>
> >>The problem always seems to happen when iterating over a proxied
> >>collection
> >>in a jsp (using jakarta JSTL c:forEach tag).
> >>
> >>See comments below:
> >>
> >>
> >>>Is the error always caused by the same class
> >>>
> >>> >>>>at
> >>> >>>>org.apache.taglibs.standard.tag.common.core.ForEachSupport.toForE
> >>> >>>
> >>> >>>achIterator
> >>>
> >>>or always caused when materialized a proxied collection?
> >>
> >>Yes it's caused by that same class, but not always from the same jsp.
> >>
> >>But it's also always when proxied collections are materialised.
> >>
> >>
> >>>Does the error occur when OJB materialize a class with clob/blob or
> >>>binary/object field?
> >>>
> >>
> >>Dont think so.
> >>
> >>
> >>>>Is there a way for me to clear the connection pool?
> >>>
> >>>You can try
> >>>ConnectionFactoryFactory.createConnectionFactory().releaseAllRe
> sources()
> >>>CFF is a singleton. This call clear the used pool and close all
> >>>connections of the pool (connections in use will be ignored AFAIK)
> >>>
> >>>Or extend ConnectionFactoryPooledImpl and add your own
> >>
> >>monitoring methods.
> >>
> >>
> >>I've tried clearing the connection pool every hour, so the connections
> >>should be fine. I havnt tried clearing the cache...
> >>
> >>How are connections obtained when cached objects have their proxied
> >>collections materialised? (note no transactions are being used)
> >>
> >>I think an object is being loaded, and stuck in the cache.
> Days later the
> >>object is being used and is having a proxied collection
> materialised, and
> >>this is using the origional connection.
> >>
> >>From what i can see in the code, a collection proxy obtains a
> broker, the
> >>broker obtains a connection - but (and i might have got the code
> >>wrong here)
> >>it should reuse the broker it used when it was initally loaded,
> >>and then the
> >>broker reuses the conenction without checking it isnt closed :
> >>ConnectionManagerImpl.getConnection
> >>
> >> // if (con == null || !isAlive(con))
> >> // isAlive check, how do we react if false and we in tx?
> >> // create a new connection isn't the right way I think.
> >> if (con == null)
> >>
> >>This only checks if it has a connection, not if the connection is valid.
> >>Maybe i should reinstate the (con == null || !isAlive(con)) ???
> >>
> >>Could this be the cause of my problem? or am i way off track!
> >>
> >>Daniel.
> >>
> >>Stack trace:
> >>
> >>org.apache.jasper.JasperException:
> >>org.apache.ojb.broker.PersistenceBrokerException:
> >>org.apache.ojb.broker.PersistenceBrokerSQLException:
> >>java.sql.SQLException:
> >>No operations allowed after connection closed.
> >>
> >>Connection was closed due to the following exception:
> >>
> >>** BEGIN NESTED EXCEPTION **
> >>
> >>java.sql.SQLException
> >>MESSAGE: Communication link failure: java.io.IOException,
> >>underlying cause:
> >>Unexpected end of input stream
> >>
> >>** BEGIN NESTED EXCEPTION **
> >>
> >>java.io.IOException
> >>MESSAGE: Unexpected end of input stream
> >>
> >>STACKTRACE:
> >>
> >>java.io.IOException: Unexpected end of input stream
> >> at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:1455)
> >> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:1826)
> >> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1098)
> >> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1192)
> >> at com.mysql.jdbc.Connection.execSQL(Connection.java:2051)
> >> at
> >>com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.
> java:1496)
> >> at
> >>org.apache.ojb.broker.accesslayer.JdbcAccessImpl.executeQuery(Unknown
> >>Source)
> >> at
> >>org.apache.ojb.broker.accesslayer.RsQueryObject.performQuery(Unkno
> >>wn Source)
> >> at org.apache.ojb.broker.accesslayer.RsIterator.<init>(Unknown
> >>Source)
> >> at
> >>org.apache.ojb.broker.core.RsIteratorFactoryImpl.createRsIterato
> r(Unknown
> >>Source)
> >> at
> >>org.apache.ojb.broker.core.PersistenceBrokerImpl.getRsIteratorFrom
> >>Query(Unkn
> >>own Source)
> >> at
> >>org.apache.ojb.broker.core.PersistenceBrokerImpl.getIteratorFromQu
> >>ery(Unknow
> >>n Source)
> >> at
> >>org.apache.ojb.broker.core.QueryReferenceBroker.getCollectionByQue
> >>ry(Unknown
> >>Source)
> >> at
> >>org.apache.ojb.broker.core.QueryReferenceBroker.getCollectionByQue
> >>ry(Unknown
> >>Source)
> >> at
> >>org.apache.ojb.broker.core.PersistenceBrokerImpl.getCollectionByQu
> >>ery(Unknow
> >>n Source)
> >> at
> >>org.apache.ojb.broker.core.DelegatingPersistenceBroker.getCollecti
> >>onByQuery(
> >>Unknown Source)
> >> at
> >>org.apache.ojb.broker.core.DelegatingPersistenceBroker.getCollecti
> >>onByQuery(
> >>Unknown Source)
> >> at
> >>org.apache.ojb.broker.core.proxy.CollectionProxyDefaultImpl.loadDa
> >>ta(Unknown
> >>Source)
> >> at
> >>org.apache.ojb.broker.core.proxy.ListProxyDefaultImpl.loadData(Unknown
> >>Source)
> >> at
> >>org.apache.ojb.broker.core.proxy.CollectionProxyDefaultImpl.getDat
> >>a(Unknown
> >>Source)
> >> at
> >>org.apache.ojb.broker.core.proxy.CollectionProxyDefaultImpl.iterat
> >>or(Unknown
> >>Source)
> >> at
> >>org.apache.taglibs.standard.tag.common.core.ForEachSupport.toForEa
> >>chIterator
> >>(ForEachSupport.java:329)
> >> at
> >>org.apache.taglibs.standard.tag.common.core.ForEachSupport.support
> >>edTypeForE
> >>achIterator(ForEachSupport.java:205)
> >> at
> >>org.apache.taglibs.standard.tag.common.core.ForEachSupport.prepare
> >>(ForEachSu
> >>pport.java:137)
> >> at
> >>javax.servlet.jsp.jstl.core.LoopTagSupport.doStartTag(LoopTagSuppo
> >>rt.java:22
> >>7)
> >> at
> >>org.apache.jsp.WEB_002dINF.jsps.manager.behavioursscreen.appraisal
> >>_jsp._jspx
> >>_meth_c_forEach_0(appraisal_jsp.java:294)
> >> at
> >>org.apache.jsp.WEB_002dINF.jsps.manager.behavioursscreen.appraisal
> >>_jsp._jspS
> >>ervice(appraisal_jsp.java:186)
> >> at
> >>org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:94)
> >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
> >> at
> >>org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrap
> >>per.java:2
> >>98)
> >> at
> >>org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:292)
> >> at
> >>org.apache.jasper.servlet.JspServlet.service(JspServlet.java:236)
> >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
> >> at
> >>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(A
> >>pplication
> >>FilterChain.java:237)
> >> at
> >>org.apache.catalina.core.ApplicationFilterChain.doFilter(Applicati
> >>onFilterCh
> >>ain.java:157)
> >> at
> >>org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationD
> >>ispatcher.
> >>java:703)
> >> at
> >>org.apache.catalina.core.ApplicationDispatcher.processRequest(Appl
> >>icationDis
> >>patcher.java:463)
> >> at
> >>org.apache.catalina.core.ApplicationDispatcher.doForward(Applicati
> >>onDispatch
> >>er.java:398)
> >> at
> >>org.apache.catalina.core.ApplicationDispatcher.forward(Application
> >>Dispatcher
> >>.java:312)
> >> at
> >>org.apache.struts.action.RequestProcessor.doForward(RequestProcess
> >>or.java:10
> >>56)
> >> at
> >>org.apache.struts.action.RequestProcessor.processForwardConfig(Req
> >>uestProces
> >>sor.java:388)
> >> at
> >>org.apache.struts.action.RequestProcessor.process(RequestProcessor
> >>.java:231)
> >> at
> >>org.apache.struts.action.ActionServlet.process(ActionServlet.java:1158)
> >> at
> >>org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:397)
> >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:697)
> >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
> >> at
> >>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(A
> >>pplication
> >>FilterChain.java:237)
> >> at
> >>org.apache.catalina.core.ApplicationFilterChain.doFilter(Applicati
> >>onFilterCh
> >>ain.java:157)
> >> at
> >>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapp
> >>erValve.ja
> >>va:214)
> >> at
> >>org.apache.catalina.core.StandardValveContext.invokeNext(StandardV
> >>alveContex
> >>t.java:104)
> >> at
> >>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.
> >>java:520)
> >> at
> >>org.apache.catalina.core.StandardContextValve.invokeInternal(Stand
> >>ardContext
> >>Valve.java:198)
> >> at
> >>org.apache.catalina.core.StandardContextValve.invoke(StandardConte
> >>xtValve.ja
> >>va:152)
> >> at
> >>org.apache.catalina.core.StandardValveContext.invokeNext(StandardV
> >>alveContex
> >>t.java:104)
> >> at
> >>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.
> >>java:520)
> >> at
> >>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValv
> >>e.java:137
> >>)
> >> at
> >>org.apache.catalina.core.StandardValveContext.invokeNext(StandardV
> >>alveContex
> >>t.java:104)
> >> at
> >>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValv
> >>e.java:117
> >>)
> >> at
> >>org.apache.catalina.core.StandardValveContext.invokeNext(StandardV
> >>alveContex
> >>t.java:102)
> >> at
> >>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.
> >>java:520)
> >> at
> >>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngine
> >>Valve.java
> >>:109)
> >> at
> >>org.apache.catalina.core.StandardValveContext.invokeNext(StandardV
> >>alveContex
> >>t.java:104)
> >> at
> >>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.
> >>java:520)
> >> at
> >>org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)
> >> at
> >>org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)
> >> at
> >>org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:296)
> >> at
> >>org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:372)
> >> at
> >>org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:694)
> >> at
> >>org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket
> >>.java:626)
> >> at
> >>org.apache.jk.common.SocketConnection.runIt(ChannelSocket.java:807)
> >> at
> >>org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(Thre
> >>adPool.jav
> >>a:644)
> >> at java.lang.Thread.run(Thread.java:534)
> >>
> >>
> >>** END NESTED EXCEPTION **
> >>
> >>
> >>
> >>
> >>
> >>
> >>---------------------------------------------------------------------
> >>To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
> >>For additional commands, e-mail: ojb-user-help@db.apache.org
> >>
> >>
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
> > For additional commands, e-mail: ojb-user-help@db.apache.org
> >
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
> For additional commands, e-mail: ojb-user-help@db.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
For additional commands, e-mail: ojb-user-help@db.apache.org
Re: Bug identified (was RE: SQLExcpetion)
Posted by Armin Waibel <ar...@apache.org>.
Hi Daniel,
many thanks for your detailed description and suggestion.
>
> The solution is in ConnectionManagerImpl.getConnection method. The
code is
> already there, but commented out:
>
>> // if (con == null || !isAlive(con))
>> // isAlive check, how do we react if false and we in tx?
>> // create a new connection isn't the right way I think.
>> if (con == null)
>
>
> I think this code should be applied - or at the very least, should be
> modified and applied if not inside a transaction:
>
> if (con == null || (!isInLocalTransaction() && !isAlive(con)) )
>
I will check in your patch ASAP. Additionally I think we should release
the closed connection instance and I will do a minor refactoring of
CollectionProxyDefaultImpl to handle the used PB instance more strict.
regards,
Armin
Daniel Perry wrote:
> I believe that I have solved this problem, and that there is a bug in OJB.
> I cannot be sure, but with the fix, the app has been running for 10 days now
> with no reported errors.
>
> It seems that in certain circumstances an old connection will be reused, and
> it will not be validated (or checked for being alive).
>
> The following conditions are required for the error to occur :
> 1. An object is cached (it will only happen with a long term cache - eg
> ObjectCacheDefaultImpl)
> 2. The object has been cached for long enough for the database connection
> that was used to load it has expired
> 3. The object was loaded from outside a transaction (or possibly also inside
> a very long living transaction)
> 4. A collection proxy (not sure if it affects other proxies) is materialised
> from within this
> object
>
> What seems to happen, is the connection which was used to load the object is
> held onto, and when the collection is materialised it reuses this stale
> connection and crashes out. It might also be caused if it has already been
> materialised and refresh=true - not sure how that works though.
>
> This bug will only affect people using OJB in accordance with the above
> criteria. ObjectCachePerBrokerImpl shouldn't be affected. Also the bug
> wont apply if using transactions - as when it is closed it drops the
> connection.
>
> The solution is in ConnectionManagerImpl.getConnection method. The code is
> already there, but commented out:
>
>> // if (con == null || !isAlive(con))
>> // isAlive check, how do we react if false and we in tx?
>> // create a new connection isn't the right way I think.
>> if (con == null)
>
>
> I think this code should be applied - or at the very least, should be
> modified and applied if not inside a transaction:
>
> if (con == null || (!isInLocalTransaction() && !isAlive(con)) )
>
> Daniel.
>
>
>>-----Original Message-----
>>From: Daniel Perry [mailto:d.perry@netcase.co.uk]
>>Sent: 02 November 2004 11:14
>>To: OJB Users List
>>Subject: RE: SQLExcpetion
>>
>>
>>Once again our app has started getting these errors (see trace below). It
>>seems to be happening after several days of operation.
>>
>>The problem always seems to happen when iterating over a proxied
>>collection
>>in a jsp (using jakarta JSTL c:forEach tag).
>>
>>See comments below:
>>
>>
>>>Is the error always caused by the same class
>>>
>>> >>>>at
>>> >>>>org.apache.taglibs.standard.tag.common.core.ForEachSupport.toForE
>>> >>>
>>> >>>achIterator
>>>
>>>or always caused when materialized a proxied collection?
>>
>>Yes it's caused by that same class, but not always from the same jsp.
>>
>>But it's also always when proxied collections are materialised.
>>
>>
>>>Does the error occur when OJB materialize a class with clob/blob or
>>>binary/object field?
>>>
>>
>>Dont think so.
>>
>>
>>>>Is there a way for me to clear the connection pool?
>>>
>>>You can try
>>>ConnectionFactoryFactory.createConnectionFactory().releaseAllResources()
>>>CFF is a singleton. This call clear the used pool and close all
>>>connections of the pool (connections in use will be ignored AFAIK)
>>>
>>>Or extend ConnectionFactoryPooledImpl and add your own
>>
>>monitoring methods.
>>
>>
>>I've tried clearing the connection pool every hour, so the connections
>>should be fine. I havnt tried clearing the cache...
>>
>>How are connections obtained when cached objects have their proxied
>>collections materialised? (note no transactions are being used)
>>
>>I think an object is being loaded, and stuck in the cache. Days later the
>>object is being used and is having a proxied collection materialised, and
>>this is using the origional connection.
>>
>>>From what i can see in the code, a collection proxy obtains a broker, the
>>broker obtains a connection - but (and i might have got the code
>>wrong here)
>>it should reuse the broker it used when it was initally loaded,
>>and then the
>>broker reuses the conenction without checking it isnt closed :
>>ConnectionManagerImpl.getConnection
>>
>> // if (con == null || !isAlive(con))
>> // isAlive check, how do we react if false and we in tx?
>> // create a new connection isn't the right way I think.
>> if (con == null)
>>
>>This only checks if it has a connection, not if the connection is valid.
>>Maybe i should reinstate the (con == null || !isAlive(con)) ???
>>
>>Could this be the cause of my problem? or am i way off track!
>>
>>Daniel.
>>
>>Stack trace:
>>
>>org.apache.jasper.JasperException:
>>org.apache.ojb.broker.PersistenceBrokerException:
>>org.apache.ojb.broker.PersistenceBrokerSQLException:
>>java.sql.SQLException:
>>No operations allowed after connection closed.
>>
>>Connection was closed due to the following exception:
>>
>>** BEGIN NESTED EXCEPTION **
>>
>>java.sql.SQLException
>>MESSAGE: Communication link failure: java.io.IOException,
>>underlying cause:
>>Unexpected end of input stream
>>
>>** BEGIN NESTED EXCEPTION **
>>
>>java.io.IOException
>>MESSAGE: Unexpected end of input stream
>>
>>STACKTRACE:
>>
>>java.io.IOException: Unexpected end of input stream
>> at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:1455)
>> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:1826)
>> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1098)
>> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1192)
>> at com.mysql.jdbc.Connection.execSQL(Connection.java:2051)
>> at
>>com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1496)
>> at
>>org.apache.ojb.broker.accesslayer.JdbcAccessImpl.executeQuery(Unknown
>>Source)
>> at
>>org.apache.ojb.broker.accesslayer.RsQueryObject.performQuery(Unkno
>>wn Source)
>> at org.apache.ojb.broker.accesslayer.RsIterator.<init>(Unknown
>>Source)
>> at
>>org.apache.ojb.broker.core.RsIteratorFactoryImpl.createRsIterator(Unknown
>>Source)
>> at
>>org.apache.ojb.broker.core.PersistenceBrokerImpl.getRsIteratorFrom
>>Query(Unkn
>>own Source)
>> at
>>org.apache.ojb.broker.core.PersistenceBrokerImpl.getIteratorFromQu
>>ery(Unknow
>>n Source)
>> at
>>org.apache.ojb.broker.core.QueryReferenceBroker.getCollectionByQue
>>ry(Unknown
>>Source)
>> at
>>org.apache.ojb.broker.core.QueryReferenceBroker.getCollectionByQue
>>ry(Unknown
>>Source)
>> at
>>org.apache.ojb.broker.core.PersistenceBrokerImpl.getCollectionByQu
>>ery(Unknow
>>n Source)
>> at
>>org.apache.ojb.broker.core.DelegatingPersistenceBroker.getCollecti
>>onByQuery(
>>Unknown Source)
>> at
>>org.apache.ojb.broker.core.DelegatingPersistenceBroker.getCollecti
>>onByQuery(
>>Unknown Source)
>> at
>>org.apache.ojb.broker.core.proxy.CollectionProxyDefaultImpl.loadDa
>>ta(Unknown
>>Source)
>> at
>>org.apache.ojb.broker.core.proxy.ListProxyDefaultImpl.loadData(Unknown
>>Source)
>> at
>>org.apache.ojb.broker.core.proxy.CollectionProxyDefaultImpl.getDat
>>a(Unknown
>>Source)
>> at
>>org.apache.ojb.broker.core.proxy.CollectionProxyDefaultImpl.iterat
>>or(Unknown
>>Source)
>> at
>>org.apache.taglibs.standard.tag.common.core.ForEachSupport.toForEa
>>chIterator
>>(ForEachSupport.java:329)
>> at
>>org.apache.taglibs.standard.tag.common.core.ForEachSupport.support
>>edTypeForE
>>achIterator(ForEachSupport.java:205)
>> at
>>org.apache.taglibs.standard.tag.common.core.ForEachSupport.prepare
>>(ForEachSu
>>pport.java:137)
>> at
>>javax.servlet.jsp.jstl.core.LoopTagSupport.doStartTag(LoopTagSuppo
>>rt.java:22
>>7)
>> at
>>org.apache.jsp.WEB_002dINF.jsps.manager.behavioursscreen.appraisal
>>_jsp._jspx
>>_meth_c_forEach_0(appraisal_jsp.java:294)
>> at
>>org.apache.jsp.WEB_002dINF.jsps.manager.behavioursscreen.appraisal
>>_jsp._jspS
>>ervice(appraisal_jsp.java:186)
>> at
>>org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:94)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
>> at
>>org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrap
>>per.java:2
>>98)
>> at
>>org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:292)
>> at
>>org.apache.jasper.servlet.JspServlet.service(JspServlet.java:236)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
>> at
>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(A
>>pplication
>>FilterChain.java:237)
>> at
>>org.apache.catalina.core.ApplicationFilterChain.doFilter(Applicati
>>onFilterCh
>>ain.java:157)
>> at
>>org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationD
>>ispatcher.
>>java:703)
>> at
>>org.apache.catalina.core.ApplicationDispatcher.processRequest(Appl
>>icationDis
>>patcher.java:463)
>> at
>>org.apache.catalina.core.ApplicationDispatcher.doForward(Applicati
>>onDispatch
>>er.java:398)
>> at
>>org.apache.catalina.core.ApplicationDispatcher.forward(Application
>>Dispatcher
>>.java:312)
>> at
>>org.apache.struts.action.RequestProcessor.doForward(RequestProcess
>>or.java:10
>>56)
>> at
>>org.apache.struts.action.RequestProcessor.processForwardConfig(Req
>>uestProces
>>sor.java:388)
>> at
>>org.apache.struts.action.RequestProcessor.process(RequestProcessor
>>.java:231)
>> at
>>org.apache.struts.action.ActionServlet.process(ActionServlet.java:1158)
>> at
>>org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:397)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:697)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:810)
>> at
>>org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(A
>>pplication
>>FilterChain.java:237)
>> at
>>org.apache.catalina.core.ApplicationFilterChain.doFilter(Applicati
>>onFilterCh
>>ain.java:157)
>> at
>>org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapp
>>erValve.ja
>>va:214)
>> at
>>org.apache.catalina.core.StandardValveContext.invokeNext(StandardV
>>alveContex
>>t.java:104)
>> at
>>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.
>>java:520)
>> at
>>org.apache.catalina.core.StandardContextValve.invokeInternal(Stand
>>ardContext
>>Valve.java:198)
>> at
>>org.apache.catalina.core.StandardContextValve.invoke(StandardConte
>>xtValve.ja
>>va:152)
>> at
>>org.apache.catalina.core.StandardValveContext.invokeNext(StandardV
>>alveContex
>>t.java:104)
>> at
>>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.
>>java:520)
>> at
>>org.apache.catalina.core.StandardHostValve.invoke(StandardHostValv
>>e.java:137
>>)
>> at
>>org.apache.catalina.core.StandardValveContext.invokeNext(StandardV
>>alveContex
>>t.java:104)
>> at
>>org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValv
>>e.java:117
>>)
>> at
>>org.apache.catalina.core.StandardValveContext.invokeNext(StandardV
>>alveContex
>>t.java:102)
>> at
>>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.
>>java:520)
>> at
>>org.apache.catalina.core.StandardEngineValve.invoke(StandardEngine
>>Valve.java
>>:109)
>> at
>>org.apache.catalina.core.StandardValveContext.invokeNext(StandardV
>>alveContex
>>t.java:104)
>> at
>>org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.
>>java:520)
>> at
>>org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:929)
>> at
>>org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:160)
>> at
>>org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:296)
>> at
>>org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:372)
>> at
>>org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:694)
>> at
>>org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket
>>.java:626)
>> at
>>org.apache.jk.common.SocketConnection.runIt(ChannelSocket.java:807)
>> at
>>org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(Thre
>>adPool.jav
>>a:644)
>> at java.lang.Thread.run(Thread.java:534)
>>
>>
>>** END NESTED EXCEPTION **
>>
>>
>>
>>
>>
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
>>For additional commands, e-mail: ojb-user-help@db.apache.org
>>
>>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
> For additional commands, e-mail: ojb-user-help@db.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
For additional commands, e-mail: ojb-user-help@db.apache.org