You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by "Dag H. Wanvik (JIRA)" <ji...@apache.org> on 2010/10/27 00:33:19 UTC

[jira] Issue Comment Edited: (DERBY-4129) jvm level Deadlock between 2 threads accessing the same connection

    [ https://issues.apache.org/jira/browse/DERBY-4129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12924986#action_12924986 ] 

Dag H. Wanvik edited comment on DERBY-4129 at 10/26/10 6:32 PM:
----------------------------------------------------------------

Saw similar code (acessing store without synchronizing on rootConnection as most JDBC API methods do to be thread safe): cf. this stack taken from RAFCOntainer4 (while investigating DERBY-4741):

Output trace from RAFContainer4:

   lcc=org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext@1e2befa

   java.lang.Exception: Stack trace
   at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(RAFContainer4.java:494)
   at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage0(RAFContainer4.java:244)
   at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(RAFContainer4.java:214)
   at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:671)
   at org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(CachedPage.java:190)
   at org.apache.derby.impl.services.cache.ConcurrentCache.find(ConcurrentCache.java:295)
   at org.apache.derby.impl.store.raw.data.FileContainer.getUserPage(FileContainer.java:2452)
   at org.apache.derby.impl.store.raw.data.FileContainer.getPage(FileContainer.java:2502)
   at org.apache.derby.impl.store.raw.data.BaseContainerHandle.getPage(BaseContainerHandle.java:319)
   at org.apache.derby.impl.store.raw.data.OverflowInputStream.fillByteHolder(OverflowInputStream.java:150)
   at org.apache.derby.impl.store.raw.data.BufferedByteHolderInputStream.read(BufferedByteHolderInputStream.java:37)
   at java.io.FilterInputStream.read(FilterInputStream.java:66)
   at org.apache.derby.impl.jdbc.PositionedStoreStream.read(PositionedStoreStream.java:131)
   at org.apache.derby.impl.jdbc.UTF8Reader.fillBuffer(UTF8Reader.java:415)
   at org.apache.derby.impl.jdbc.UTF8Reader.skip(UTF8Reader.java:232)
   at org.apache.derby.impl.jdbc.StoreStreamClob.getReader(StoreStreamClob.java:236)
   at org.apache.derby.impl.jdbc.ClobUpdatableReader.updateReaderIfRequired(ClobUpdatableReader.java:210)
   at org.apache.derby.impl.jdbc.ClobUpdatableReader.read(ClobUpdatableReader.java:120)
   at InterruptTestClob$WorkerThread.run(InterruptTestClob.java:171)

I don't believe this stack holds the monitor on rootConnection.... since none of the JDBC methods seem grab it. I note that UTF8Reader#fillBuffer *does* call EmbedConnection#setupContextStack, cf the fact that lcc is set when retrieved via getContext from RAFContainer4.

Additionally, the Clob code doesn't ever call ConnectionChild#handleException to clean up errors (close connection or shut down database) - EmbedClob does, at least in some cases.  In the above case, this meant that an error thrown from RAFContainer4 (FILE_IO_INTERRUPTED, database level), seen wrapped as an IOException by the Clob#read call, did not lead to Derby shutting down until the next operation against the Connection was initiated and hit a problem.. Is this the way it is intended to be? I think the CLOB code should both synchronize and handle errors as the rest of the JDBC code does?


      was (Author: dagw):
    Saw similar code (acessing store without synchronizing on rootConnection as most JDBC API methods do to be thread safe): cf. this stack taken from RAFCOntainer4 (while investigating DERBY-4741):

Output trace from RAFContainer4:

   lcc=org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext@1e2befa

   java.lang.Exception: Stack trace
   at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(RAFContainer4.java:494)
   at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage0(RAFContainer4.java:244)
   at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(RAFContainer4.java:214)
   at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:671)
   at org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(CachedPage.java:190)
   at org.apache.derby.impl.services.cache.ConcurrentCache.find(ConcurrentCache.java:295)
   at org.apache.derby.impl.store.raw.data.FileContainer.getUserPage(FileContainer.java:2452)
   at org.apache.derby.impl.store.raw.data.FileContainer.getPage(FileContainer.java:2502)
   at org.apache.derby.impl.store.raw.data.BaseContainerHandle.getPage(BaseContainerHandle.java:319)
   at org.apache.derby.impl.store.raw.data.OverflowInputStream.fillByteHolder(OverflowInputStream.java:150)
   at org.apache.derby.impl.store.raw.data.BufferedByteHolderInputStream.read(BufferedByteHolderInputStream.java:37)
   at java.io.FilterInputStream.read(FilterInputStream.java:66)
   at org.apache.derby.impl.jdbc.PositionedStoreStream.read(PositionedStoreStream.java:131)
   at org.apache.derby.impl.jdbc.UTF8Reader.fillBuffer(UTF8Reader.java:415)
   at org.apache.derby.impl.jdbc.UTF8Reader.skip(UTF8Reader.java:232)
   at org.apache.derby.impl.jdbc.StoreStreamClob.getReader(StoreStreamClob.java:236)
   at org.apache.derby.impl.jdbc.ClobUpdatableReader.updateReaderIfRequired(ClobUpdatableReader.java:210)
   at org.apache.derby.impl.jdbc.ClobUpdatableReader.read(ClobUpdatableReader.java:120)
   at InterruptTestClob$WorkerThread.run(InterruptTestClob.java:171)

I don't believe this stack holds the monitor on rootConnection.... since none of the JDBC methods seem grab it. I note that UTF8Reader#fillBuffer *does* call EmbedConnection#setupContextStack, cf the fact that lcc is set. 

Additionally, the Clob code doesn't ever call ConnectionChild#handleException to clean up errors (close connection or shut down database) - EmbedClob does, at least in some cases.  In the above case, this meant that an error thrown from RAFContainer4 (FILE_IO_INTERRUPTED, database level), seen wrapped as an IOException by the Clob#read call, did not lead to Derby shutting down until the next operation against the Connection was initiated and hit a problem.. Is this the way it is intended to be? I think the CLOB code should both synchronize and handle errors as the rest of the JDBC code does?

  
> jvm level Deadlock between 2 threads accessing the same connection
> ------------------------------------------------------------------
>
>                 Key: DERBY-4129
>                 URL: https://issues.apache.org/jira/browse/DERBY-4129
>             Project: Derby
>          Issue Type: Bug
>          Components: JDBC, SQL, Store
>    Affects Versions: 10.4.2.0
>         Environment: Linux 2.6.18-53.1.14.el5 #1 SMP Wed Mar 5 11:36:49 EST 2008 i686 i686 i386 GNU/Linux
>            Reporter: Malchenko Inna
>         Attachments: BlobDeadlock.java, threaddump.txt
>
>
> Threads working with embedded derby get deadlocked.
> Thread dump:
> Found one Java-level deadlock:
> =============================
> "http-8080-13":
>   waiting to lock monitor 0x09d1a37c (object 0x9560fcd8, a org.apache.derby.impl.store.raw.data.BaseContainerHandle),
>   which is held by "dwc.ma.MAJobsManager:dwc.ma.MAManager"
> "dwc.ma.MAJobsManager:dwc.ma.MAManager":
>   waiting to lock monitor 0x09d1a57c (object 0x93da0180, a org.apache.derby.impl.store.raw.data.StoredPage),
>   which is held by "http-8080-13"
> Java stack information for the threads listed above:
> ===================================================
> "http-8080-13":
> 	at java.util.Observable.deleteObserver(Observable.java:78)
> 	- waiting to lock <0x9560fcd8> (a org.apache.derby.impl.store.raw.data.BaseContainerHandle)
> 	at org.apache.derby.impl.store.raw.data.BasePage.releaseExclusive(Unknown Source)
> 	- locked <0x93da0180> (a org.apache.derby.impl.store.raw.data.StoredPage)
> 	at org.apache.derby.impl.store.raw.data.CachedPage.releaseExclusive(Unknown Source)
> 	at org.apache.derby.impl.store.raw.data.StoredPage.releaseExclusive(Unknown Source)
> 	at org.apache.derby.impl.store.raw.data.BasePage.unlatch(Unknown Source)
> 	at org.apache.derby.impl.store.raw.data.OverflowInputStream.fillByteHolder(Unknown Source)
> 	at org.apache.derby.impl.store.raw.data.BufferedByteHolderInputStream.read(Unknown Source)
> 	at java.io.DataInputStream.readFully(DataInputStream.java:176)
> 	at java.io.DataInputStream.readFully(DataInputStream.java:152)
> 	at org.apache.derby.iapi.types.SQLBinary.readExternal(Unknown Source)
> 	at org.apache.derby.iapi.types.SQLBinary.getValue(Unknown Source)
> 	at org.apache.derby.iapi.types.SQLBinary.getBytes(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedResultSet.getBytes(Unknown Source)
> 	at dwc.ma.MAOrderProcessor.loadData(MAOrderProcessor.java:147)
> 	- locked <0x91f211b0> (a java.lang.Object)
> 	- locked <0x91f6a068> (a dwc.ma.MAOrderProcessor)
> 	at dwc.ma.MAOrderProcessor.generateBundlesForUser(MAOrderProcessor.java:538)
> 	- locked <0x91f6a068> (a dwc.ma.MAOrderProcessor)
> 	at dwc.ma.MARequestProcessor$HBundleRequest.process(MARequestProcessor.java:239)
> 	at dwc.ma.MARequestProcessor.processRequest(MARequestProcessor.java:39)
> 	at dwc.ma.doProc.doPost(doProc.java:92)
> 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
> 	at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
> 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> 	at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:738)
> 	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> 	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> 	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> 	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> 	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
> 	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> 	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> 	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
> 	at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
> 	at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
> 	at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
> 	at java.lang.Thread.run(Thread.java:595)
> "dwc.ma.MAJobsManager:dwc.ma.MAManager":
> 	at org.apache.derby.impl.store.raw.data.BasePage.releaseExclusive(Unknown Source)
> 	- waiting to lock <0x93da0180> (a org.apache.derby.impl.store.raw.data.StoredPage)
> 	at org.apache.derby.impl.store.raw.data.CachedPage.releaseExclusive(Unknown Source)
> 	at org.apache.derby.impl.store.raw.data.StoredPage.releaseExclusive(Unknown Source)
> 	at org.apache.derby.impl.store.raw.data.BasePage.update(Unknown Source)
> 	at java.util.Observable.notifyObservers(Observable.java:142)
> 	at java.util.Observable.notifyObservers(Observable.java:98)
> 	at org.apache.derby.impl.store.raw.data.BaseContainerHandle.informObservers(Unknown Source)
> 	at org.apache.derby.impl.store.raw.data.BaseContainerHandle.close(Unknown Source)
> 	- locked <0x9560fcd8> (a org.apache.derby.impl.store.raw.data.BaseContainerHandle)
> 	at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.close(Unknown Source)
> 	at org.apache.derby.impl.store.access.conglomerate.GenericController.close(Unknown Source)
> 	at org.apache.derby.impl.store.access.conglomerate.GenericConglomerateController.closeForEndTransaction(Unknown Source)
> 	at org.apache.derby.impl.store.access.RAMTransaction.closeControllers(Unknown Source)
> 	at org.apache.derby.impl.store.access.RAMTransaction.commit(Unknown Source)
> 	at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.doCommit(Unknown Source)
> 	at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.userCommit(Unknown Source)
> 	at org.apache.derby.impl.jdbc.TransactionResourceImpl.commit(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedConnection.commitIfAutoCommit(Unknown Source)
> 	at org.apache.derby.impl.jdbc.ConnectionChild.commitIfAutoCommit(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedStatement.resultSetClosing(Unknown Source)
> 	at org.apache.derby.impl.jdbc.EmbedResultSet.close(Unknown Source)
> 	- locked <0x91f0b7d0> (a org.apache.derby.impl.jdbc.EmbedConnection30)
> 	at dwc.ma.MAStatsProcessor$UserStatsBundlesProc.run(MAStatsProcessor.java:117)
> 	at dwc.ma.MAJobsManager$Runner.run(MAJobsManager.java:84)
> 	- locked <0x91fce338> (a java.util.LinkedList)
> 	at java.lang.Thread.run(Thread.java:595)
> Found 1 deadlock.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.