You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by "Bergquist, Brett" <BB...@canoga.com> on 2011/12/21 15:14:44 UTC
Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
I have a problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1 but at this point I cannot tell if it is a Derby problem or a Glassfish problem so I am looking for some guidance.
A database pool is created using the "org.apache.derby.jdbc.ClientXADataSource" XADataSource. Connection validation is turned on and uses "auto-commit" to validate the connection and attempts to close all connections on any validation failure.
The deadlock is triggered when I have a couple of transactions that cause a Derby locks deadlock. Deadlock detection and tracing is turned on in Derby and it correctly detects the locking deadlock and terminates one of the transactions. Soon after this, however, Glassfish becomes stuck. Here is the only reference to Derby in the "jstack" output of Glassfish:
"PMCollectionActivationManager.newPmTestResult" daemon prio=3 tid=0x0000000103926800 nid=0xaf runnable [0xfffffffc218fa000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.client.net.Reply.fill(Unknown Source)
at org.apache.derby.client.net.Reply.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.client.net.Reply.readDssHeader(Unknown Source)
at org.apache.derby.client.net.Reply.startSameIdChainParse(Unknown Source)
at org.apache.derby.client.net.NetXAConnectionReply.readLocalXACommit(Unknown Source)
at org.apache.derby.client.net.NetXAConnection.readLocalXACommit_(Unknown Source)
at org.apache.derby.client.net.NetXAConnection.readCommit(Unknown Source)
at org.apache.derby.client.net.NetConnection.readXACommit_(Unknown Source)
at org.apache.derby.client.am.Connection.readCommit(Unknown Source)
at org.apache.derby.client.am.Connection.readAutoCommit(Unknown Source)
at org.apache.derby.client.am.Connection.flowClose(Unknown Source)
at org.apache.derby.client.am.Connection.closeResourcesX(Unknown Source)
at org.apache.derby.client.am.Connection.closeResources(Unknown Source)
- locked <0xfffffffc91831e18> (a org.apache.derby.client.net.NetConnection40)
at org.apache.derby.client.net.NetConnection.closeResources(Unknown Source)
- locked <0xfffffffc91831e18> (a org.apache.derby.client.net.NetConnection40)
at org.apache.derby.client.ClientPooledConnection.close(Unknown Source)
- locked <0xfffffffc91831cf8> (a org.apache.derby.client.ClientXAConnection40)
at org.apache.derby.client.ClientXAConnection.close(Unknown Source)
- locked <0xfffffffc91831cf8> (a org.apache.derby.client.ClientXAConnection40)
at com.sun.gjc.spi.ManagedConnection.destroy(ManagedConnection.java:276)
at com.sun.enterprise.resource.ConnectorAllocator.destroyResource(ConnectorAllocator.java:175)
at com.sun.enterprise.resource.AbstractResourcePool.destroyResource(AbstractResourcePool.java:986)
at com.sun.enterprise.resource.AbstractResourcePool.emptyPool(AbstractResourcePool.java:1635)
- locked <0xfffffffc714c8da0> (a com.sun.enterprise.resource.SJSASResourcePool)
at com.sun.enterprise.resource.AbstractResourcePool.doFailAllConnectionsProcessing(AbstractResourcePool.java:1213)
at com.sun.enterprise.resource.AbstractResourcePool.resourceErrorOccurred(AbstractResourcePool.java:1163)
- locked <0xfffffffc714c8da0> (a com.sun.enterprise.resource.SJSASResourcePool)
at com.sun.enterprise.resource.PoolManagerImpl.putbackResourceToPool(PoolManagerImpl.java:445)
at com.sun.enterprise.resource.PoolManagerImpl.resourceErrorOccurred(PoolManagerImpl.java:411)
at com.sun.enterprise.resource.ConnectorAllocator$ConnectionListenerImpl.connectionErrorOccurred(ConnectorAllocator.java:98)
at com.sun.gjc.spi.ManagedConnection.connectionErrorOccurred(ManagedConnection.java:662)
at com.sun.gjc.spi.ManagedConnectionFactory.getInvalidConnections(ManagedConnectionFactory.java:249)
at com.sun.enterprise.resource.AbstractConnectorAllocator.getInvalidConnections(AbstractConnectorAllocator.java:99)
at com.sun.enterprise.resource.AbstractConnectorAllocator.isConnectionValid(AbstractConnectorAllocator.java:111)
at com.sun.enterprise.resource.AbstractResourcePool.isConnectionValid(AbstractResourcePool.java:722)
at com.sun.enterprise.resource.AbstractResourcePool.getUnenlistedResource(AbstractResourcePool.java:668)
at com.sun.enterprise.resource.AbstractResourcePool.internalGetResource(AbstractResourcePool.java:609)
at com.sun.enterprise.resource.AbstractResourcePool.getResource(AbstractResourcePool.java:458)
at com.sun.enterprise.resource.PoolManagerImpl.getResourceFromPool(PoolManagerImpl.java:248)
at com.sun.enterprise.resource.PoolManagerImpl.getResource(PoolManagerImpl.java:176)
at com.sun.enterprise.connectors.ConnectionManagerImpl.internalGetConnection(ConnectionManagerImpl.java:337)
at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:189)
at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:165)
at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:158)
at com.sun.gjc.spi.base.DataSource.getConnection(DataSource.java:108)
at org.eclipse.persistence.sessions.JNDIConnector.connect(JNDIConnector.java:133)
at org.eclipse.persistence.sessions.JNDIConnector.connect(JNDIConnector.java:94)
at org.eclipse.persistence.sessions.DatasourceLogin.connectToDatasource(DatasourceLogin.java:162)
at org.eclipse.persistence.internal.databaseaccess.DatasourceAccessor.connectInternal(DatasourceAccessor.java:324)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.connectInternal(DatabaseAccessor.java:276)
at org.eclipse.persistence.internal.databaseaccess.DatasourceAccessor.reconnect(DatasourceAccessor.java:546)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.reconnect(DatabaseAccessor.java:1400)
at org.eclipse.persistence.internal.databaseaccess.DatasourceAccessor.incrementCallCount(DatasourceAccessor.java:300)
- locked <0xfffffffc9d592f48> (a org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:556)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:512)
at org.eclipse.persistence.sessions.server.ServerSession.executeCall(ServerSession.java:536)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:205)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:191)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeSelectCall(DatasourceCallQueryMechanism.java:262)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.selectAllRows(DatasourceCallQueryMechanism.java:599)
at org.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectAllRowsFromTable(ExpressionQueryMechanism.java:2518)
at org.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectAllRows(ExpressionQueryMechanism.java:2476)
at org.eclipse.persistence.queries.ReadAllQuery.executeObjectLevelReadQuery(ReadAllQuery.java:481)
at org.eclipse.persistence.queries.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:928)
at org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:664)
at org.eclipse.persistence.queries.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:889)
at org.eclipse.persistence.queries.ReadAllQuery.execute(ReadAllQuery.java:458)
at org.eclipse.persistence.queries.ObjectLevelReadQuery.executeInUnitOfWork(ObjectLevelReadQuery.java:952)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2750)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1181)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1165)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1139)
at org.eclipse.persistence.internal.jpa.EJBQueryImpl.executeReadQuery(EJBQueryImpl.java:389)
at org.eclipse.persistence.internal.jpa.EJBQueryImpl.getSingleResult(EJBQueryImpl.java:594)
at com.canoga.nms.pkg.pcs.v1.entity.NpaResultsCircuitFacade.findByCircuit(NpaResultsCircuitFacade.java:57)
at sun.reflect.GeneratedMethodAccessor599.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.enterprise.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1011)
at com.sun.enterprise.security.SecurityUtil.invoke(SecurityUtil.java:175)
at com.sun.ejb.containers.BaseContainer.invokeTargetBeanMethod(BaseContainer.java:2929)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4020)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:197)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:83)
at $Proxy76.findByCircuit(Unknown Source)
at com.canoga.nms.pkg.pcs.v1.collection.service.PMCollectionServiceReceiver.getOrCreateCircuit(PMCollectionServiceReceiver.java:671)
at com.canoga.nms.pkg.pcs.v1.collection.service.PMCollectionServiceReceiver.insertPmTestResult(PMCollectionServiceReceiver.java:381)
at com.canoga.nms.pkg.pcs.v1.collection.service.PMCollectionServiceReceiver.newPmTestResult(PMCollectionServiceReceiver.java:208)
at sun.reflect.GeneratedMethodAccessor611.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.enterprise.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1011)
at com.sun.enterprise.security.SecurityUtil.invoke(SecurityUtil.java:175)
at com.sun.ejb.containers.BaseContainer.invokeTargetBeanMethod(BaseContainer.java:2929)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4020)
at com.sun.ejb.containers.MessageBeanContainer.deliverMessage(MessageBeanContainer.java:1111)
at com.sun.ejb.containers.MessageBeanListenerImpl.deliverMessage(MessageBeanListenerImpl.java:74)
at com.sun.enterprise.connectors.inflow.MessageEndpointInvocationHandler.invoke(MessageEndpointInvocationHandler.java:179)
at $Proxy154.newPmTestResult(Unknown Source)
at com.canoga.j2ee.connector.pmra.spi.PMCollectionActivation.newPmTestResult(PMCollectionActivation.java:76)
at com.canoga.j2ee.connector.pmra.spi.PMCollectionActivationManager$2.run(PMCollectionActivationManager.java:191)
at com.sun.enterprise.connectors.work.OneWork.doWork(OneWork.java:77)
at com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl$WorkerThread.run(ThreadPoolImpl.java:555)
All of the other threads that need to access the database are stuck waiting for "0xfffffffc714c8da0" which is locked in the thread above. It appears that this thread never makes any progress as I run "jstack" over and over and this thread is always at this point.
The Derby "jstack" trace looks like:
2011-12-21 05:43:35
Full thread dump Java HotSpot(TM) 64-Bit Server VM (17.1-b03 mixed mode):
"Attach Listener" daemon prio=3 tid=0x000000010481a800 nid=0x122 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"DRDAConnThread_99" prio=3 tid=0x00000001044e3800 nid=0xe4 in Object.wait() [0xfffffffd262ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_98" prio=3 tid=0x000000010466c000 nid=0xe3 in Object.wait() [0xfffffffd264ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_97" prio=3 tid=0x00000001045c7000 nid=0xe2 in Object.wait() [0xfffffffd266ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_96" prio=3 tid=0x0000000101628800 nid=0xe1 runnable [0xfffffffd268fe000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_95" prio=3 tid=0x000000010449c000 nid=0xe0 in Object.wait() [0xfffffffd26aff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_94" prio=3 tid=0x0000000104111000 nid=0xdf runnable [0xfffffffd26cff000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_93" prio=3 tid=0x00000001015e3800 nid=0xde in Object.wait() [0xfffffffd26efe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_92" prio=3 tid=0x0000000101ed4800 nid=0xdd in Object.wait() [0xfffffffd270fe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_91" prio=3 tid=0x0000000102443000 nid=0xdc in Object.wait() [0xfffffffd272ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_90" prio=3 tid=0x0000000100e9b800 nid=0xdb in Object.wait() [0xfffffffd274ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_89" prio=3 tid=0x00000001034f9000 nid=0xda runnable [0xfffffffd276fe000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_88" prio=3 tid=0x0000000101092800 nid=0xd9 in Object.wait() [0xfffffffd278ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_87" prio=3 tid=0x00000001011ee800 nid=0xd8 in Object.wait() [0xfffffffd27aff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_86" prio=3 tid=0x0000000103a90800 nid=0xd7 in Object.wait() [0xfffffffd27cff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_85" prio=3 tid=0x000000010445b800 nid=0xd6 in Object.wait() [0xfffffffd27efe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_84" prio=3 tid=0x000000010445b000 nid=0xd5 in Object.wait() [0xfffffffd280fe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_83" prio=3 tid=0x0000000104566000 nid=0xd4 in Object.wait() [0xfffffffd282ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_82" prio=3 tid=0x00000001016c1000 nid=0xd3 in Object.wait() [0xfffffffd284ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_81" prio=3 tid=0x0000000101f20000 nid=0xd2 in Object.wait() [0xfffffffd286ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_80" prio=3 tid=0x0000000101153800 nid=0xd1 in Object.wait() [0xfffffffd288ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_79" prio=3 tid=0x0000000103bc5000 nid=0xd0 in Object.wait() [0xfffffffd28aff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_78" prio=3 tid=0x0000000102968800 nid=0xcf in Object.wait() [0xfffffffd28cff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_77" prio=3 tid=0x0000000103c69800 nid=0xce in Object.wait() [0xfffffffd28efe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_76" prio=3 tid=0x0000000102b2b000 nid=0xcd in Object.wait() [0xfffffffd290fe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_75" prio=3 tid=0x000000010359e000 nid=0xcc in Object.wait() [0xfffffffd292ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_74" prio=3 tid=0x0000000102d41800 nid=0xcb in Object.wait() [0xfffffffd294ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_73" prio=3 tid=0x00000001022e8800 nid=0xca runnable [0xfffffffd296ff000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_72" prio=3 tid=0x0000000103674800 nid=0xc9 in Object.wait() [0xfffffffd298ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_71" prio=3 tid=0x0000000102a5d000 nid=0xc8 in Object.wait() [0xfffffffd29aff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_70" prio=3 tid=0x000000010143b800 nid=0xc7 in Object.wait() [0xfffffffd29cff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_69" prio=3 tid=0x0000000102331800 nid=0xc6 in Object.wait() [0xfffffffd29efe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_68" prio=3 tid=0x00000001035be000 nid=0xc5 in Object.wait() [0xfffffffd2a0fe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_67" prio=3 tid=0x00000001019fc000 nid=0xc4 in Object.wait() [0xfffffffd2a2ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_66" prio=3 tid=0x0000000102289800 nid=0xc3 in Object.wait() [0xfffffffd2a4ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_65" prio=3 tid=0x00000001013fa000 nid=0xc2 in Object.wait() [0xfffffffd2a6ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_64" prio=3 tid=0x0000000103028000 nid=0xc1 in Object.wait() [0xfffffffd2a8ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_63" prio=3 tid=0x0000000101a43800 nid=0xc0 in Object.wait() [0xfffffffd2aaff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_62" prio=3 tid=0x0000000101c79800 nid=0xbf in Object.wait() [0xfffffffd2acff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_61" prio=3 tid=0x0000000103ab9800 nid=0xbe in Object.wait() [0xfffffffd2aefe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_60" prio=3 tid=0x0000000101a82000 nid=0xbd in Object.wait() [0xfffffffd2b0fe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_59" prio=3 tid=0x00000001014d4800 nid=0xbc in Object.wait() [0xfffffffd2b2ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_58" prio=3 tid=0x0000000101e93800 nid=0xbb runnable [0xfffffffd2b4fe000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_57" prio=3 tid=0x000000010224c000 nid=0xba in Object.wait() [0xfffffffd2b6ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_56" prio=3 tid=0x00000001034d9000 nid=0xb9 in Object.wait() [0xfffffffd2b8ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_55" prio=3 tid=0x0000000102c96800 nid=0xb6 in Object.wait() [0xfffffffd2bafe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_54" prio=3 tid=0x000000010222f000 nid=0xb5 in Object.wait() [0xfffffffd2bcfe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_53" prio=3 tid=0x00000001016e6000 nid=0xb4 in Object.wait() [0xfffffffd2beff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_52" prio=3 tid=0x00000001010d0000 nid=0xb3 in Object.wait() [0xfffffffd2c0ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_51" prio=3 tid=0x0000000101288800 nid=0xb2 in Object.wait() [0xfffffffd2c2ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_50" prio=3 tid=0x0000000100ddd000 nid=0xb1 in Object.wait() [0xfffffffd2c4ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_49" prio=3 tid=0x00000001039d9000 nid=0xb0 in Object.wait() [0xfffffffd2c6ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_48" prio=3 tid=0x00000001014e8000 nid=0xaf in Object.wait() [0xfffffffd2c8ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_47" prio=3 tid=0x0000000102cb9800 nid=0xae runnable [0xfffffffd2cafe000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_46" prio=3 tid=0x0000000102346800 nid=0xad in Object.wait() [0xfffffffd2ccfe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_45" prio=3 tid=0x0000000102372800 nid=0xac in Object.wait() [0xfffffffd2ceff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_44" prio=3 tid=0x0000000102a28800 nid=0xab in Object.wait() [0xfffffffd2d0ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_43" prio=3 tid=0x0000000103112800 nid=0xaa in Object.wait() [0xfffffffd2d2ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_42" prio=3 tid=0x0000000100e10000 nid=0xa9 in Object.wait() [0xfffffffd2d4ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_41" prio=3 tid=0x000000010247f800 nid=0xa8 in Object.wait() [0xfffffffd2d6ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_40" prio=3 tid=0x0000000102945000 nid=0xa7 in Object.wait() [0xfffffffd2d8ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_39" prio=3 tid=0x0000000102347800 nid=0xa6 in Object.wait() [0xfffffffd2dafe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_38" prio=3 tid=0x0000000102e03000 nid=0xa5 runnable [0xfffffffd2dcfe000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_37" prio=3 tid=0x0000000101910800 nid=0xa4 in Object.wait() [0xfffffffd2deff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_36" prio=3 tid=0x00000001022bd800 nid=0xa3 in Object.wait() [0xfffffffd2e0ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_35" prio=3 tid=0x00000001029a1800 nid=0xa2 in Object.wait() [0xfffffffd2e2ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_34" prio=3 tid=0x0000000102108800 nid=0xa1 in Object.wait() [0xfffffffd2e4ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_33" prio=3 tid=0x0000000102226000 nid=0xa0 in Object.wait() [0xfffffffd2e6ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_32" prio=3 tid=0x00000001029a5000 nid=0x9f in Object.wait() [0xfffffffd2e8ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_31" prio=3 tid=0x00000001010ff000 nid=0x9e in Object.wait() [0xfffffffd2eafe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_30" prio=3 tid=0x0000000101bad000 nid=0x9d in Object.wait() [0xfffffffd2ecfe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_29" prio=3 tid=0x0000000101dbe000 nid=0x32 runnable [0xfffffffd2eefe000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_28" prio=3 tid=0x00000001011c4000 nid=0x31 in Object.wait() [0xfffffffd2f0ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_27" prio=3 tid=0x00000001021fd000 nid=0x2f runnable [0xfffffffd2f4ff000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_26" prio=3 tid=0x0000000100120000 nid=0x2e in Object.wait() [0xfffffffd2f6fe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_25" prio=3 tid=0x000000010011e000 nid=0x2d runnable [0xfffffffd2f8fe000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_24" prio=3 tid=0x000000010011d000 nid=0x2c in Object.wait() [0xfffffffd2faff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_23" prio=3 tid=0x000000010011c000 nid=0x2b in Object.wait() [0xfffffffd2fcff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_22" prio=3 tid=0x0000000100fd9800 nid=0x2a runnable [0xfffffffd2feff000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_21" prio=3 tid=0x0000000100fd9000 nid=0x29 in Object.wait() [0xffffffff706ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_20" prio=3 tid=0x0000000101dfa000 nid=0x28 in Object.wait() [0xffffffff708ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_19" prio=3 tid=0x00000001029d7000 nid=0x27 runnable [0xffffffff70aff000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_18" prio=3 tid=0x00000001029d8000 nid=0x26 runnable [0xffffffff70cfe000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_17" prio=3 tid=0x00000001022ba800 nid=0x25 runnable [0xffffffff70efe000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_16" prio=3 tid=0x0000000102374000 nid=0x24 runnable [0xffffffff710fe000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_15" prio=3 tid=0x00000001021fc000 nid=0x23 runnable [0xffffffff712ff000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_14" prio=3 tid=0x0000000100d07000 nid=0x22 in Object.wait() [0xffffffff714ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_13" prio=3 tid=0x0000000101b4a000 nid=0x21 runnable [0xffffffff716ff000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_12" prio=3 tid=0x000000010108b000 nid=0x20 runnable [0xffffffff718ff000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_11" prio=3 tid=0x0000000101960800 nid=0x1f runnable [0xffffffff71aff000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_10" prio=3 tid=0x0000000102362800 nid=0x1e in Object.wait() [0xffffffff71cfe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_9" prio=3 tid=0x0000000100df9800 nid=0x1d runnable [0xffffffff71efe000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_8" prio=3 tid=0x0000000100d47000 nid=0x1c in Object.wait() [0xffffffff720ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_7" prio=3 tid=0x0000000101134000 nid=0x1b in Object.wait() [0xffffffff722ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_6" prio=3 tid=0x0000000100fe8000 nid=0x1a runnable [0xffffffff724fe000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"DRDAConnThread_5" prio=3 tid=0x0000000100d2e000 nid=0x19 in Object.wait() [0xffffffff726ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"derby.rawStoreDaemon" daemon prio=3 tid=0x0000000100fe7000 nid=0x18 in Object.wait() [0xffffffff728ff000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.apache.derby.impl.services.daemon.BasicDaemon.rest(Unknown Source)
- locked <0xfffffffd77095100> (a org.apache.derby.impl.services.daemon.BasicDaemon)
at org.apache.derby.impl.services.daemon.BasicDaemon.run(Unknown Source)
at java.lang.Thread.run(Thread.java:662)
"DRDAConnThread_4" prio=3 tid=0x0000000101365000 nid=0x17 runnable [0xffffffff72aff000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.impl.drda.DDMReader.fill(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.impl.drda.DDMReader.readDssHeader(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.processCommands(Unknown Source)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"derby.rawStoreDaemon" daemon prio=3 tid=0x0000000100d3b000 nid=0x16 in Object.wait() [0xffffffff732ff000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.apache.derby.impl.services.daemon.BasicDaemon.rest(Unknown Source)
- locked <0xfffffffd76fd1e90> (a org.apache.derby.impl.services.daemon.BasicDaemon)
at org.apache.derby.impl.services.daemon.BasicDaemon.run(Unknown Source)
at java.lang.Thread.run(Thread.java:662)
"DRDAConnThread_3" prio=3 tid=0x000000010130d000 nid=0x15 in Object.wait() [0xffffffff734fe000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.getNextSession(Unknown Source)
- locked <0xfffffffd73baa288> (a java.util.Vector)
at org.apache.derby.impl.drda.DRDAConnThread.run(Unknown Source)
"NetworkServerThread_2" prio=3 tid=0x0000000100fdd000 nid=0x14 runnable [0xffffffff736fe000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
- locked <0xfffffffd77039ae8> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:453)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at org.apache.derby.impl.drda.ClientThread$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.derby.impl.drda.ClientThread.run(Unknown Source)
"Timer-0" daemon prio=3 tid=0x000000010104b800 nid=0x13 in Object.wait() [0xffffffff73aff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xfffffffd77054aa0> (a java.util.TaskQueue)
at java.lang.Object.wait(Object.java:485)
at java.util.TimerThread.mainLoop(Timer.java:483)
- locked <0xfffffffd77054aa0> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)
"Low Memory Detector" daemon prio=3 tid=0x0000000100b09000 nid=0x11 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"CompilerThread1" daemon prio=3 tid=0x0000000100b06000 nid=0x10 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"CompilerThread0" daemon prio=3 tid=0x0000000100b03000 nid=0xf waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" daemon prio=3 tid=0x0000000100b01000 nid=0xe runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Finalizer" daemon prio=3 tid=0x0000000100adf000 nid=0xd in Object.wait() [0xffffffff7b7ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
- locked <0xfffffffd77054f00> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
"Reference Handler" daemon prio=3 tid=0x0000000100ad8000 nid=0xc in Object.wait() [0xffffffff7b9ff000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
- locked <0xfffffffd73df7618> (a java.lang.ref.Reference$Lock)
"main" prio=3 tid=0x0000000100117000 nid=0x2 in Object.wait() [0xffffffff7c6fd000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xfffffffd77039aa0> (a java.lang.Object)
at java.lang.Object.wait(Object.java:485)
at org.apache.derby.impl.drda.NetworkServerControlImpl.blockingStart(Unknown Source)
- locked <0xfffffffd77039aa0> (a java.lang.Object)
at org.apache.derby.impl.drda.NetworkServerControlImpl.executeWork(Unknown Source)
at org.apache.derby.drda.NetworkServerControl.main(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.enterprise.cli.commands.DerbyControl.invokeNetworkServerControl(DerbyControl.java:132)
at com.sun.enterprise.cli.commands.DerbyControl.main(DerbyControl.java:215)
"VM Thread" prio=3 tid=0x0000000100ad4000 nid=0xb runnable
"GC task thread#0 (ParallelGC)" prio=3 tid=0x0000000100127000 nid=0x3 runnable
"GC task thread#1 (ParallelGC)" prio=3 tid=0x0000000100128800 nid=0x4 runnable
"GC task thread#2 (ParallelGC)" prio=3 tid=0x000000010012a000 nid=0x5 runnable
"GC task thread#3 (ParallelGC)" prio=3 tid=0x000000010012b800 nid=0x6 runnable
"GC task thread#4 (ParallelGC)" prio=3 tid=0x000000010012d800 nid=0x7 runnable
"GC task thread#5 (ParallelGC)" prio=3 tid=0x000000010012f000 nid=0x8 runnable
"GC task thread#6 (ParallelGC)" prio=3 tid=0x0000000100130800 nid=0x9 runnable
"GC task thread#7 (ParallelGC)" prio=3 tid=0x0000000100132000 nid=0xa runnable
"VM Periodic Task Thread" prio=3 tid=0x0000000100b1e000 nid=0x12 waiting on condition
JNI global references: 955
Nothing really looks suspicious here except the lack of any observable work being done. The only other thing that is interesting is that there 374 transactions still active in the database as reported by:
1 row selected
ij> select count(*) from syscs_diag.transaction_table where status != 'IDLE';
1
-----------
374
Unfortunately I was not able to look at the transactions before the server got reset, but I am in the process of trying to replicate this again.
It is a if the ClientXADataSource is trying to read from a connection that does not exist anymore.
I have no problem with trying to debug this as I have both Glassfish built from source and Derby built from source. What I am hoping to get is some direction on where to look. From looking at the source of the client driver where it appears to be stuck I see:
do {
try {
// oops, we shouldn't expose the agent's input stream here, collapse this into a read method on the agent
actualBytesRead = netAgent_.getInputStream().read(buffer_, count_, buffer_.length - count_);
} catch (java.io.IOException ioe) {
netAgent_.throwCommunicationsFailure(ioe);
} finally {
if (agent_.loggingEnabled()) {
((NetLogWriter) netAgent_.logWriter_).traceProtocolFlow(buffer_,
count_,
actualBytesRead,
NetLogWriter.TYPE_TRACE_RECEIVE,
"Reply",
"fill",
2); // tracepoint
}
}
// DERBY-2747: only count if we actually read something
if (actualBytesRead > 0) {
count_ += actualBytesRead;
totalBytesRead += actualBytesRead;
}
} while ((totalBytesRead < minimumBytesNeeded) && (actualBytesRead != -1));
So it seems that there are too possible ways this could get stuck. Either "actualBytesRead" is returning "-1" but no IOException being thrown or "actualBytesRead" is returning "0". Any thoughts on this?
Any and all help, pointers where to look, etc. will be greatly appreciated.
Re: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
Posted by Katherine Marsden <km...@sbcglobal.net>.
On 12/21/2011 1:31 PM, Katherine Marsden wrote:
One more random bit of information for your test case. I recall when I
needed to add an XA Specific test case adding the code below to
SavepointJdbc30Test to get the existing test to run as well with XA. You
might try doing that with the existing deadlock test cases to pop the
issue. I don't think though that it tests two phase commit.
// Repeat the embedded tests obtaining a connection from
// an XA data source if it is supported. This is not supported
// under JSR169.
if (JDBC.vmSupportsJDBC3()) {
embedded = new TestSuite(
"SavepointJdbc30_JSR169Test:embedded XADataSource");
embedded.addTestSuite(SavepointJdbc30Test.class);
embedded.addTest(getEmbeddedSuite("SavepointJdbc30_JSR169Test:"
+ "embedded only XADataSource"));
suite.addTest(TestConfiguration.connectionXADecorator(embedded));
// Repeat the client tests obtaining a
connection from
// an XA data source if it is supported. This is not supported
// under JSR169.
client = new TestSuite("SavepointJdbc30_JSR169Test:client
XADatasource");
client.addTestSuite(SavepointJdbc30Test.class);
suite.addTest(TestConfiguration.clientServerDecorator(TestConfiguration.connectionXADecorator(client)));
}
It would be neat to have modes in which all the tests could be run
using different types of connections, etc.
https://issues.apache.org/jira/browse/DERBY-1134
Kathey
Re: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
Posted by Katherine Marsden <km...@sbcglobal.net>.
On 12/21/2011 12:40 PM, Bergquist, Brett wrote:
>
> This also triggers another problem that I am having in production. If
> these transactions are left in this state, Derby will build thousands
> of transaction files in the "log" database. If the server is not
> stopped and restarted soon after these start occurring, then it may
> take days for Derby to boot the database.
>
Uncompleted XA transactions will persist even after the the server is
stopped and have to be recovered XAResource.recover() and told
explicitly whether to commit or rollback. An example of using
XAResource.recover() is in XATest testInterleavingTransactions(). Once
the core problem is determined, you should run an xa recovery
operation to clean up the residual transactions.
> I assume that these are XA transactions that did not complete. I have
> Glassfish to timeout transactions after 15 minutes but this appears
> not to be working. I would like to set the
> "derby.jdbc.xaTransactionTimeout" property but I am leery because of"
>
> https://issues.apache.org/jira/browse/DERBY-4109
>
I can't really say to much about DERBY-4109 as we don't have a
reproduction and I don't understand the circumstances under which it
occurs, but in your case, I tend to think extending timeout is not
going to help as it seems you have hit a protocol exchange where the
server will never respond to the client.
Looking forward to seeing the excerpts from the client and server traces
after the deadlock and hearing how your simple deadlock test goes. If
you need a simple standalone template for XA to start from, I think
ReproDerby1016.java attached to DERBY-1016 might work.
Thanks
Kathey
RE: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
Posted by "Bergquist, Brett" <BB...@canoga.com>.
This also triggers another problem that I am having in production. If these transactions are left in this state, Derby will build thousands of transaction files in the "log" database. If the server is not stopped and restarted soon after these start occurring, then it may take days for Derby to boot the database.
I assume that these are XA transactions that did not complete. I have Glassfish to timeout transactions after 15 minutes but this appears not to be working. I would like to set the "derby.jdbc.xaTransactionTimeout" property but I am leery because of"
https://issues.apache.org/jira/browse/DERBY-4109
From: Bergquist, Brett [mailto:BBergquist@canoga.com]
Sent: Wednesday, December 21, 2011 3:33 PM
To: derby-dev@db.apache.org
Subject: RE: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
After shutting down the application server but leaving the database engine running, it still has transactions:
6789036870 (4871251,2b04000025fa2e6273756e332d7369743233372c7365727665722c5033373030,73756e332d7369743233372c7365727665722c50333730302c00) CSEM UserTransaction ACTIVE (1735812,468154) <null>
6789064593 (4871251,1512000025fa2e6273756e332d7369743233372c7365727665722c5033373030,73756e332d7369743233372c7365727665722c50333730302c00) CSEM UserTransaction ACTIVE <null> select max(csid) from ( select min(cs.id) as csid, min(cs.configuration_number) as csnum, cbe.id as cbid from --DERBY-PROPERTIES joinOrder=FIXED/n core_v1.configurable_hardware ch join core_v1.configuration_set cs on cs.configurable_hardware_id = ch.id join core_v1.configurationset_configurationbundle cscb on cscb.configurationset_id = cs.id join core_v1.configuration_bundle cb on cb.id = cscb.configurationbundle_id join pkg_9145e10g.configuration_bundle_9145e10g cbe on cbe.id = cb.id where ch.id = ? and cb.bundle_name = ? group by cbe.id ) as lcs
6789041545 (4871251,2306000025fa2e6273756e332d7369743233372c7365727665722c5033373030,73756e332d7369743233372c7365727665722c50333730302c00) CSEM UserTransaction ACTIVE <null> select max(csid) from ( select min(cs.id) as csid, min(cs.configuration_number) as csnum, cbe.id as cbid from --DERBY-PROPERTIES joinOrder=FIXED/n core_v1.configurable_hardware ch join core_v1.configuration_set cs on cs.configurable_hardware_id = ch.id join core_v1.configurationset_configurationbundle cscb on cscb.configurationset_id = cs.id join core_v1.configuration_bundle cb on cb.id = cscb.configurationbundle_id join pkg_9145e10g.configuration_bundle_9145e10g cbe on cbe.id = cb.id where ch.id = ? and cb.bundle_name = ? group by cbe.id ) as lcs
6789042080 <null> CSEM UserTransaction ACTIVE <null> SELECT t0.ID, t0.DTYPE, t0.BUNDLE_NAME, t0.OPLOCK, t1.ID, t2.ID, t2.PM_END_DELAY, t2.PM_BETWEEN_TIME, t2.PM_DIS_OVER_TIME, t2.PM_DIS_END_DELAY, t2.PM_SCHEDULER_POLICY, t2.PM_SCHEDULER_STATE FROM CORE_V1.CONFIGURATIONSET_CONFIGURATIONBUNDLE t4, CORE_V1.CONFIGURATION_SET t3, PKG_9145E10G.PM_SCHEDULER_CONFIG_BUNDLE t2, PKG_9145E10G.CONFIGURATION_BUNDLE_9145E10G t1, CORE_V1.CONFIGURATION_BUNDLE t0 WHERE ((((t3.ID = CAST (? AS INTEGER )) AND (t0.BUNDLE_NAME = CAST (? AS VARCHAR(32672) ))) AND (((t2.ID = t0.ID) AND (t1.ID = t0.ID)) AND (t0.DTYPE = 'PM_SCHEDULER_CONFIG_BUNDLE_9145E10G'))) AND ((t4.CONFIGURATIONBUNDLE_ID = t0.ID) AND (t3.ID = t4.CONFIGURATIONSET_ID)))
From: Katherine Marsden [mailto:kmarsdenderby@sbcglobal.net]
Sent: Wednesday, December 21, 2011 2:46 PM
To: derby-dev@db.apache.org
Subject: Re: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
On 12/21/2011 11:20 AM, Bergquist, Brett wrote:
I'm having some trouble getting client side tracing to work. The connections are managed by Glassfish connection pool so I don't know where to set the traceDirectory and traceLevel properties. Can these be specified as properties like the password, etc.
They can be set on the connection URL or with undocumented system properties, documented here #:)
http://wiki.apache.org/db-derby/UndocumentedDerbyBehavior
Looking at the info, again I am curious if there are corresponding server side traces in the derby.log.
Also it would be interesting to see if there are at this point any XA Transactions in need of recovery in the database.
Just exit your application and connect with ij and run:
select * from SYSCS_DIAG.TRANSACTION_TABLE ;
RE: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
Posted by "Bergquist, Brett" <BB...@canoga.com>.
After shutting down the application server but leaving the database engine running, it still has transactions:
6789036870 (4871251,2b04000025fa2e6273756e332d7369743233372c7365727665722c5033373030,73756e332d7369743233372c7365727665722c50333730302c00) CSEM UserTransaction ACTIVE (1735812,468154) <null>
6789064593 (4871251,1512000025fa2e6273756e332d7369743233372c7365727665722c5033373030,73756e332d7369743233372c7365727665722c50333730302c00) CSEM UserTransaction ACTIVE <null> select max(csid) from ( select min(cs.id) as csid, min(cs.configuration_number) as csnum, cbe.id as cbid from --DERBY-PROPERTIES joinOrder=FIXED/n core_v1.configurable_hardware ch join core_v1.configuration_set cs on cs.configurable_hardware_id = ch.id join core_v1.configurationset_configurationbundle cscb on cscb.configurationset_id = cs.id join core_v1.configuration_bundle cb on cb.id = cscb.configurationbundle_id join pkg_9145e10g.configuration_bundle_9145e10g cbe on cbe.id = cb.id where ch.id = ? and cb.bundle_name = ? group by cbe.id ) as lcs
6789041545 (4871251,2306000025fa2e6273756e332d7369743233372c7365727665722c5033373030,73756e332d7369743233372c7365727665722c50333730302c00) CSEM UserTransaction ACTIVE <null> select max(csid) from ( select min(cs.id) as csid, min(cs.configuration_number) as csnum, cbe.id as cbid from --DERBY-PROPERTIES joinOrder=FIXED/n core_v1.configurable_hardware ch join core_v1.configuration_set cs on cs.configurable_hardware_id = ch.id join core_v1.configurationset_configurationbundle cscb on cscb.configurationset_id = cs.id join core_v1.configuration_bundle cb on cb.id = cscb.configurationbundle_id join pkg_9145e10g.configuration_bundle_9145e10g cbe on cbe.id = cb.id where ch.id = ? and cb.bundle_name = ? group by cbe.id ) as lcs
6789042080 <null> CSEM UserTransaction ACTIVE <null> SELECT t0.ID, t0.DTYPE, t0.BUNDLE_NAME, t0.OPLOCK, t1.ID, t2.ID, t2.PM_END_DELAY, t2.PM_BETWEEN_TIME, t2.PM_DIS_OVER_TIME, t2.PM_DIS_END_DELAY, t2.PM_SCHEDULER_POLICY, t2.PM_SCHEDULER_STATE FROM CORE_V1.CONFIGURATIONSET_CONFIGURATIONBUNDLE t4, CORE_V1.CONFIGURATION_SET t3, PKG_9145E10G.PM_SCHEDULER_CONFIG_BUNDLE t2, PKG_9145E10G.CONFIGURATION_BUNDLE_9145E10G t1, CORE_V1.CONFIGURATION_BUNDLE t0 WHERE ((((t3.ID = CAST (? AS INTEGER )) AND (t0.BUNDLE_NAME = CAST (? AS VARCHAR(32672) ))) AND (((t2.ID = t0.ID) AND (t1.ID = t0.ID)) AND (t0.DTYPE = 'PM_SCHEDULER_CONFIG_BUNDLE_9145E10G'))) AND ((t4.CONFIGURATIONBUNDLE_ID = t0.ID) AND (t3.ID = t4.CONFIGURATIONSET_ID)))
From: Katherine Marsden [mailto:kmarsdenderby@sbcglobal.net]
Sent: Wednesday, December 21, 2011 2:46 PM
To: derby-dev@db.apache.org
Subject: Re: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
On 12/21/2011 11:20 AM, Bergquist, Brett wrote:
I'm having some trouble getting client side tracing to work. The connections are managed by Glassfish connection pool so I don't know where to set the traceDirectory and traceLevel properties. Can these be specified as properties like the password, etc.
They can be set on the connection URL or with undocumented system properties, documented here #:)
http://wiki.apache.org/db-derby/UndocumentedDerbyBehavior
Looking at the info, again I am curious if there are corresponding server side traces in the derby.log.
Also it would be interesting to see if there are at this point any XA Transactions in need of recovery in the database.
Just exit your application and connect with ij and run:
select * from SYSCS_DIAG.TRANSACTION_TABLE ;
RE: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
Posted by "Bergquist, Brett" <BB...@canoga.com>.
I opened DERBY-5552
https://issues.apache.org/jira/browse/DERBY-5552
I attached client side traces using traceLevel=2145 (TRACE_XA_CALLS|TRACE_PROTOCOL_FLOWS|TRACE_CONNECTS|TRACE_CONNECTION_CALL). I don't know if more is needed.
I have sever side traces but it is 93Mb uncompressed and 13Mb compressed. Is there something I should look for in there to narrow down which trace files to include and upload?
I have attached jstack traces of the application server and the Derby Network Server at shutdown showing the hung threads.
I have attached a the output of "select * from syscs_diag.transaction_table" when there are no clients and no other database action showing transactions that are still present.
I am trying to narrow down a test case better but have not been able to at this point. This is repeatable with my J2EE application every time however with the test setup that I have. Any further areas to look at with a debugger or outputting more tracing information will be greatly appreciated.
From: Katherine Marsden [mailto:kmarsdenderby@sbcglobal.net]
Sent: Wednesday, December 21, 2011 7:25 PM
To: derby-dev@db.apache.org
Subject: Re: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
On 12/21/2011 3:14 PM, Bergquist, Brett wrote:
Will get to this tomorrow but I do see one comment in the code that I don't understand:
In DRDAConnThread.java, I see:
if (severity > CodePoint.SVRCOD_ERROR)
{
// For a session ending error > CodePoint.SRVCOD_ERROR you cannot
// send a SQLERRRM. A CMDCHKRM is required. In XA if there is a
// lock timeout it ends the whole session. I am not sure this
// is the correct behaviour but if it occurs we have to send
// a CMDCHKRM instead of SQLERRM
writeCMDCHKRM(severity);
}
So what does the comment "In XA if there is a lock timeout it ends the whole session" refer to. Why would a lock timeout be any different than any other standard database error. It is like this is hinting at what is happening.
This is a real XA transaction.
What I see is that after the timeout is hit (I see it hit in Timeout.java) the error is propagated to the app server. The app server then attempts to get the error text (I don't have the code handy) which attempts to send a request back to the Derby. This then fails with a No Connection error being returned back from Derby. It is as if after this error, the connection between the app server and Derby is no longer once there this is hit.
I agree that would not be the correct behavior if a lock timeout killed the session. As this is a server side comment it would imply that this is a problem with embedded as well as well, but hard to believe it would not have been exposed before now. Thanks for working on reproduction for this. I don't see the comment in the original code import but the annotation is not clear as it mentions the back out of another fix, so I am not sure who first noticed this behavior.
Re: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
Posted by Katherine Marsden <km...@sbcglobal.net>.
On 12/21/2011 3:14 PM, Bergquist, Brett wrote:
>
> Will get to this tomorrow but I do see one comment in the code that I
> don't understand:
>
> In DRDAConnThread.java, I see:
>
> if (severity > CodePoint.SVRCOD_ERROR)
>
> {
>
> // For a session
> ending error > CodePoint.SRVCOD_ERROR you cannot
>
> // send a SQLERRRM. A
> CMDCHKRM is required. In XA if there is a
>
> // lock timeout it
> ends the whole session. I am not sure this
>
> // is the correct
> behaviour but if it occurs we have to send
>
> // a CMDCHKRM instead
> of SQLERRM
>
> writeCMDCHKRM(severity);
>
> }
>
> So what does the comment "In XA if there is a lock timeout it ends the
> whole session" refer to. Why would a lock timeout be any different
> than any other standard database error. It is like this is hinting at
> what is happening.
>
> This is a real XA transaction.
>
> What I see is that after the timeout is hit (I see it hit in
> Timeout.java) the error is propagated to the app server. The app
> server then attempts to get the error text (I don't have the code
> handy) which attempts to send a request back to the Derby. This then
> fails with a No Connection error being returned back from Derby. It
> is as if after this error, the connection between the app server and
> Derby is no longer once there this is hit.
>
I agree that would not be the correct behavior if a lock timeout killed
the session. As this is a server side comment it would imply that this
is a problem with embedded as well as well, but hard to believe it would
not have been exposed before now. Thanks for working on reproduction for
this. I don't see the comment in the original code import but the
annotation is not clear as it mentions the back out of another fix, so I
am not sure who first noticed this behavior.
RE: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
Posted by "Bergquist, Brett" <BB...@canoga.com>.
Will get to this tomorrow but I do see one comment in the code that I don't understand:
In DRDAConnThread.java, I see:
if (severity > CodePoint.SVRCOD_ERROR)
{
// For a session ending error > CodePoint.SRVCOD_ERROR you cannot
// send a SQLERRRM. A CMDCHKRM is required. In XA if there is a
// lock timeout it ends the whole session. I am not sure this
// is the correct behaviour but if it occurs we have to send
// a CMDCHKRM instead of SQLERRM
writeCMDCHKRM(severity);
}
So what does the comment "In XA if there is a lock timeout it ends the whole session" refer to. Why would a lock timeout be any different than any other standard database error. It is like this is hinting at what is happening.
This is a real XA transaction.
What I see is that after the timeout is hit (I see it hit in Timeout.java) the error is propagated to the app server. The app server then attempts to get the error text (I don't have the code handy) which attempts to send a request back to the Derby. This then fails with a No Connection error being returned back from Derby. It is as if after this error, the connection between the app server and Derby is no longer once there this is hit.
I am going to go try to follow through the code and get a smaller reproducible example.
From: Katherine Marsden [mailto:kmarsdenderby@sbcglobal.net]
Sent: Wednesday, December 21, 2011 3:41 PM
To: derby-dev@db.apache.org
Subject: Re: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
On 12/21/2011 12:04 PM, Bergquist, Brett wrote:
Nothing in the Derby log other than it logging a deadlock with the statements and a lock timeout with its statements and it indicating that cleanup had started and completed.
I will enable tracing with the documented (undocumented system property). Thanks for that information!
I will check for the XA transactions the next time I reproduce this.
Maybe you could point me into the correct area to look. This seems to be triggered either through a lock timeout or a deadlock. The connection that this is occurring through is an XA connection. I see the logging of this in the server log but I am trying to find out where that would be logged from. It seems after this occurs and because of the way connection pool is being validated and recreated on error by Glassfish (configured to do so), it gets into this state. What I don't understand is why this type of error would cause the connection to appear to be invalid and I am trying to work through both the Glassfish source and the Derby source to find out. The connection is correctly handling other errors such as a duplication trying to be inserted and this does not trigger the connection to appear to be invalid. So I am trying to understand why a lock timeout or deadlock detection might do so.
This problem has only cropped up recently when they started performing multiple requests that I know have a deadlock path through them. I can fix that problem later but this is a system level problem that I need to resolve.
I really do appreciate the help and guidance and am willing to try to work though this. I have to figure this out and either patch Glassfish or Derby in any case as my customer (think very very large wireless carrier) is getting pretty PO'ed.
The one thing I think of specifically with a deadlock is that it will automatically rollback the victim transaction and that might throw off this client logic regarding the state of the server. But I would think if there were just a simple problem with deadlocks it would have showed up before now. That said I don't see any specific tests in our XA tests: org.apache.derbyTesting.functionTests.tests.jdbapi.XATest or org.apache.derbyTesting.functionTests.tests.jdbcapi.XATransactionTest that test XAConnections with deadlocks.
Is this a local transaction on an XA connection or a real XA transaction with two phase commit?
You might want to try to test and an XAConnection with a simple deadlock case locally to see if that pops a reproduction. org.apache.derbyTesting.functionTests.tests.lang.DeadlockDetectionTest and org.apache.derbyTesting.functionTests.tests.lang have some examples of deadlocks.
HTH,
Kathey
Re: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
Posted by Katherine Marsden <km...@sbcglobal.net>.
On 12/21/2011 12:04 PM, Bergquist, Brett wrote:
>
> Nothing in the Derby log other than it logging a deadlock with the
> statements and a lock timeout with its statements and it indicating
> that cleanup had started and completed.
>
> I will enable tracing with the documented (undocumented system
> property). Thanks for that information!
>
> I will check for the XA transactions the next time I reproduce this.
>
> Maybe you could point me into the correct area to look. This seems to
> be triggered either through a lock timeout or a deadlock. The
> connection that this is occurring through is an XA connection. I see
> the logging of this in the server log but I am trying to find out
> where that would be logged from. It seems after this occurs and
> because of the way connection pool is being validated and recreated on
> error by Glassfish (configured to do so), it gets into this state.
> What I don't understand is why this type of error would cause the
> connection to appear to be invalid and I am trying to work through
> both the Glassfish source and the Derby source to find out. The
> connection is correctly handling other errors such as a duplication
> trying to be inserted and this does not trigger the connection to
> appear to be invalid. So I am trying to understand why a lock
> timeout or deadlock detection might do so.
>
> This problem has only cropped up recently when they started performing
> multiple requests that I know have a deadlock path through them. I
> can fix that problem later but this is a system level problem that I
> need to resolve.
>
> I really do appreciate the help and guidance and am willing to try to
> work though this. I have to figure this out and either patch
> Glassfish or Derby in any case as my customer (think very very large
> wireless carrier) is getting pretty PO'ed.
>
The one thing I think of specifically with a deadlock is that it will
automatically rollback the victim transaction and that might throw off
this client logic regarding the state of the server. But I would
think if there were just a simple problem with deadlocks it would have
showed up before now. That said I don't see any specific tests in our XA
tests: org.apache.derbyTesting.functionTests.tests.jdbapi.XATest or
org.apache.derbyTesting.functionTests.tests.jdbcapi.XATransactionTest
that test XAConnections with deadlocks.
Is this a local transaction on an XA connection or a real XA
transaction with two phase commit?
You might want to try to test and an XAConnection with a simple
deadlock case locally to see if that pops a reproduction.
org.apache.derbyTesting.functionTests.tests.lang.DeadlockDetectionTest
and org.apache.derbyTesting.functionTests.tests.lang have some examples
of deadlocks.
HTH,
Kathey
RE: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
Posted by "Bergquist, Brett" <BB...@canoga.com>.
Nothing in the Derby log other than it logging a deadlock with the statements and a lock timeout with its statements and it indicating that cleanup had started and completed.
I will enable tracing with the documented (undocumented system property). Thanks for that information!
I will check for the XA transactions the next time I reproduce this.
Maybe you could point me into the correct area to look. This seems to be triggered either through a lock timeout or a deadlock. The connection that this is occurring through is an XA connection. I see the logging of this in the server log but I am trying to find out where that would be logged from. It seems after this occurs and because of the way connection pool is being validated and recreated on error by Glassfish (configured to do so), it gets into this state. What I don't understand is why this type of error would cause the connection to appear to be invalid and I am trying to work through both the Glassfish source and the Derby source to find out. The connection is correctly handling other errors such as a duplication trying to be inserted and this does not trigger the connection to appear to be invalid. So I am trying to understand why a lock timeout or deadlock detection might do so.
This problem has only cropped up recently when they started performing multiple requests that I know have a deadlock path through them. I can fix that problem later but this is a system level problem that I need to resolve.
I really do appreciate the help and guidance and am willing to try to work though this. I have to figure this out and either patch Glassfish or Derby in any case as my customer (think very very large wireless carrier) is getting pretty PO'ed.
From: Katherine Marsden [mailto:kmarsdenderby@sbcglobal.net]
Sent: Wednesday, December 21, 2011 2:46 PM
To: derby-dev@db.apache.org
Subject: Re: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
On 12/21/2011 11:20 AM, Bergquist, Brett wrote:
I'm having some trouble getting client side tracing to work. The connections are managed by Glassfish connection pool so I don't know where to set the traceDirectory and traceLevel properties. Can these be specified as properties like the password, etc.
They can be set on the connection URL or with undocumented system properties, documented here #:)
http://wiki.apache.org/db-derby/UndocumentedDerbyBehavior
Looking at the info, again I am curious if there are corresponding server side traces in the derby.log.
Also it would be interesting to see if there are at this point any XA Transactions in need of recovery in the database.
Just exit your application and connect with ij and run:
select * from SYSCS_DIAG.TRANSACTION_TABLE ;
Re: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
Posted by Katherine Marsden <km...@sbcglobal.net>.
On 12/21/2011 11:20 AM, Bergquist, Brett wrote:
>
> I'm having some trouble getting client side tracing to work. The
> connections are managed by Glassfish connection pool so I don't know
> where to set the traceDirectory and traceLevel properties. Can these
> be specified as properties like the password, etc.
>
They can be set on the connection URL or with undocumented system
properties, documented here #:)
http://wiki.apache.org/db-derby/UndocumentedDerbyBehavior
Looking at the info, again I am curious if there are corresponding
server side traces in the derby.log.
Also it would be interesting to see if there are at this point any XA
Transactions in need of recovery in the database.
Just exit your application and connect with ij and run:
select * from SYSCS_DIAG.TRANSACTION_TABLE ;
RE: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
Posted by "Bergquist, Brett" <BB...@canoga.com>.
I'm having some trouble getting client side tracing to work. The connections are managed by Glassfish connection pool so I don't know where to set the traceDirectory and traceLevel properties. Can these be specified as properties like the password, etc.
Just a little more info as well.
Once of the threads got this error:
[#|2011-12-21T10:28:54.224-0800|INFO|sun-appserver2.1|javax.enterprise.system.container.ejb|_ThreadID=20;_ThreadName=httpSSLWorkerThread-8080-0;|
javax.ejb.TransactionRolledbackLocalException: Exception thrown from bean; nested exception is: Exception [EclipseLink-4002] (Eclipse Persistence Services - 1.1.3.v20091002-r5404): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLTransactionRollbackException: DERBY SQL error: SQLCODE: -1, SQLSTATE: 40XL2, SQLERRMC:
There was a long time where a lock could not be obtained and then next thing reported by the thread is:
[#|2011-12-21T10:28:54.269-0800|WARNING|sun-appserver2.1|javax.enterprise.system.core.transaction|_ThreadID=20;_ThreadName=httpSSLWorkerThread-8080-0;_RequestID=5e6bd3ac-516f-4d96-8018-baf3ce187f69;|JTS5064: Unexpected exception occurred while delisting the resource
org.apache.derby.client.am.XaException: XAER_RMFAIL : No current connection.
at org.apache.derby.client.net.NetXAResource.throwXAException(NetXAResource.java:766)
at org.apache.derby.client.net.NetXAResource.throwXAException(NetXAResource.java:663)
at org.apache.derby.client.net.NetXAResource.connectionClosedFailure(NetXAResource.java:933)
at org.apache.derby.client.net.NetXAResource.end(NetXAResource.java:230)
at com.sun.gjc.spi.XAResourceImpl.end(XAResourceImpl.java:100)
at com.sun.jts.jta.TransactionState.rollback(TransactionState.java:199)
at com.sun.jts.jtsxa.OTSResourceImpl.rollback(OTSResourceImpl.java:311)
....
And then later:
[#|2011-12-21T10:28:54.380-0800|SEVERE|sun-appserver2.1|javax.enterprise.system.core.transaction|_ThreadID=20;_ThreadName=httpSSLWorkerThread-8080-0;org.omg.CORBA.INTERNAL: vmcid: 0x0 minor code: 0 completed: Maybe;rollback;_RequestID=5e6bd3ac-516f-4d96-8018-baf3ce187f69;|JTS5031: Exception [org.omg.CORBA.INTERNAL: vmcid: 0x0 minor code: 0 completed: Maybe] on Resource [rollback] operation.|#]
And then later:
[#|2011-12-21T10:28:54.382-0800|WARNING|sun-appserver2.1|javax.enterprise.resource.resourceadapter|_ThreadID=20;_ThreadName=httpSSLWorkerThread-8080-0;_RequestID=5e6bd3ac-516f-4d96-8018-baf3ce187f69;|RAR7115: Unable to set ClientInfo for connection
java.sql.SQLClientInfoException: No current connection.
at org.apache.derby.client.net.NetConnection40.setClientInfo(NetConnection40.java:274)
at org.apache.derby.client.am.LogicalConnection40.setClientInfo(LogicalConnection40.java:254)
at com.sun.gjc.spi.jdbc40.ConnectionHolder40.setClientInfo(ConnectionHolder40.java:304)
at com.sun.gjc.spi.jdbc40.ConnectionHolder40.close(ConnectionHolder40.java:513)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.closeDatasourceConnection(DatabaseAccessor.java:450)
at org.eclipse.persistence.internal.databaseaccess.DatasourceAccessor.closeConnection(DatasourceAccessor.java:488)
Another thread saw:
[#|2011-12-21T10:28:54.315-0800|INFO|sun-appserver2.1|javax.enterprise.system.container.ejb|_ThreadID=75;_ThreadName=httpSSLWorkerThread-8080-3;|
javax.ejb.TransactionRolledbackLocalException: Exception thrown from bean; nested exception is: Exception [EclipseLink-4002] (Eclipse Persistence Services - 1.1.3.v20091002-r5404): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLTransactionRollbackException: DERBY SQL error: SQLCODE: -1, SQLSTATE: 40XL2, SQLERRMC:
Wed Dec 21 10:28:54 PST 2011
And then saw:
[#|2011-12-21T10:28:54.326-0800|WARNING|sun-appserver2.1|javax.enterprise.system.core.transaction|_ThreadID=75;_ThreadName=httpSSLWorkerThread-8080-3;_RequestID=b2c68a79-1eb1-497d-aab5-f3875f656ed4;|JTS5064: Unexpected exception occurred while delisting the resource
java.lang.NullPointerException
at org.apache.derby.client.net.Request.sendBytes(Request.java:1289)
at org.apache.derby.client.net.Request.flush(Request.java:1283)
at org.apache.derby.client.net.NetAgent.sendRequest(NetAgent.java:393)
at org.apache.derby.client.net.NetAgent.flush_(NetAgent.java:273)
at org.apache.derby.client.am.Agent.flowOutsideUOW(Agent.java:196)
at org.apache.derby.client.net.NetXAResource.end(NetXAResource.java:242)
at com.sun.gjc.spi.XAResourceImpl.end(XAResourceImpl.java:100)
and then saw:
[#|2011-12-21T10:28:54.389-0800|WARNING|sun-appserver2.1|javax.enterprise.resource.resourceadapter|_ThreadID=75;_ThreadName=httpSSLWorkerThread-8080-3;_RequestID=b2c68a79-1eb1-497d-aab5-f3875f656ed4;|RAR7115: Unable to set ClientInfo for connection
java.sql.SQLClientInfoException: No current connection.
at org.apache.derby.client.net.NetConnection40.setClientInfo(NetConnection40.java:274)
at org.apache.derby.client.am.LogicalConnection40.setClientInfo(LogicalConnection40.java:254)
at com.sun.gjc.spi.jdbc40.ConnectionHolder40.setClientInfo(ConnectionHolder40.java:304)
at com.sun.gjc.spi.jdbc40.ConnectionHolder40.close(ConnectionHolder40.java:513)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.closeDatasourceConnection(DatabaseAccessor.java:450)
at org.eclipse.persistence.internal.databaseaccess.DatasourceAccessor.closeConnection(DatasourceAccessor.java:488)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.closeConnection(DatabaseAccessor.java:473)
at org.eclipse.persistence.internal.databaseaccess.DatasourceAccessor.closeJTSConnection(DatasourceAccessor.java:174)
So it looks like something is closing the internals of the connections in not a good way as the NullPointerException above can only occur if the socketOutputStream passed to sendBytes is null.
Note that the Glassfish database connection pool is setup to close all connection on a connection error.
From: Bergquist, Brett [mailto:BBergquist@canoga.com]
Sent: Wednesday, December 21, 2011 11:34 AM
To: derby-dev@db.apache.org
Subject: RE: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
I cannot upgrade to 10.8.8.2 at this point as this is in production without upgrade possibility. A also at this point, I don't want to change to many variables and have the problem disappear. Right now I can get it to reproduce pretty readily.
I was able to connect with a debug and indeed the thread is blocking on the "java.net.SocketInputStream.read". Through the debugger if I force an interrupt, I returns from the socket call with an I/O error. So does this mean that the problem is in the derby server side since the socket is valid and not closed before I issue the interrupt?
I am going to enable tracing and see what I find.
From: Katherine Marsden [mailto:kmarsdenderby@sbcglobal.net]
Sent: Wednesday, December 21, 2011 11:14 AM
To: derby-dev@db.apache.org
Subject: Re: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
On 12/21/2011 6:14 AM, Bergquist, Brett wrote:
I have a problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1 but at this point I cannot tell if it is a Derby problem or a Glassfish problem so I am looking for some guidance.
A database pool is created using the "org.apache.derby.jdbc.ClientXADataSource" XADataSource. Connection validation is turned on and uses "auto-commit" to validate the connection and attempts to close all connections on any validation failure.
The deadlock is triggered when I have a couple of transactions that cause a Derby locks deadlock. Deadlock detection and tracing is turned on in Derby and it correctly detects the locking deadlock and terminates one of the transactions. Soon after this, however, Glassfish becomes stuck. Here is the only reference to Derby in the "jstack" output of Glassfish:
"PMCollectionActivationManager.newPmTestResult" daemon prio=3 tid=0x0000000103926800 nid=0xaf runnable [0xfffffffc218fa000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.client.net.Reply.fill(Unknown Source)
at org.apache.derby.client.net.Reply.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.client.net.Reply.readDssHeader(Unknown Source)
at org.apache.derby.client.net.Reply.startSameIdChainParse(Unknown Source)
at org.apache.derby.client.net.NetXAConnectionReply.readLocalXACommit(Unknown Source)
at org.apache.derby.client.net.NetXAConnection.readLocalXACommit_(Unknown Source)
at org.apache.derby.client.net.NetXAConnection.readCommit(Unknown Source)
at org.apache.derby.client.net.NetConnection.readXACommit_(Unknown Source)
at org.apache.derby.client.am.Connection.readCommit(Unknown Source)
at org.apache.derby.client.am.Connection.readAutoCommit(Unknown Source)
at org.apache.derby.client.am.Connection.flowClose(Unknown Source)
at org.apache.derby.client.am.Connection.closeResourcesX(Unknown Source)
at org.apache.derby.client.am.Connection.closeResources(Unknown Source)
- locked <0xfffffffc91831e18> (a org.apache.derby.client.net.NetConnection40)
at org.apache.derby.client.net.NetConnection.closeResources(Unknown Source)
- locked <0xfffffffc91831e18> (a org.apache.derby.client.net.NetConnection40)
at org.apache.derby.client.ClientPooledConnection.close(Unknown Source)
- locked <0xfffffffc91831cf8> (a org.apache.derby.client.ClientXAConnection40)
at org.apache.derby.client.ClientXAConnection.close(Unknown Source)
- locked <0xfffffffc91831cf8> (a org.apache.derby.client.ClientXAConnection40)
at com.sun.gjc.spi.ManagedConnection.destroy(ManagedConnection.java:276)
First thing I would suggest is move up to 10.8.2.2 just to get running on the latest.
I wonder is there indication in the derby.log what went wrong here? If not, I would suggest enabling client and server side DRDA tracing to get a clue as to what went wrong with the protocol here.
http://wiki.apache.org/db-derby/ProtocolDebuggingTips
Maybe there is some issue with the logic that dermines if autocommit will generate flow.
// precondition: autoCommit_ is true
void readAutoCommit() throws SqlException {
if (willAutoCommitGenerateFlow()) {
readCommit();
}
}
public boolean willAutoCommitGenerateFlow() throws org.apache.derby.client.am.SqlException {
if (!autoCommit_) {
return false;
}
if (! allowLocalCommitRollback_()) {
return false;
}
return true;
}
Re: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
Posted by Katherine Marsden <km...@sbcglobal.net>.
On 12/21/2011 8:34 AM, Bergquist, Brett wrote:
>
> I cannot upgrade to 10.8.8.2 at this point as this is in production
> without upgrade possibility. A also at this point, I don't want to
> change to many variables and have the problem disappear. Right now I
> can get it to reproduce pretty readily.
>
> I was able to connect with a debug and indeed the thread is blocking
> on the "java.net.SocketInputStream.read". Through the debugger if I
> force an interrupt, I returns from the socket call with an I/O error.
> So does this mean that the problem is in the derby server side since
> the socket is valid and not closed before I issue the interrupt?
>
If you are not seeing the expected SYNCCRD response from the server in
your trace, I would expect it is either a) A problem on the server side
that it is not being sent or b) a problem on the client side in terms of
determining wither this should flow. Was there anything of interest in
the derby.log?
I would suggest you go ahead and file a Jira issue so you can attach the
trace or trace excerpts and reproduction when you get one. Please put a
pointer to this email thread in there.
Thanks
Kathey
RE: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
Posted by "Bergquist, Brett" <BB...@canoga.com>.
I cannot upgrade to 10.8.8.2 at this point as this is in production without upgrade possibility. A also at this point, I don't want to change to many variables and have the problem disappear. Right now I can get it to reproduce pretty readily.
I was able to connect with a debug and indeed the thread is blocking on the "java.net.SocketInputStream.read". Through the debugger if I force an interrupt, I returns from the socket call with an I/O error. So does this mean that the problem is in the derby server side since the socket is valid and not closed before I issue the interrupt?
I am going to enable tracing and see what I find.
From: Katherine Marsden [mailto:kmarsdenderby@sbcglobal.net]
Sent: Wednesday, December 21, 2011 11:14 AM
To: derby-dev@db.apache.org
Subject: Re: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
On 12/21/2011 6:14 AM, Bergquist, Brett wrote:
I have a problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1 but at this point I cannot tell if it is a Derby problem or a Glassfish problem so I am looking for some guidance.
A database pool is created using the "org.apache.derby.jdbc.ClientXADataSource" XADataSource. Connection validation is turned on and uses "auto-commit" to validate the connection and attempts to close all connections on any validation failure.
The deadlock is triggered when I have a couple of transactions that cause a Derby locks deadlock. Deadlock detection and tracing is turned on in Derby and it correctly detects the locking deadlock and terminates one of the transactions. Soon after this, however, Glassfish becomes stuck. Here is the only reference to Derby in the "jstack" output of Glassfish:
"PMCollectionActivationManager.newPmTestResult" daemon prio=3 tid=0x0000000103926800 nid=0xaf runnable [0xfffffffc218fa000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.derby.client.net.Reply.fill(Unknown Source)
at org.apache.derby.client.net.Reply.ensureALayerDataInBuffer(Unknown Source)
at org.apache.derby.client.net.Reply.readDssHeader(Unknown Source)
at org.apache.derby.client.net.Reply.startSameIdChainParse(Unknown Source)
at org.apache.derby.client.net.NetXAConnectionReply.readLocalXACommit(Unknown Source)
at org.apache.derby.client.net.NetXAConnection.readLocalXACommit_(Unknown Source)
at org.apache.derby.client.net.NetXAConnection.readCommit(Unknown Source)
at org.apache.derby.client.net.NetConnection.readXACommit_(Unknown Source)
at org.apache.derby.client.am.Connection.readCommit(Unknown Source)
at org.apache.derby.client.am.Connection.readAutoCommit(Unknown Source)
at org.apache.derby.client.am.Connection.flowClose(Unknown Source)
at org.apache.derby.client.am.Connection.closeResourcesX(Unknown Source)
at org.apache.derby.client.am.Connection.closeResources(Unknown Source)
- locked <0xfffffffc91831e18> (a org.apache.derby.client.net.NetConnection40)
at org.apache.derby.client.net.NetConnection.closeResources(Unknown Source)
- locked <0xfffffffc91831e18> (a org.apache.derby.client.net.NetConnection40)
at org.apache.derby.client.ClientPooledConnection.close(Unknown Source)
- locked <0xfffffffc91831cf8> (a org.apache.derby.client.ClientXAConnection40)
at org.apache.derby.client.ClientXAConnection.close(Unknown Source)
- locked <0xfffffffc91831cf8> (a org.apache.derby.client.ClientXAConnection40)
at com.sun.gjc.spi.ManagedConnection.destroy(ManagedConnection.java:276)
First thing I would suggest is move up to 10.8.2.2 just to get running on the latest.
I wonder is there indication in the derby.log what went wrong here? If not, I would suggest enabling client and server side DRDA tracing to get a clue as to what went wrong with the protocol here.
http://wiki.apache.org/db-derby/ProtocolDebuggingTips
Maybe there is some issue with the logic that dermines if autocommit will generate flow.
// precondition: autoCommit_ is true
void readAutoCommit() throws SqlException {
if (willAutoCommitGenerateFlow()) {
readCommit();
}
}
public boolean willAutoCommitGenerateFlow() throws org.apache.derby.client.am.SqlException {
if (!autoCommit_) {
return false;
}
if (! allowLocalCommitRollback_()) {
return false;
}
return true;
}
Re: Problem with a deadlock with Derby 10.8.1.2 and Glassfish V2.1.1
Posted by Katherine Marsden <km...@sbcglobal.net>.
On 12/21/2011 6:14 AM, Bergquist, Brett wrote:
>
> I have a problem with a deadlock with Derby 10.8.1.2 and Glassfish
> V2.1.1 but at this point I cannot tell if it is a Derby problem or a
> Glassfish problem so I am looking for some guidance.
>
> A database pool is created using the
> "org.apache.derby.jdbc.ClientXADataSource" XADataSource. Connection
> validation is turned on and uses "auto-commit" to validate the
> connection and attempts to close all connections on any validation
> failure.
>
> The deadlock is triggered when I have a couple of transactions that
> cause a Derby locks deadlock. Deadlock detection and tracing is
> turned on in Derby and it correctly detects the locking deadlock and
> terminates one of the transactions. Soon after this, however,
> Glassfish becomes stuck. Here is the only reference to Derby in the
> "jstack" output of Glassfish:
>
> "PMCollectionActivationManager.newPmTestResult" daemon prio=3
> tid=0x0000000103926800 nid=0xaf runnable [0xfffffffc218fa000]
>
> java.lang.Thread.State: RUNNABLE
>
> at java.net.SocketInputStream.socketRead0(Native Method)
>
> at
> java.net.SocketInputStream.read(SocketInputStream.java:129)
>
> at org.apache.derby.client.net.Reply.fill(Unknown Source)
>
> at
> org.apache.derby.client.net.Reply.ensureALayerDataInBuffer(Unknown Source)
>
> at
> org.apache.derby.client.net.Reply.readDssHeader(Unknown Source)
>
> at
> org.apache.derby.client.net.Reply.startSameIdChainParse(Unknown Source)
>
> at
> org.apache.derby.client.net.NetXAConnectionReply.readLocalXACommit(Unknown
> Source)
>
> at
> org.apache.derby.client.net.NetXAConnection.readLocalXACommit_(Unknown
> Source)
>
> at
> org.apache.derby.client.net.NetXAConnection.readCommit(Unknown Source)
>
> at
> org.apache.derby.client.net.NetConnection.readXACommit_(Unknown Source)
>
> at
> org.apache.derby.client.am.Connection.readCommit(Unknown Source)
>
> at
> org.apache.derby.client.am.Connection.readAutoCommit(Unknown Source)
>
> at
> org.apache.derby.client.am.Connection.flowClose(Unknown Source)
>
> at
> org.apache.derby.client.am.Connection.closeResourcesX(Unknown Source)
>
> at
> org.apache.derby.client.am.Connection.closeResources(Unknown Source)
>
> - locked <0xfffffffc91831e18> (a
> org.apache.derby.client.net.NetConnection40)
>
> at
> org.apache.derby.client.net.NetConnection.closeResources(Unknown Source)
>
> - locked <0xfffffffc91831e18> (a
> org.apache.derby.client.net.NetConnection40)
>
> at
> org.apache.derby.client.ClientPooledConnection.close(Unknown Source)
>
> - locked <0xfffffffc91831cf8> (a
> org.apache.derby.client.ClientXAConnection40)
>
> at
> org.apache.derby.client.ClientXAConnection.close(Unknown Source)
>
> - locked <0xfffffffc91831cf8> (a
> org.apache.derby.client.ClientXAConnection40)
>
> at
> com.sun.gjc.spi.ManagedConnection.destroy(ManagedConnection.java:276)
>
First thing I would suggest is move up to 10.8.2.2 just to get running
on the latest.
I wonder is there indication in the derby.log what went wrong here? If
not, I would suggest enabling client and server side DRDA tracing to get
a clue as to what went wrong with the protocol here.
http://wiki.apache.org/db-derby/ProtocolDebuggingTips
Maybe there is some issue with the logic that dermines if autocommit
will generate flow.
// precondition: autoCommit_ is true
void readAutoCommit() throws SqlException {
if (willAutoCommitGenerateFlow()) {
readCommit();
}
}
public boolean willAutoCommitGenerateFlow() throws
org.apache.derby.client.am.SqlException {
if (!autoCommit_) {
return false;
}
if (! allowLocalCommitRollback_()) {
return false;
}
return true;
}