You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@activemq.apache.org by "Dan Groves (JIRA)" <ji...@apache.org> on 2017/04/21 21:30:04 UTC
[jira] [Updated] (AMQ-6660) Deadlock closing a connection due to an
exception
[ https://issues.apache.org/jira/browse/AMQ-6660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dan Groves updated AMQ-6660:
----------------------------
Description:
While recovering from a period of network instability, our application deadlocked in ActiveMQ code.
{noformat}
Found one Java-level deadlock:
=============================
"ActiveMQ Connection Executor: tcp:///<ip address>:61616@51026":
waiting to lock monitor 0x00007f60fc0060b8 (object 0x0000000773f00308, a java.lang.Object),
which is held by "privateJmsInPriority.container-1"
"privateJmsInPriority.container-1":
waiting to lock monitor 0x00007f610840dbc8 (object 0x0000000773f99930, a org.apache.activemq.ActiveMQConnection),
which is held by "ActiveMQ Connection Executor: tcp:///<ip address>:61616@51026"
Java stack information for the threads listed above:
===================================================
"ActiveMQ Connection Executor: tcp:///<ip address>:61616@51026":
at org.apache.activemq.SimplePriorityMessageDispatchChannel.close(SimplePriorityMessageDispatchChannel.java:154)
- waiting to lock <0x0000000773f00308> (a java.lang.Object)
at org.apache.activemq.ActiveMQMessageConsumer.dispose(ActiveMQMessageConsumer.java:800)
at org.apache.activemq.ActiveMQSession.dispose(ActiveMQSession.java:689)
- locked <0x0000000773f003d8> (a org.apache.activemq.ActiveMQSession)
at org.apache.activemq.ActiveMQConnection.close(ActiveMQConnection.java:659)
- locked <0x0000000773f99930> (a org.apache.activemq.ActiveMQConnection)
at org.springframework.jms.connection.SingleConnectionFactory.closeConnection(SingleConnectionFactory.java:456)
at org.springframework.jms.connection.SingleConnectionFactory.resetConnection(SingleConnectionFactory.java:345)
- locked <0x0000000773f9a590> (a java.lang.Object)
at org.springframework.jms.connection.CachingConnectionFactory.resetConnection(CachingConnectionFactory.java:207)
at org.springframework.jms.connection.SingleConnectionFactory.onException(SingleConnectionFactory.java:323)
at org.springframework.jms.connection.SingleConnectionFactory$AggregatedExceptionListener.onException(SingleConnectionFactory.java:673)
- locked <0x0000000773f9a590> (a java.lang.Object)
at org.apache.activemq.ActiveMQConnection$5.run(ActiveMQConnection.java:1976)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
"privateJmsInPriority.container-1":
at org.apache.activemq.ActiveMQConnection.getScheduler(ActiveMQConnection.java:2519)
- waiting to lock <0x0000000773f99930> (a org.apache.activemq.ActiveMQConnection)
at org.apache.activemq.ActiveMQSession.getScheduler(ActiveMQSession.java:2074)
at org.apache.activemq.ActiveMQMessageConsumer.rollback(ActiveMQMessageConsumer.java:1241)
- locked <0x0000000773f01030> (a java.util.LinkedList)
- locked <0x0000000773f00308> (a java.lang.Object)
at org.apache.activemq.ActiveMQMessageConsumer$5.afterRollback(ActiveMQMessageConsumer.java:1032)
at org.apache.activemq.TransactionContext.afterRollback(TransactionContext.java:157)
at org.apache.activemq.TransactionContext.commit(TransactionContext.java:332)
at org.apache.activemq.ActiveMQSession.commit(ActiveMQSession.java:561)
at sun.reflect.GeneratedMethodAccessor897.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.jms.connection.CachingConnectionFactory$CachedSessionInvocationHandler.invoke(CachingConnectionFactory.java:386)
at com.sun.proxy.$Proxy49.commit(Unknown Source)
at org.springframework.jms.support.JmsUtils.commitIfNecessary(JmsUtils.java:217)
at org.springframework.jms.listener.AbstractMessageListenerContainer.commitIfNecessary(AbstractMessageListenerContainer.java:761)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:665)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:319)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1166)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1158)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1055)
at java.lang.Thread.run(Thread.java:745)
Found 1 deadlock.
{noformat}
was:
While recovering from a period of network instability, our application deadlocked in ActiveMQ code.
{noformat}
Found one Java-level deadlock:
=============================
"ActiveMQ Connection Executor: tcp:///20.172.19.10:61616@51026":
waiting to lock monitor 0x00007f60fc0060b8 (object 0x0000000773f00308, a java.lang.Object),
which is held by "privateJmsInPriority.container-1"
"privateJmsInPriority.container-1":
waiting to lock monitor 0x00007f610840dbc8 (object 0x0000000773f99930, a org.apache.activemq.ActiveMQConnection),
which is held by "ActiveMQ Connection Executor: tcp:///20.172.19.10:61616@51026"
Java stack information for the threads listed above:
===================================================
"ActiveMQ Connection Executor: tcp:///20.172.19.10:61616@51026":
at org.apache.activemq.SimplePriorityMessageDispatchChannel.close(SimplePriorityMessageDispatchChannel.java:154)
- waiting to lock <0x0000000773f00308> (a java.lang.Object)
at org.apache.activemq.ActiveMQMessageConsumer.dispose(ActiveMQMessageConsumer.java:800)
at org.apache.activemq.ActiveMQSession.dispose(ActiveMQSession.java:689)
- locked <0x0000000773f003d8> (a org.apache.activemq.ActiveMQSession)
at org.apache.activemq.ActiveMQConnection.close(ActiveMQConnection.java:659)
- locked <0x0000000773f99930> (a org.apache.activemq.ActiveMQConnection)
at org.springframework.jms.connection.SingleConnectionFactory.closeConnection(SingleConnectionFactory.java:456)
at org.springframework.jms.connection.SingleConnectionFactory.resetConnection(SingleConnectionFactory.java:345)
- locked <0x0000000773f9a590> (a java.lang.Object)
at org.springframework.jms.connection.CachingConnectionFactory.resetConnection(CachingConnectionFactory.java:207)
at org.springframework.jms.connection.SingleConnectionFactory.onException(SingleConnectionFactory.java:323)
at org.springframework.jms.connection.SingleConnectionFactory$AggregatedExceptionListener.onException(SingleConnectionFactory.java:673)
- locked <0x0000000773f9a590> (a java.lang.Object)
at org.apache.activemq.ActiveMQConnection$5.run(ActiveMQConnection.java:1976)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
"privateJmsInPriority.container-1":
at org.apache.activemq.ActiveMQConnection.getScheduler(ActiveMQConnection.java:2519)
- waiting to lock <0x0000000773f99930> (a org.apache.activemq.ActiveMQConnection)
at org.apache.activemq.ActiveMQSession.getScheduler(ActiveMQSession.java:2074)
at org.apache.activemq.ActiveMQMessageConsumer.rollback(ActiveMQMessageConsumer.java:1241)
- locked <0x0000000773f01030> (a java.util.LinkedList)
- locked <0x0000000773f00308> (a java.lang.Object)
at org.apache.activemq.ActiveMQMessageConsumer$5.afterRollback(ActiveMQMessageConsumer.java:1032)
at org.apache.activemq.TransactionContext.afterRollback(TransactionContext.java:157)
at org.apache.activemq.TransactionContext.commit(TransactionContext.java:332)
at org.apache.activemq.ActiveMQSession.commit(ActiveMQSession.java:561)
at sun.reflect.GeneratedMethodAccessor897.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.jms.connection.CachingConnectionFactory$CachedSessionInvocationHandler.invoke(CachingConnectionFactory.java:386)
at com.sun.proxy.$Proxy49.commit(Unknown Source)
at org.springframework.jms.support.JmsUtils.commitIfNecessary(JmsUtils.java:217)
at org.springframework.jms.listener.AbstractMessageListenerContainer.commitIfNecessary(AbstractMessageListenerContainer.java:761)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:665)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:319)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1166)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1158)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1055)
at java.lang.Thread.run(Thread.java:745)
Found 1 deadlock.
{noformat}
> Deadlock closing a connection due to an exception
> -------------------------------------------------
>
> Key: AMQ-6660
> URL: https://issues.apache.org/jira/browse/AMQ-6660
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.8.0
> Reporter: Dan Groves
>
> While recovering from a period of network instability, our application deadlocked in ActiveMQ code.
> {noformat}
> Found one Java-level deadlock:
> =============================
> "ActiveMQ Connection Executor: tcp:///<ip address>:61616@51026":
> waiting to lock monitor 0x00007f60fc0060b8 (object 0x0000000773f00308, a java.lang.Object),
> which is held by "privateJmsInPriority.container-1"
> "privateJmsInPriority.container-1":
> waiting to lock monitor 0x00007f610840dbc8 (object 0x0000000773f99930, a org.apache.activemq.ActiveMQConnection),
> which is held by "ActiveMQ Connection Executor: tcp:///<ip address>:61616@51026"
> Java stack information for the threads listed above:
> ===================================================
> "ActiveMQ Connection Executor: tcp:///<ip address>:61616@51026":
> at org.apache.activemq.SimplePriorityMessageDispatchChannel.close(SimplePriorityMessageDispatchChannel.java:154)
> - waiting to lock <0x0000000773f00308> (a java.lang.Object)
> at org.apache.activemq.ActiveMQMessageConsumer.dispose(ActiveMQMessageConsumer.java:800)
> at org.apache.activemq.ActiveMQSession.dispose(ActiveMQSession.java:689)
> - locked <0x0000000773f003d8> (a org.apache.activemq.ActiveMQSession)
> at org.apache.activemq.ActiveMQConnection.close(ActiveMQConnection.java:659)
> - locked <0x0000000773f99930> (a org.apache.activemq.ActiveMQConnection)
> at org.springframework.jms.connection.SingleConnectionFactory.closeConnection(SingleConnectionFactory.java:456)
> at org.springframework.jms.connection.SingleConnectionFactory.resetConnection(SingleConnectionFactory.java:345)
> - locked <0x0000000773f9a590> (a java.lang.Object)
> at org.springframework.jms.connection.CachingConnectionFactory.resetConnection(CachingConnectionFactory.java:207)
> at org.springframework.jms.connection.SingleConnectionFactory.onException(SingleConnectionFactory.java:323)
> at org.springframework.jms.connection.SingleConnectionFactory$AggregatedExceptionListener.onException(SingleConnectionFactory.java:673)
> - locked <0x0000000773f9a590> (a java.lang.Object)
> at org.apache.activemq.ActiveMQConnection$5.run(ActiveMQConnection.java:1976)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> "privateJmsInPriority.container-1":
> at org.apache.activemq.ActiveMQConnection.getScheduler(ActiveMQConnection.java:2519)
> - waiting to lock <0x0000000773f99930> (a org.apache.activemq.ActiveMQConnection)
> at org.apache.activemq.ActiveMQSession.getScheduler(ActiveMQSession.java:2074)
> at org.apache.activemq.ActiveMQMessageConsumer.rollback(ActiveMQMessageConsumer.java:1241)
> - locked <0x0000000773f01030> (a java.util.LinkedList)
> - locked <0x0000000773f00308> (a java.lang.Object)
> at org.apache.activemq.ActiveMQMessageConsumer$5.afterRollback(ActiveMQMessageConsumer.java:1032)
> at org.apache.activemq.TransactionContext.afterRollback(TransactionContext.java:157)
> at org.apache.activemq.TransactionContext.commit(TransactionContext.java:332)
> at org.apache.activemq.ActiveMQSession.commit(ActiveMQSession.java:561)
> at sun.reflect.GeneratedMethodAccessor897.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.springframework.jms.connection.CachingConnectionFactory$CachedSessionInvocationHandler.invoke(CachingConnectionFactory.java:386)
> at com.sun.proxy.$Proxy49.commit(Unknown Source)
> at org.springframework.jms.support.JmsUtils.commitIfNecessary(JmsUtils.java:217)
> at org.springframework.jms.listener.AbstractMessageListenerContainer.commitIfNecessary(AbstractMessageListenerContainer.java:761)
> at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:665)
> at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:319)
> at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
> at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1166)
> at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1158)
> at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1055)
> at java.lang.Thread.run(Thread.java:745)
> Found 1 deadlock.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)