You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@activemq.apache.org by "Frederik Petersen (JIRA)" <ji...@apache.org> on 2019/03/13 15:39:00 UTC
[jira] [Updated] (ARTEMIS-2276) Deadlock between QueueImpl and
TransactionImpl
[ https://issues.apache.org/jira/browse/ARTEMIS-2276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Frederik Petersen updated ARTEMIS-2276:
---------------------------------------
Description:
Hi,
we just experienced another deadlock in artemis.
I am attaching the thread dump. The issue seems to be here:
{code:java}
"Thread-0 (activemq-netty-threads)" Id=43 BLOCKED on org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1 owned by "Thread-7 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$5@5bda80bf)" Id=87
at org.apache.activemq.artemis.core.server.impl.QueueImpl.getRefsOperation(QueueImpl.java:1332)
- blocked on org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1
at org.apache.activemq.artemis.core.server.impl.QueueImpl.getRefsOperation(QueueImpl.java:1328)
at org.apache.activemq.artemis.core.server.impl.QueueImpl.acknowledge(QueueImpl.java:1295)
at org.apache.activemq.artemis.core.server.impl.MessageReferenceImpl.acknowledge(MessageReferenceImpl.java:207)
at org.apache.activemq.artemis.core.server.impl.MessageReferenceImpl.acknowledge(MessageReferenceImpl.java:199)
at org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.acknowledge(ServerConsumerImpl.java:874)
- locked org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl@31c578da
at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.acknowledge(ServerSessionImpl.java:891)
at org.apache.activemq.artemis.core.protocol.stomp.StompSession$1.run(StompSession.java:159)
at org.apache.activemq.artemis.core.protocol.stomp.StompSession.afterDelivery(StompSession.java:111)
at org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:469)
at org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:2936)
at org.apache.activemq.artemis.core.server.impl.QueueImpl.deliverDirect(QueueImpl.java:2921)
- locked org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217
[...]
"Thread-7 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$5@5bda80bf)" Id=87 BLOCKED on org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217 owned by "Thread-0 (activemq-netty-threads)" Id=43
at org.apache.activemq.artemis.core.server.impl.RefsOperation.afterCommit(RefsOperation.java:165)
- blocked on org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217
at org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.afterCommit(TransactionImpl.java:546)
- locked org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1
[...]
{code}
Thread-0 locks QueueImpl@6b7ed217.
Thread-7 locks TransactionImpl@196f00e1.
Thread-0 waits for TransactionImpl@196f00e1.
Thread-7 waits for QueueImpl@6b7ed217.
We had a few mqtt producers, stomp consumers and stomp producers. A few hundred items/sec. deliverDirect disabled on mqtt as suggested by [~clebertsuconic] here: [https://github.com/apache/activemq-artemis/pull/2577#issuecomment-472070556]
But I guess this dead lock is not the same as the one discussed there.
Thanks for your help.
was:
Hi,
we just experienced another deadlock in artemis.
I am attaching the thread dump. The issue seems to be here:
{code:java}
"Thread-0 (activemq-netty-threads)" Id=43 BLOCKED on org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1 owned by "Thread-7 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$5@5bda80bf)" Id=87
at org.apache.activemq.artemis.core.server.impl.QueueImpl.getRefsOperation(QueueImpl.java:1332)
- blocked on org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1
at org.apache.activemq.artemis.core.server.impl.QueueImpl.getRefsOperation(QueueImpl.java:1328)
at org.apache.activemq.artemis.core.server.impl.QueueImpl.acknowledge(QueueImpl.java:1295)
at org.apache.activemq.artemis.core.server.impl.MessageReferenceImpl.acknowledge(MessageReferenceImpl.java:207)
at org.apache.activemq.artemis.core.server.impl.MessageReferenceImpl.acknowledge(MessageReferenceImpl.java:199)
at org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.acknowledge(ServerConsumerImpl.java:874)
- locked org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl@31c578da
at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.acknowledge(ServerSessionImpl.java:891)
at org.apache.activemq.artemis.core.protocol.stomp.StompSession$1.run(StompSession.java:159)
at org.apache.activemq.artemis.core.protocol.stomp.StompSession.afterDelivery(StompSession.java:111)
at org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:469)
at org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:2936)
at org.apache.activemq.artemis.core.server.impl.QueueImpl.deliverDirect(QueueImpl.java:2921)
- locked org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217
[...]
"Thread-7 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$5@5bda80bf)" Id=87 BLOCKED on org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217 owned by "Thread-0 (activemq-netty-threads)" Id=43
at org.apache.activemq.artemis.core.server.impl.RefsOperation.afterCommit(RefsOperation.java:165)
- blocked on org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217
at org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.afterCommit(TransactionImpl.java:546)
- locked org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1
[...]
{code}
Thread-0 locks QueueImpl@6b7ed217.
Thread-7 locks TransactionImpl@196f00e1.
Thread-0 waits for TransactionImpl@196f00e1.
Thread-7 waits for QueueImpl@6b7ed217.
We had a few mqtt producers, stomp consumers and stomp producers. A few hundred items/sec. deliverDirect disabled on mqtt as suggested by [~clebertsuconic]
here: [https://github.com/apache/activemq-artemis/pull/2577#issuecomment-472070556]
But I guess this dead lockis not the same as the one discussed there.
Thanks for your help.
> Deadlock between QueueImpl and TransactionImpl
> ----------------------------------------------
>
> Key: ARTEMIS-2276
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2276
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Components: Broker
> Affects Versions: 2.6.4
> Reporter: Frederik Petersen
> Priority: Major
> Attachments: artemis_thread_dump_2.log
>
>
> Hi,
> we just experienced another deadlock in artemis.
> I am attaching the thread dump. The issue seems to be here:
>
> {code:java}
> "Thread-0 (activemq-netty-threads)" Id=43 BLOCKED on org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1 owned by "Thread-7 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$5@5bda80bf)" Id=87
> at org.apache.activemq.artemis.core.server.impl.QueueImpl.getRefsOperation(QueueImpl.java:1332)
> - blocked on org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1
> at org.apache.activemq.artemis.core.server.impl.QueueImpl.getRefsOperation(QueueImpl.java:1328)
> at org.apache.activemq.artemis.core.server.impl.QueueImpl.acknowledge(QueueImpl.java:1295)
> at org.apache.activemq.artemis.core.server.impl.MessageReferenceImpl.acknowledge(MessageReferenceImpl.java:207)
> at org.apache.activemq.artemis.core.server.impl.MessageReferenceImpl.acknowledge(MessageReferenceImpl.java:199)
> at org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.acknowledge(ServerConsumerImpl.java:874)
> - locked org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl@31c578da
> at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.acknowledge(ServerSessionImpl.java:891)
> at org.apache.activemq.artemis.core.protocol.stomp.StompSession$1.run(StompSession.java:159)
> at org.apache.activemq.artemis.core.protocol.stomp.StompSession.afterDelivery(StompSession.java:111)
> at org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:469)
> at org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:2936)
> at org.apache.activemq.artemis.core.server.impl.QueueImpl.deliverDirect(QueueImpl.java:2921)
> - locked org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217
> [...]
> "Thread-7 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$5@5bda80bf)" Id=87 BLOCKED on org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217 owned by "Thread-0 (activemq-netty-threads)" Id=43
> at org.apache.activemq.artemis.core.server.impl.RefsOperation.afterCommit(RefsOperation.java:165)
> - blocked on org.apache.activemq.artemis.core.server.impl.QueueImpl@6b7ed217
> at org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.afterCommit(TransactionImpl.java:546)
> - locked org.apache.activemq.artemis.core.transaction.impl.TransactionImpl@196f00e1
> [...]
> {code}
>
> Thread-0 locks QueueImpl@6b7ed217.
> Thread-7 locks TransactionImpl@196f00e1.
> Thread-0 waits for TransactionImpl@196f00e1.
> Thread-7 waits for QueueImpl@6b7ed217.
> We had a few mqtt producers, stomp consumers and stomp producers. A few hundred items/sec. deliverDirect disabled on mqtt as suggested by [~clebertsuconic] here: [https://github.com/apache/activemq-artemis/pull/2577#issuecomment-472070556]
> But I guess this dead lock is not the same as the one discussed there.
> Thanks for your help.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)