You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Dennis Klinkott (JIRA)" <ji...@apache.org> on 2010/05/25 19:50:51 UTC
[jira] Created: (AMQ-2749) Broker hangs in TopicSubscription.add
Broker hangs in TopicSubscription.add
-------------------------------------
Key: AMQ-2749
URL: https://issues.apache.org/activemq/browse/AMQ-2749
Project: ActiveMQ
Issue Type: Bug
Components: Broker
Affects Versions: 5.3.1
Environment: red had enterprise
Reporter: Dennis Klinkott
Our broker suddenly hang in production environment. We created a bunch of jstacks to see what the reason is.
The jstacks show that the broker waits with many threads in doMessageSend:
e.g.
"ActiveMQ Transport: tcp:///10.231.233.117:51088" daemon prio=10 tid=0x00002aab84016800 nid=0x1f19 waiting for monitor entry [0x000000006a8f2000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:402)
- waiting to lock <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
at org.apache.activemq.advisory.AdvisoryBroker.addDestinationInfo(AdvisoryBroker.java:173)
at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
at org.apache.activemq.broker.MutableBrokerFilter.addDestinationInfo(MutableBrokerFilter.java:221)
at org.apache.activemq.broker.TransportConnection.processAddDestination(TransportConnection.java:467)
at org.apache.activemq.command.DestinationInfo.visit(DestinationInfo.java:122)
at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
at org.apache.activemq.transport.stomp.ProtocolConverter.createTempQueue(ProtocolConverter.java:611)
at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertDestination(LegacyFrameTranslator.java:130)
at org.apache.activemq.transport.stomp.FrameTranslator$Helper.copyStandardHeadersFromFrameToMessage(FrameTranslator.java:93)
at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertFrame(LegacyFrameTranslator.java:54)
at org.apache.activemq.transport.stomp.ProtocolConverter.convertMessage(ProtocolConverter.java:591)
at org.apache.activemq.transport.stomp.ProtocolConverter.onStompSend(ProtocolConverter.java:231)
at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:173)
at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
at java.lang.Thread.run(Thread.java:619)
It is waiting to lock 0x00002aaab46f8938. This is the thread that holds the lock:
"ActiveMQ Transport: tcp:///10.231.233.117:43020" daemon prio=10 tid=0x00000000608d8800 nid=0xc6e in Object.wait() [0x000000007d41c000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.apache.activemq.broker.region.TopicSubscription.add(TopicSubscription.java:106)
- locked <0x00002aaab57d34e8> (a java.lang.Object)
at org.apache.activemq.broker.region.policy.SimpleDispatchPolicy.dispatch(SimpleDispatchPolicy.java:49)
- locked <0x00002aaab46f98e0> (a java.util.concurrent.CopyOnWriteArrayList)
at org.apache.activemq.broker.region.Topic.dispatch(Topic.java:585)
at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:443)
- locked <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
at org.apache.activemq.advisory.AdvisoryBroker.removeDestination(AdvisoryBroker.java:185)
at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
at org.apache.activemq.broker.MutableBrokerFilter.removeDestination(MutableBrokerFilter.java:153)
at org.apache.activemq.broker.TransportConnection.processRemoveConnection(TransportConnection.java:713)
- locked <0x00002aaac32b10c0> (a org.apache.activemq.broker.jmx.ManagedTransportConnection)
at org.apache.activemq.command.RemoveInfo.visit(RemoveInfo.java:72)
at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
at org.apache.activemq.transport.stomp.ProtocolConverter.onStompDisconnect(ProtocolConverter.java:545)
at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:189)
at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
at java.lang.Thread.run(Thread.java:619)
So it is the client 10.231.233.117 that has a stomp-connection to the broker, causes a TopicSubscription.add and causes the broker freeze.
This is how we configure our openwire transport-connector in the activemq.xml:
<transportConnectors>
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
<transportConnector name="stomp" uri="stomp://0.0.0.0:61613"/>
</transportConnectors>
We took jstack-dumps with a distance of 9 minutes. Both are attached. In both dumps it is the same TopicSubscription.add that causes the hang.
How can we solve this issue? Looks like the connection should run into an timeout but doesn't. Is this an activemq bug? Can we somehow set the broker openwire timeout parameters?
Thanks for any hints.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-2749) Broker hangs in TopicSubscription.add
Posted by "Dennis Klinkott (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Klinkott updated AMQ-2749:
---------------------------------
Attachment: jmx console screenshot.jpg
> Broker hangs in TopicSubscription.add
> -------------------------------------
>
> Key: AMQ-2749
> URL: https://issues.apache.org/activemq/browse/AMQ-2749
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.1
> Environment: red had enterprise
> Reporter: Dennis Klinkott
> Attachments: activemq.xml, jmx console screenshot.jpg, jstack.2010-05-25-14-01-30, jstack.2010-05-25-14-10-54
>
>
> Our broker suddenly hang in production environment. We created a bunch of jstacks to see what the reason is.
> The jstacks show that the broker waits with many threads in doMessageSend:
> e.g.
> "ActiveMQ Transport: tcp:///10.231.233.117:51088" daemon prio=10 tid=0x00002aab84016800 nid=0x1f19 waiting for monitor entry [0x000000006a8f2000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:402)
> - waiting to lock <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
> at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
> at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
> at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
> at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
> at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
> at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
> at org.apache.activemq.advisory.AdvisoryBroker.addDestinationInfo(AdvisoryBroker.java:173)
> at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
> at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
> at org.apache.activemq.broker.MutableBrokerFilter.addDestinationInfo(MutableBrokerFilter.java:221)
> at org.apache.activemq.broker.TransportConnection.processAddDestination(TransportConnection.java:467)
> at org.apache.activemq.command.DestinationInfo.visit(DestinationInfo.java:122)
> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
> at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
> at org.apache.activemq.transport.stomp.ProtocolConverter.createTempQueue(ProtocolConverter.java:611)
> at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertDestination(LegacyFrameTranslator.java:130)
> at org.apache.activemq.transport.stomp.FrameTranslator$Helper.copyStandardHeadersFromFrameToMessage(FrameTranslator.java:93)
> at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertFrame(LegacyFrameTranslator.java:54)
> at org.apache.activemq.transport.stomp.ProtocolConverter.convertMessage(ProtocolConverter.java:591)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompSend(ProtocolConverter.java:231)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:173)
> at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
> at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
> at java.lang.Thread.run(Thread.java:619)
> It is waiting to lock 0x00002aaab46f8938. This is the thread that holds the lock:
> "ActiveMQ Transport: tcp:///10.231.233.117:43020" daemon prio=10 tid=0x00000000608d8800 nid=0xc6e in Object.wait() [0x000000007d41c000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at org.apache.activemq.broker.region.TopicSubscription.add(TopicSubscription.java:106)
> - locked <0x00002aaab57d34e8> (a java.lang.Object)
> at org.apache.activemq.broker.region.policy.SimpleDispatchPolicy.dispatch(SimpleDispatchPolicy.java:49)
> - locked <0x00002aaab46f98e0> (a java.util.concurrent.CopyOnWriteArrayList)
> at org.apache.activemq.broker.region.Topic.dispatch(Topic.java:585)
> at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:443)
> - locked <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
> at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
> at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
> at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
> at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
> at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
> at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
> at org.apache.activemq.advisory.AdvisoryBroker.removeDestination(AdvisoryBroker.java:185)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
> at org.apache.activemq.broker.MutableBrokerFilter.removeDestination(MutableBrokerFilter.java:153)
> at org.apache.activemq.broker.TransportConnection.processRemoveConnection(TransportConnection.java:713)
> - locked <0x00002aaac32b10c0> (a org.apache.activemq.broker.jmx.ManagedTransportConnection)
> at org.apache.activemq.command.RemoveInfo.visit(RemoveInfo.java:72)
> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
> at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompDisconnect(ProtocolConverter.java:545)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:189)
> at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
> at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
> at java.lang.Thread.run(Thread.java:619)
> So it is the client 10.231.233.117 that has a stomp-connection to the broker, causes a TopicSubscription.add and causes the broker freeze.
> This is how we configure our openwire transport-connector in the activemq.xml:
> <transportConnectors>
> <transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
> <transportConnector name="stomp" uri="stomp://0.0.0.0:61613"/>
> </transportConnectors>
> We took jstack-dumps with a distance of 9 minutes. Both are attached. In both dumps it is the same TopicSubscription.add that causes the hang.
> How can we solve this issue? Looks like the connection should run into an timeout but doesn't. Is this an activemq bug? Can we somehow set the broker openwire timeout parameters?
> Thanks for any hints.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-2749) Broker hangs in TopicSubscription.add
Posted by "Dennis Klinkott (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=59542#action_59542 ]
Dennis Klinkott commented on AMQ-2749:
--------------------------------------
Now we got the same problem on a preprod machine.
This time it is not a stomp connection that causes the freeze.
This is the thread that holds the lock that blocks all other threads:
"ActiveMQ Transport: tcp:///10.231.243.16:57992" daemon prio=10 tid=0x00002ab309c44800 nid=0xd40 in Object.wait() [0x00002ab364174000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.apache.activemq.broker.region.TopicSubscription.add(TopicSubscription.java:106)
- locked <0x00002aab30c0e238> (a java.lang.Object)
at org.apache.activemq.broker.region.policy.SimpleDispatchPolicy.dispatch(SimpleDispatchPolicy.java:49)
- locked <0x00002aaab3f5eaa8> (a java.util.concurrent.CopyOnWriteArrayList)
at org.apache.activemq.broker.region.Topic.dispatch(Topic.java:585)
at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:443)
- locked <0x00002aaab3f5de00> (a org.apache.activemq.broker.region.Topic)
at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
at org.apache.activemq.advisory.AdvisoryBroker.addConsumer(AdvisoryBroker.java:116)
at org.apache.activemq.broker.BrokerFilter.addConsumer(BrokerFilter.java:86)
at org.apache.activemq.broker.MutableBrokerFilter.addConsumer(MutableBrokerFilter.java:93)
at org.apache.activemq.broker.TransportConnection.processAddConsumer(TransportConnection.java:533)
at org.apache.activemq.command.ConsumerInfo.visit(ConsumerInfo.java:349)
at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
at org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:216)
- locked <0x00002aab2fd02060> (a org.apache.activemq.transport.InactivityMonitor$1)
at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
at java.lang.Thread.run(Thread.java:619)
I wonder how TopicSubscription.add can freeze?
> Broker hangs in TopicSubscription.add
> -------------------------------------
>
> Key: AMQ-2749
> URL: https://issues.apache.org/activemq/browse/AMQ-2749
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.1
> Environment: red had enterprise
> Reporter: Dennis Klinkott
> Attachments: jstack.2010-05-25-14-01-30, jstack.2010-05-25-14-10-54
>
>
> Our broker suddenly hang in production environment. We created a bunch of jstacks to see what the reason is.
> The jstacks show that the broker waits with many threads in doMessageSend:
> e.g.
> "ActiveMQ Transport: tcp:///10.231.233.117:51088" daemon prio=10 tid=0x00002aab84016800 nid=0x1f19 waiting for monitor entry [0x000000006a8f2000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:402)
> - waiting to lock <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
> at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
> at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
> at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
> at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
> at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
> at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
> at org.apache.activemq.advisory.AdvisoryBroker.addDestinationInfo(AdvisoryBroker.java:173)
> at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
> at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
> at org.apache.activemq.broker.MutableBrokerFilter.addDestinationInfo(MutableBrokerFilter.java:221)
> at org.apache.activemq.broker.TransportConnection.processAddDestination(TransportConnection.java:467)
> at org.apache.activemq.command.DestinationInfo.visit(DestinationInfo.java:122)
> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
> at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
> at org.apache.activemq.transport.stomp.ProtocolConverter.createTempQueue(ProtocolConverter.java:611)
> at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertDestination(LegacyFrameTranslator.java:130)
> at org.apache.activemq.transport.stomp.FrameTranslator$Helper.copyStandardHeadersFromFrameToMessage(FrameTranslator.java:93)
> at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertFrame(LegacyFrameTranslator.java:54)
> at org.apache.activemq.transport.stomp.ProtocolConverter.convertMessage(ProtocolConverter.java:591)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompSend(ProtocolConverter.java:231)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:173)
> at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
> at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
> at java.lang.Thread.run(Thread.java:619)
> It is waiting to lock 0x00002aaab46f8938. This is the thread that holds the lock:
> "ActiveMQ Transport: tcp:///10.231.233.117:43020" daemon prio=10 tid=0x00000000608d8800 nid=0xc6e in Object.wait() [0x000000007d41c000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at org.apache.activemq.broker.region.TopicSubscription.add(TopicSubscription.java:106)
> - locked <0x00002aaab57d34e8> (a java.lang.Object)
> at org.apache.activemq.broker.region.policy.SimpleDispatchPolicy.dispatch(SimpleDispatchPolicy.java:49)
> - locked <0x00002aaab46f98e0> (a java.util.concurrent.CopyOnWriteArrayList)
> at org.apache.activemq.broker.region.Topic.dispatch(Topic.java:585)
> at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:443)
> - locked <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
> at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
> at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
> at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
> at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
> at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
> at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
> at org.apache.activemq.advisory.AdvisoryBroker.removeDestination(AdvisoryBroker.java:185)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
> at org.apache.activemq.broker.MutableBrokerFilter.removeDestination(MutableBrokerFilter.java:153)
> at org.apache.activemq.broker.TransportConnection.processRemoveConnection(TransportConnection.java:713)
> - locked <0x00002aaac32b10c0> (a org.apache.activemq.broker.jmx.ManagedTransportConnection)
> at org.apache.activemq.command.RemoveInfo.visit(RemoveInfo.java:72)
> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
> at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompDisconnect(ProtocolConverter.java:545)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:189)
> at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
> at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
> at java.lang.Thread.run(Thread.java:619)
> So it is the client 10.231.233.117 that has a stomp-connection to the broker, causes a TopicSubscription.add and causes the broker freeze.
> This is how we configure our openwire transport-connector in the activemq.xml:
> <transportConnectors>
> <transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
> <transportConnector name="stomp" uri="stomp://0.0.0.0:61613"/>
> </transportConnectors>
> We took jstack-dumps with a distance of 9 minutes. Both are attached. In both dumps it is the same TopicSubscription.add that causes the hang.
> How can we solve this issue? Looks like the connection should run into an timeout but doesn't. Is this an activemq bug? Can we somehow set the broker openwire timeout parameters?
> Thanks for any hints.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-2749) Broker hangs in TopicSubscription.add
Posted by "Dennis Klinkott (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=59544#action_59544 ]
Dennis Klinkott commented on AMQ-2749:
--------------------------------------
I've attached our activemq.xml and a screenshot of a jmx console to a similar broker that is running since 5 days.
we do not have any warnings or errors in the log.
if we reached the temp store or memory limit on the pending message cursor, would we get a warning log entry?
> Broker hangs in TopicSubscription.add
> -------------------------------------
>
> Key: AMQ-2749
> URL: https://issues.apache.org/activemq/browse/AMQ-2749
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.1
> Environment: red had enterprise
> Reporter: Dennis Klinkott
> Attachments: activemq.xml, jmx console screenshot.jpg, jstack.2010-05-25-14-01-30, jstack.2010-05-25-14-10-54
>
>
> Our broker suddenly hang in production environment. We created a bunch of jstacks to see what the reason is.
> The jstacks show that the broker waits with many threads in doMessageSend:
> e.g.
> "ActiveMQ Transport: tcp:///10.231.233.117:51088" daemon prio=10 tid=0x00002aab84016800 nid=0x1f19 waiting for monitor entry [0x000000006a8f2000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:402)
> - waiting to lock <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
> at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
> at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
> at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
> at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
> at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
> at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
> at org.apache.activemq.advisory.AdvisoryBroker.addDestinationInfo(AdvisoryBroker.java:173)
> at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
> at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
> at org.apache.activemq.broker.MutableBrokerFilter.addDestinationInfo(MutableBrokerFilter.java:221)
> at org.apache.activemq.broker.TransportConnection.processAddDestination(TransportConnection.java:467)
> at org.apache.activemq.command.DestinationInfo.visit(DestinationInfo.java:122)
> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
> at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
> at org.apache.activemq.transport.stomp.ProtocolConverter.createTempQueue(ProtocolConverter.java:611)
> at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertDestination(LegacyFrameTranslator.java:130)
> at org.apache.activemq.transport.stomp.FrameTranslator$Helper.copyStandardHeadersFromFrameToMessage(FrameTranslator.java:93)
> at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertFrame(LegacyFrameTranslator.java:54)
> at org.apache.activemq.transport.stomp.ProtocolConverter.convertMessage(ProtocolConverter.java:591)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompSend(ProtocolConverter.java:231)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:173)
> at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
> at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
> at java.lang.Thread.run(Thread.java:619)
> It is waiting to lock 0x00002aaab46f8938. This is the thread that holds the lock:
> "ActiveMQ Transport: tcp:///10.231.233.117:43020" daemon prio=10 tid=0x00000000608d8800 nid=0xc6e in Object.wait() [0x000000007d41c000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at org.apache.activemq.broker.region.TopicSubscription.add(TopicSubscription.java:106)
> - locked <0x00002aaab57d34e8> (a java.lang.Object)
> at org.apache.activemq.broker.region.policy.SimpleDispatchPolicy.dispatch(SimpleDispatchPolicy.java:49)
> - locked <0x00002aaab46f98e0> (a java.util.concurrent.CopyOnWriteArrayList)
> at org.apache.activemq.broker.region.Topic.dispatch(Topic.java:585)
> at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:443)
> - locked <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
> at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
> at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
> at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
> at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
> at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
> at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
> at org.apache.activemq.advisory.AdvisoryBroker.removeDestination(AdvisoryBroker.java:185)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
> at org.apache.activemq.broker.MutableBrokerFilter.removeDestination(MutableBrokerFilter.java:153)
> at org.apache.activemq.broker.TransportConnection.processRemoveConnection(TransportConnection.java:713)
> - locked <0x00002aaac32b10c0> (a org.apache.activemq.broker.jmx.ManagedTransportConnection)
> at org.apache.activemq.command.RemoveInfo.visit(RemoveInfo.java:72)
> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
> at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompDisconnect(ProtocolConverter.java:545)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:189)
> at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
> at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
> at java.lang.Thread.run(Thread.java:619)
> So it is the client 10.231.233.117 that has a stomp-connection to the broker, causes a TopicSubscription.add and causes the broker freeze.
> This is how we configure our openwire transport-connector in the activemq.xml:
> <transportConnectors>
> <transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
> <transportConnector name="stomp" uri="stomp://0.0.0.0:61613"/>
> </transportConnectors>
> We took jstack-dumps with a distance of 9 minutes. Both are attached. In both dumps it is the same TopicSubscription.add that causes the hang.
> How can we solve this issue? Looks like the connection should run into an timeout but doesn't. Is this an activemq bug? Can we somehow set the broker openwire timeout parameters?
> Thanks for any hints.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-2749) Broker hangs in TopicSubscription.add
Posted by "Dennis Klinkott (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Klinkott updated AMQ-2749:
---------------------------------
Attachment: jstack.2010-05-25-14-01-30
jstack.2010-05-25-14-10-54
> Broker hangs in TopicSubscription.add
> -------------------------------------
>
> Key: AMQ-2749
> URL: https://issues.apache.org/activemq/browse/AMQ-2749
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.1
> Environment: red had enterprise
> Reporter: Dennis Klinkott
> Attachments: jstack.2010-05-25-14-01-30, jstack.2010-05-25-14-10-54
>
>
> Our broker suddenly hang in production environment. We created a bunch of jstacks to see what the reason is.
> The jstacks show that the broker waits with many threads in doMessageSend:
> e.g.
> "ActiveMQ Transport: tcp:///10.231.233.117:51088" daemon prio=10 tid=0x00002aab84016800 nid=0x1f19 waiting for monitor entry [0x000000006a8f2000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:402)
> - waiting to lock <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
> at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
> at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
> at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
> at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
> at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
> at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
> at org.apache.activemq.advisory.AdvisoryBroker.addDestinationInfo(AdvisoryBroker.java:173)
> at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
> at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
> at org.apache.activemq.broker.MutableBrokerFilter.addDestinationInfo(MutableBrokerFilter.java:221)
> at org.apache.activemq.broker.TransportConnection.processAddDestination(TransportConnection.java:467)
> at org.apache.activemq.command.DestinationInfo.visit(DestinationInfo.java:122)
> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
> at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
> at org.apache.activemq.transport.stomp.ProtocolConverter.createTempQueue(ProtocolConverter.java:611)
> at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertDestination(LegacyFrameTranslator.java:130)
> at org.apache.activemq.transport.stomp.FrameTranslator$Helper.copyStandardHeadersFromFrameToMessage(FrameTranslator.java:93)
> at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertFrame(LegacyFrameTranslator.java:54)
> at org.apache.activemq.transport.stomp.ProtocolConverter.convertMessage(ProtocolConverter.java:591)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompSend(ProtocolConverter.java:231)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:173)
> at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
> at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
> at java.lang.Thread.run(Thread.java:619)
> It is waiting to lock 0x00002aaab46f8938. This is the thread that holds the lock:
> "ActiveMQ Transport: tcp:///10.231.233.117:43020" daemon prio=10 tid=0x00000000608d8800 nid=0xc6e in Object.wait() [0x000000007d41c000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at org.apache.activemq.broker.region.TopicSubscription.add(TopicSubscription.java:106)
> - locked <0x00002aaab57d34e8> (a java.lang.Object)
> at org.apache.activemq.broker.region.policy.SimpleDispatchPolicy.dispatch(SimpleDispatchPolicy.java:49)
> - locked <0x00002aaab46f98e0> (a java.util.concurrent.CopyOnWriteArrayList)
> at org.apache.activemq.broker.region.Topic.dispatch(Topic.java:585)
> at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:443)
> - locked <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
> at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
> at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
> at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
> at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
> at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
> at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
> at org.apache.activemq.advisory.AdvisoryBroker.removeDestination(AdvisoryBroker.java:185)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
> at org.apache.activemq.broker.MutableBrokerFilter.removeDestination(MutableBrokerFilter.java:153)
> at org.apache.activemq.broker.TransportConnection.processRemoveConnection(TransportConnection.java:713)
> - locked <0x00002aaac32b10c0> (a org.apache.activemq.broker.jmx.ManagedTransportConnection)
> at org.apache.activemq.command.RemoveInfo.visit(RemoveInfo.java:72)
> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
> at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompDisconnect(ProtocolConverter.java:545)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:189)
> at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
> at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
> at java.lang.Thread.run(Thread.java:619)
> So it is the client 10.231.233.117 that has a stomp-connection to the broker, causes a TopicSubscription.add and causes the broker freeze.
> This is how we configure our openwire transport-connector in the activemq.xml:
> <transportConnectors>
> <transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
> <transportConnector name="stomp" uri="stomp://0.0.0.0:61613"/>
> </transportConnectors>
> We took jstack-dumps with a distance of 9 minutes. Both are attached. In both dumps it is the same TopicSubscription.add that causes the hang.
> How can we solve this issue? Looks like the connection should run into an timeout but doesn't. Is this an activemq bug? Can we somehow set the broker openwire timeout parameters?
> Thanks for any hints.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-2749) Broker hangs in TopicSubscription.add
Posted by "Dennis Klinkott (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=59568#action_59568 ]
Dennis Klinkott commented on AMQ-2749:
--------------------------------------
We might have reached the 64M memory limit.
We changed our configuration and specified a high memory limit. Hopefully this will solve our issue.
> Broker hangs in TopicSubscription.add
> -------------------------------------
>
> Key: AMQ-2749
> URL: https://issues.apache.org/activemq/browse/AMQ-2749
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.1
> Environment: red had enterprise
> Reporter: Dennis Klinkott
> Attachments: activemq.xml, jmx console screenshot.jpg, jstack.2010-05-25-14-01-30, jstack.2010-05-25-14-10-54
>
>
> Our broker suddenly hang in production environment. We created a bunch of jstacks to see what the reason is.
> The jstacks show that the broker waits with many threads in doMessageSend:
> e.g.
> "ActiveMQ Transport: tcp:///10.231.233.117:51088" daemon prio=10 tid=0x00002aab84016800 nid=0x1f19 waiting for monitor entry [0x000000006a8f2000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:402)
> - waiting to lock <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
> at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
> at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
> at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
> at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
> at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
> at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
> at org.apache.activemq.advisory.AdvisoryBroker.addDestinationInfo(AdvisoryBroker.java:173)
> at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
> at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
> at org.apache.activemq.broker.MutableBrokerFilter.addDestinationInfo(MutableBrokerFilter.java:221)
> at org.apache.activemq.broker.TransportConnection.processAddDestination(TransportConnection.java:467)
> at org.apache.activemq.command.DestinationInfo.visit(DestinationInfo.java:122)
> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
> at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
> at org.apache.activemq.transport.stomp.ProtocolConverter.createTempQueue(ProtocolConverter.java:611)
> at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertDestination(LegacyFrameTranslator.java:130)
> at org.apache.activemq.transport.stomp.FrameTranslator$Helper.copyStandardHeadersFromFrameToMessage(FrameTranslator.java:93)
> at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertFrame(LegacyFrameTranslator.java:54)
> at org.apache.activemq.transport.stomp.ProtocolConverter.convertMessage(ProtocolConverter.java:591)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompSend(ProtocolConverter.java:231)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:173)
> at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
> at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
> at java.lang.Thread.run(Thread.java:619)
> It is waiting to lock 0x00002aaab46f8938. This is the thread that holds the lock:
> "ActiveMQ Transport: tcp:///10.231.233.117:43020" daemon prio=10 tid=0x00000000608d8800 nid=0xc6e in Object.wait() [0x000000007d41c000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at org.apache.activemq.broker.region.TopicSubscription.add(TopicSubscription.java:106)
> - locked <0x00002aaab57d34e8> (a java.lang.Object)
> at org.apache.activemq.broker.region.policy.SimpleDispatchPolicy.dispatch(SimpleDispatchPolicy.java:49)
> - locked <0x00002aaab46f98e0> (a java.util.concurrent.CopyOnWriteArrayList)
> at org.apache.activemq.broker.region.Topic.dispatch(Topic.java:585)
> at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:443)
> - locked <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
> at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
> at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
> at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
> at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
> at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
> at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
> at org.apache.activemq.advisory.AdvisoryBroker.removeDestination(AdvisoryBroker.java:185)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
> at org.apache.activemq.broker.MutableBrokerFilter.removeDestination(MutableBrokerFilter.java:153)
> at org.apache.activemq.broker.TransportConnection.processRemoveConnection(TransportConnection.java:713)
> - locked <0x00002aaac32b10c0> (a org.apache.activemq.broker.jmx.ManagedTransportConnection)
> at org.apache.activemq.command.RemoveInfo.visit(RemoveInfo.java:72)
> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
> at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompDisconnect(ProtocolConverter.java:545)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:189)
> at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
> at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
> at java.lang.Thread.run(Thread.java:619)
> So it is the client 10.231.233.117 that has a stomp-connection to the broker, causes a TopicSubscription.add and causes the broker freeze.
> This is how we configure our openwire transport-connector in the activemq.xml:
> <transportConnectors>
> <transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
> <transportConnector name="stomp" uri="stomp://0.0.0.0:61613"/>
> </transportConnectors>
> We took jstack-dumps with a distance of 9 minutes. Both are attached. In both dumps it is the same TopicSubscription.add that causes the hang.
> How can we solve this issue? Looks like the connection should run into an timeout but doesn't. Is this an activemq bug? Can we somehow set the broker openwire timeout parameters?
> Thanks for any hints.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-2749) Broker hangs in TopicSubscription.add
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=59543#action_59543 ]
Gary Tully commented on AMQ-2749:
---------------------------------
from the code: http://svn.apache.org/viewvc/activemq/tags/activemq-5.3.1/activemq-core/src/main/java/org/apache/activemq/broker/region/TopicSubscription.java?view=markup
it looks like u have reached the temp store or memory limit on the pending message cursor. which it is depends on the pending message cursor u are using.
> Broker hangs in TopicSubscription.add
> -------------------------------------
>
> Key: AMQ-2749
> URL: https://issues.apache.org/activemq/browse/AMQ-2749
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.1
> Environment: red had enterprise
> Reporter: Dennis Klinkott
> Attachments: jstack.2010-05-25-14-01-30, jstack.2010-05-25-14-10-54
>
>
> Our broker suddenly hang in production environment. We created a bunch of jstacks to see what the reason is.
> The jstacks show that the broker waits with many threads in doMessageSend:
> e.g.
> "ActiveMQ Transport: tcp:///10.231.233.117:51088" daemon prio=10 tid=0x00002aab84016800 nid=0x1f19 waiting for monitor entry [0x000000006a8f2000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:402)
> - waiting to lock <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
> at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
> at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
> at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
> at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
> at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
> at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
> at org.apache.activemq.advisory.AdvisoryBroker.addDestinationInfo(AdvisoryBroker.java:173)
> at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
> at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
> at org.apache.activemq.broker.MutableBrokerFilter.addDestinationInfo(MutableBrokerFilter.java:221)
> at org.apache.activemq.broker.TransportConnection.processAddDestination(TransportConnection.java:467)
> at org.apache.activemq.command.DestinationInfo.visit(DestinationInfo.java:122)
> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
> at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
> at org.apache.activemq.transport.stomp.ProtocolConverter.createTempQueue(ProtocolConverter.java:611)
> at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertDestination(LegacyFrameTranslator.java:130)
> at org.apache.activemq.transport.stomp.FrameTranslator$Helper.copyStandardHeadersFromFrameToMessage(FrameTranslator.java:93)
> at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertFrame(LegacyFrameTranslator.java:54)
> at org.apache.activemq.transport.stomp.ProtocolConverter.convertMessage(ProtocolConverter.java:591)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompSend(ProtocolConverter.java:231)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:173)
> at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
> at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
> at java.lang.Thread.run(Thread.java:619)
> It is waiting to lock 0x00002aaab46f8938. This is the thread that holds the lock:
> "ActiveMQ Transport: tcp:///10.231.233.117:43020" daemon prio=10 tid=0x00000000608d8800 nid=0xc6e in Object.wait() [0x000000007d41c000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at org.apache.activemq.broker.region.TopicSubscription.add(TopicSubscription.java:106)
> - locked <0x00002aaab57d34e8> (a java.lang.Object)
> at org.apache.activemq.broker.region.policy.SimpleDispatchPolicy.dispatch(SimpleDispatchPolicy.java:49)
> - locked <0x00002aaab46f98e0> (a java.util.concurrent.CopyOnWriteArrayList)
> at org.apache.activemq.broker.region.Topic.dispatch(Topic.java:585)
> at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:443)
> - locked <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
> at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
> at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
> at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
> at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
> at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
> at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
> at org.apache.activemq.advisory.AdvisoryBroker.removeDestination(AdvisoryBroker.java:185)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
> at org.apache.activemq.broker.MutableBrokerFilter.removeDestination(MutableBrokerFilter.java:153)
> at org.apache.activemq.broker.TransportConnection.processRemoveConnection(TransportConnection.java:713)
> - locked <0x00002aaac32b10c0> (a org.apache.activemq.broker.jmx.ManagedTransportConnection)
> at org.apache.activemq.command.RemoveInfo.visit(RemoveInfo.java:72)
> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
> at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompDisconnect(ProtocolConverter.java:545)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:189)
> at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
> at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
> at java.lang.Thread.run(Thread.java:619)
> So it is the client 10.231.233.117 that has a stomp-connection to the broker, causes a TopicSubscription.add and causes the broker freeze.
> This is how we configure our openwire transport-connector in the activemq.xml:
> <transportConnectors>
> <transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
> <transportConnector name="stomp" uri="stomp://0.0.0.0:61613"/>
> </transportConnectors>
> We took jstack-dumps with a distance of 9 minutes. Both are attached. In both dumps it is the same TopicSubscription.add that causes the hang.
> How can we solve this issue? Looks like the connection should run into an timeout but doesn't. Is this an activemq bug? Can we somehow set the broker openwire timeout parameters?
> Thanks for any hints.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-2749) Broker hangs in TopicSubscription.add
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=59548#action_59548 ]
Gary Tully commented on AMQ-2749:
---------------------------------
I just added a log message if the subscription is blocked as in your stack trace to trunk. r948489. will be in tonights 5.4-SNAPSHOT
While u don't have any usage limits specified, there are some default values, 64M memory, 10G temp and 100G store. Would you be using that much temp space?
> Broker hangs in TopicSubscription.add
> -------------------------------------
>
> Key: AMQ-2749
> URL: https://issues.apache.org/activemq/browse/AMQ-2749
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.1
> Environment: red had enterprise
> Reporter: Dennis Klinkott
> Attachments: activemq.xml, jmx console screenshot.jpg, jstack.2010-05-25-14-01-30, jstack.2010-05-25-14-10-54
>
>
> Our broker suddenly hang in production environment. We created a bunch of jstacks to see what the reason is.
> The jstacks show that the broker waits with many threads in doMessageSend:
> e.g.
> "ActiveMQ Transport: tcp:///10.231.233.117:51088" daemon prio=10 tid=0x00002aab84016800 nid=0x1f19 waiting for monitor entry [0x000000006a8f2000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:402)
> - waiting to lock <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
> at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
> at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
> at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
> at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
> at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
> at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
> at org.apache.activemq.advisory.AdvisoryBroker.addDestinationInfo(AdvisoryBroker.java:173)
> at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
> at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
> at org.apache.activemq.broker.MutableBrokerFilter.addDestinationInfo(MutableBrokerFilter.java:221)
> at org.apache.activemq.broker.TransportConnection.processAddDestination(TransportConnection.java:467)
> at org.apache.activemq.command.DestinationInfo.visit(DestinationInfo.java:122)
> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
> at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
> at org.apache.activemq.transport.stomp.ProtocolConverter.createTempQueue(ProtocolConverter.java:611)
> at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertDestination(LegacyFrameTranslator.java:130)
> at org.apache.activemq.transport.stomp.FrameTranslator$Helper.copyStandardHeadersFromFrameToMessage(FrameTranslator.java:93)
> at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertFrame(LegacyFrameTranslator.java:54)
> at org.apache.activemq.transport.stomp.ProtocolConverter.convertMessage(ProtocolConverter.java:591)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompSend(ProtocolConverter.java:231)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:173)
> at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
> at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
> at java.lang.Thread.run(Thread.java:619)
> It is waiting to lock 0x00002aaab46f8938. This is the thread that holds the lock:
> "ActiveMQ Transport: tcp:///10.231.233.117:43020" daemon prio=10 tid=0x00000000608d8800 nid=0xc6e in Object.wait() [0x000000007d41c000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at org.apache.activemq.broker.region.TopicSubscription.add(TopicSubscription.java:106)
> - locked <0x00002aaab57d34e8> (a java.lang.Object)
> at org.apache.activemq.broker.region.policy.SimpleDispatchPolicy.dispatch(SimpleDispatchPolicy.java:49)
> - locked <0x00002aaab46f98e0> (a java.util.concurrent.CopyOnWriteArrayList)
> at org.apache.activemq.broker.region.Topic.dispatch(Topic.java:585)
> at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:443)
> - locked <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
> at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
> at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
> at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
> at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
> at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
> at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
> at org.apache.activemq.advisory.AdvisoryBroker.removeDestination(AdvisoryBroker.java:185)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
> at org.apache.activemq.broker.MutableBrokerFilter.removeDestination(MutableBrokerFilter.java:153)
> at org.apache.activemq.broker.TransportConnection.processRemoveConnection(TransportConnection.java:713)
> - locked <0x00002aaac32b10c0> (a org.apache.activemq.broker.jmx.ManagedTransportConnection)
> at org.apache.activemq.command.RemoveInfo.visit(RemoveInfo.java:72)
> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
> at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompDisconnect(ProtocolConverter.java:545)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:189)
> at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
> at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
> at java.lang.Thread.run(Thread.java:619)
> So it is the client 10.231.233.117 that has a stomp-connection to the broker, causes a TopicSubscription.add and causes the broker freeze.
> This is how we configure our openwire transport-connector in the activemq.xml:
> <transportConnectors>
> <transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
> <transportConnector name="stomp" uri="stomp://0.0.0.0:61613"/>
> </transportConnectors>
> We took jstack-dumps with a distance of 9 minutes. Both are attached. In both dumps it is the same TopicSubscription.add that causes the hang.
> How can we solve this issue? Looks like the connection should run into an timeout but doesn't. Is this an activemq bug? Can we somehow set the broker openwire timeout parameters?
> Thanks for any hints.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-2749) Broker hangs in TopicSubscription.add
Posted by "Dennis Klinkott (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dennis Klinkott updated AMQ-2749:
---------------------------------
Attachment: activemq.xml
> Broker hangs in TopicSubscription.add
> -------------------------------------
>
> Key: AMQ-2749
> URL: https://issues.apache.org/activemq/browse/AMQ-2749
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.1
> Environment: red had enterprise
> Reporter: Dennis Klinkott
> Attachments: activemq.xml, jstack.2010-05-25-14-01-30, jstack.2010-05-25-14-10-54
>
>
> Our broker suddenly hang in production environment. We created a bunch of jstacks to see what the reason is.
> The jstacks show that the broker waits with many threads in doMessageSend:
> e.g.
> "ActiveMQ Transport: tcp:///10.231.233.117:51088" daemon prio=10 tid=0x00002aab84016800 nid=0x1f19 waiting for monitor entry [0x000000006a8f2000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:402)
> - waiting to lock <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
> at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
> at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
> at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
> at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
> at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
> at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
> at org.apache.activemq.advisory.AdvisoryBroker.addDestinationInfo(AdvisoryBroker.java:173)
> at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
> at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
> at org.apache.activemq.broker.MutableBrokerFilter.addDestinationInfo(MutableBrokerFilter.java:221)
> at org.apache.activemq.broker.TransportConnection.processAddDestination(TransportConnection.java:467)
> at org.apache.activemq.command.DestinationInfo.visit(DestinationInfo.java:122)
> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
> at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
> at org.apache.activemq.transport.stomp.ProtocolConverter.createTempQueue(ProtocolConverter.java:611)
> at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertDestination(LegacyFrameTranslator.java:130)
> at org.apache.activemq.transport.stomp.FrameTranslator$Helper.copyStandardHeadersFromFrameToMessage(FrameTranslator.java:93)
> at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertFrame(LegacyFrameTranslator.java:54)
> at org.apache.activemq.transport.stomp.ProtocolConverter.convertMessage(ProtocolConverter.java:591)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompSend(ProtocolConverter.java:231)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:173)
> at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
> at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
> at java.lang.Thread.run(Thread.java:619)
> It is waiting to lock 0x00002aaab46f8938. This is the thread that holds the lock:
> "ActiveMQ Transport: tcp:///10.231.233.117:43020" daemon prio=10 tid=0x00000000608d8800 nid=0xc6e in Object.wait() [0x000000007d41c000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at org.apache.activemq.broker.region.TopicSubscription.add(TopicSubscription.java:106)
> - locked <0x00002aaab57d34e8> (a java.lang.Object)
> at org.apache.activemq.broker.region.policy.SimpleDispatchPolicy.dispatch(SimpleDispatchPolicy.java:49)
> - locked <0x00002aaab46f98e0> (a java.util.concurrent.CopyOnWriteArrayList)
> at org.apache.activemq.broker.region.Topic.dispatch(Topic.java:585)
> at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:443)
> - locked <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
> at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
> at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
> at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
> at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
> at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
> at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
> at org.apache.activemq.advisory.AdvisoryBroker.removeDestination(AdvisoryBroker.java:185)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
> at org.apache.activemq.broker.MutableBrokerFilter.removeDestination(MutableBrokerFilter.java:153)
> at org.apache.activemq.broker.TransportConnection.processRemoveConnection(TransportConnection.java:713)
> - locked <0x00002aaac32b10c0> (a org.apache.activemq.broker.jmx.ManagedTransportConnection)
> at org.apache.activemq.command.RemoveInfo.visit(RemoveInfo.java:72)
> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
> at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompDisconnect(ProtocolConverter.java:545)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:189)
> at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
> at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
> at java.lang.Thread.run(Thread.java:619)
> So it is the client 10.231.233.117 that has a stomp-connection to the broker, causes a TopicSubscription.add and causes the broker freeze.
> This is how we configure our openwire transport-connector in the activemq.xml:
> <transportConnectors>
> <transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
> <transportConnector name="stomp" uri="stomp://0.0.0.0:61613"/>
> </transportConnectors>
> We took jstack-dumps with a distance of 9 minutes. Both are attached. In both dumps it is the same TopicSubscription.add that causes the hang.
> How can we solve this issue? Looks like the connection should run into an timeout but doesn't. Is this an activemq bug? Can we somehow set the broker openwire timeout parameters?
> Thanks for any hints.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-2749) Broker hangs in TopicSubscription.add
Posted by "Bruce Snyder (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bruce Snyder updated AMQ-2749:
------------------------------
Fix Version/s: 5.5.0
(was: 5.4.1)
> Broker hangs in TopicSubscription.add
> -------------------------------------
>
> Key: AMQ-2749
> URL: https://issues.apache.org/activemq/browse/AMQ-2749
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.1
> Environment: red had enterprise
> Reporter: Dennis Klinkott
> Fix For: 5.5.0
>
> Attachments: activemq.xml, jmx console screenshot.jpg, jstack.2010-05-25-14-01-30, jstack.2010-05-25-14-10-54
>
>
> Our broker suddenly hang in production environment. We created a bunch of jstacks to see what the reason is.
> The jstacks show that the broker waits with many threads in doMessageSend:
> e.g.
> "ActiveMQ Transport: tcp:///10.231.233.117:51088" daemon prio=10 tid=0x00002aab84016800 nid=0x1f19 waiting for monitor entry [0x000000006a8f2000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:402)
> - waiting to lock <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
> at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
> at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
> at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
> at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
> at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
> at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
> at org.apache.activemq.advisory.AdvisoryBroker.addDestinationInfo(AdvisoryBroker.java:173)
> at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
> at org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:214)
> at org.apache.activemq.broker.MutableBrokerFilter.addDestinationInfo(MutableBrokerFilter.java:221)
> at org.apache.activemq.broker.TransportConnection.processAddDestination(TransportConnection.java:467)
> at org.apache.activemq.command.DestinationInfo.visit(DestinationInfo.java:122)
> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
> at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
> at org.apache.activemq.transport.stomp.ProtocolConverter.createTempQueue(ProtocolConverter.java:611)
> at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertDestination(LegacyFrameTranslator.java:130)
> at org.apache.activemq.transport.stomp.FrameTranslator$Helper.copyStandardHeadersFromFrameToMessage(FrameTranslator.java:93)
> at org.apache.activemq.transport.stomp.LegacyFrameTranslator.convertFrame(LegacyFrameTranslator.java:54)
> at org.apache.activemq.transport.stomp.ProtocolConverter.convertMessage(ProtocolConverter.java:591)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompSend(ProtocolConverter.java:231)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:173)
> at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
> at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
> at java.lang.Thread.run(Thread.java:619)
> It is waiting to lock 0x00002aaab46f8938. This is the thread that holds the lock:
> "ActiveMQ Transport: tcp:///10.231.233.117:43020" daemon prio=10 tid=0x00000000608d8800 nid=0xc6e in Object.wait() [0x000000007d41c000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at org.apache.activemq.broker.region.TopicSubscription.add(TopicSubscription.java:106)
> - locked <0x00002aaab57d34e8> (a java.lang.Object)
> at org.apache.activemq.broker.region.policy.SimpleDispatchPolicy.dispatch(SimpleDispatchPolicy.java:49)
> - locked <0x00002aaab46f98e0> (a java.util.concurrent.CopyOnWriteArrayList)
> at org.apache.activemq.broker.region.Topic.dispatch(Topic.java:585)
> at org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:443)
> - locked <0x00002aaab46f8938> (a org.apache.activemq.broker.region.Topic)
> at org.apache.activemq.broker.region.Topic.send(Topic.java:384)
> at org.apache.activemq.broker.region.DestinationFilter.send(DestinationFilter.java:93)
> at org.apache.activemq.broker.region.virtual.VirtualTopicInterceptor.send(VirtualTopicInterceptor.java:46)
> at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:354)
> at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:445)
> at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:443)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:373)
> at org.apache.activemq.advisory.AdvisoryBroker.fireAdvisory(AdvisoryBroker.java:368)
> at org.apache.activemq.advisory.AdvisoryBroker.removeDestination(AdvisoryBroker.java:185)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
> at org.apache.activemq.broker.BrokerFilter.removeDestination(BrokerFilter.java:146)
> at org.apache.activemq.broker.MutableBrokerFilter.removeDestination(MutableBrokerFilter.java:153)
> at org.apache.activemq.broker.TransportConnection.processRemoveConnection(TransportConnection.java:713)
> - locked <0x00002aaac32b10c0> (a org.apache.activemq.broker.jmx.ManagedTransportConnection)
> at org.apache.activemq.command.RemoveInfo.visit(RemoveInfo.java:72)
> at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:300)
> at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:178)
> at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> at org.apache.activemq.transport.stomp.StompTransportFilter.sendToActiveMQ(StompTransportFilter.java:82)
> at org.apache.activemq.transport.stomp.ProtocolConverter.sendToActiveMQ(ProtocolConverter.java:135)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompDisconnect(ProtocolConverter.java:545)
> at org.apache.activemq.transport.stomp.ProtocolConverter.onStompCommand(ProtocolConverter.java:189)
> at org.apache.activemq.transport.stomp.StompTransportFilter.onCommand(StompTransportFilter.java:71)
> at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:204)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:186)
> at java.lang.Thread.run(Thread.java:619)
> So it is the client 10.231.233.117 that has a stomp-connection to the broker, causes a TopicSubscription.add and causes the broker freeze.
> This is how we configure our openwire transport-connector in the activemq.xml:
> <transportConnectors>
> <transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
> <transportConnector name="stomp" uri="stomp://0.0.0.0:61613"/>
> </transportConnectors>
> We took jstack-dumps with a distance of 9 minutes. Both are attached. In both dumps it is the same TopicSubscription.add that causes the hang.
> How can we solve this issue? Looks like the connection should run into an timeout but doesn't. Is this an activemq bug? Can we somehow set the broker openwire timeout parameters?
> Thanks for any hints.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.