You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Mats Henrikson (JIRA)" <ji...@apache.org> on 2010/07/12 01:47:54 UTC
[jira] Commented: (AMQ-2332) it's possible to cause a deadlock in
ActiveMQ 5.2 using the client
[ https://issues.apache.org/activemq/browse/AMQ-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=60594#action_60594 ]
Mats Henrikson commented on AMQ-2332:
-------------------------------------
This has been resolved as Fixed, was it really? Because the last comment on the ticket reads to me like it was just closed saying I should adjust my socket timeout.
I also don't think this has anything to do with socket timeouts - as long as the client stays connected the deadlock will persist (and it may even persist after the client has disconnected...)
Anyway, just wondering if it was "Fixed" or if it should be marked "Wont do"...
> it's possible to cause a deadlock in ActiveMQ 5.2 using the client
> ------------------------------------------------------------------
>
> Key: AMQ-2332
> URL: https://issues.apache.org/activemq/browse/AMQ-2332
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.2.0
> Reporter: Mats Henrikson
> Assignee: Rob Davies
> Fix For: 5.4.0
>
>
> Hi,
> We have found a deadlock in ActiveMQ 5.2. It's possible that we may be causing it by a multi-threading problem in the client, we are currently looking into this. However, I feel that this is still a bug in the broker since it shouldn't be possible to deadlock the broker by incorrect use of the client.
> The situation is that we have two clients connected to the broker, one over TCP using the Java library, and one connecting over SSL (our own implementation, this may also be involved in creating the problem) using the NMS library in .NET.
> The Java client connects fine and subscribes to some topics and queues and starts sending messages to the topics. The NMS client connects and attempts to connect to some of the same topics, as well as creating a temporary queue. As it is doing this initial setup stuff it hangs and goes no further. From this point I don't think any more messages get delivered on the topic, and Java clients trying to send messages to it hang on the producers send() call indefinitely.
> The deadlock appears to occur because a thread is stuck actually writing data back to the NMS client (the bottom stack trace below), another thread (originating from the Java client) is trying to acquire the write lock so that it too can write to the client (middle stack trace) while still owning a lock on the list of consumers of a topic, meanwhile a third thread (this one origination from the NMS client again) is trying to get that lock on the list of consumers in order to add to it (top stack trace). For some reason the first thread trying to write to the NMS client never finishes, even when the NMS client is killed, and so the lock is never released and everything involving that topic grinds to a halt.
> I have looked but haven't been able to find anybody else reporting this issue.
> I enabled JMX as well as debug mode and got the following stack traces from jconsole, and tracked down the locks involved in the deadlock using a debugger:
> Name: ActiveMQ Transport: ssl:///10.247.39.150:59759
> State: BLOCKED on java.util.concurrent.CopyOnWriteArrayList@509c6c30 owned by: ActiveMQ Transport: tcp:///192.168.32.73:45666
> Stack trace:
> org.apache.activemq.broker.region.Topic.addSubscription(Topic.java:147)
> org.apache.activemq.broker.region.AbstractRegion.addConsumer(AbstractRegion.java:275)
> org.apache.activemq.broker.region.TopicRegion.addConsumer(TopicRegion.java:108)
> org.apache.activemq.broker.region.RegionBroker.addConsumer(RegionBroker.java:375)
> org.apache.activemq.broker.BrokerFilter.addConsumer(BrokerFilter.java:86)
> org.apache.activemq.broker.BrokerFilter.addConsumer(BrokerFilter.java:86)
> org.apache.activemq.advisory.AdvisoryBroker.addConsumer(AdvisoryBroker.java:83)
> org.apache.activemq.broker.BrokerFilter.addConsumer(BrokerFilter.java:86)
> org.apache.activemq.broker.BrokerFilter.addConsumer(BrokerFilter.java:86)
> org.apache.activemq.broker.BrokerFilter.addConsumer(BrokerFilter.java:86)
> org.apache.activemq.security.AuthorizationBroker.addConsumer(AuthorizationBroker.java:138)
> org.apache.activemq.broker.MutableBrokerFilter.addConsumer(MutableBrokerFilter.java:93)
> org.apache.activemq.broker.TransportConnection.processAddConsumer(TransportConnection.java:541)
> org.apache.activemq.command.ConsumerInfo.visit(ConsumerInfo.java:345)
> org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:305)
> org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:179)
> org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:143)
> org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:206)
> org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> org.apache.activemq.transport.tcp.SslTransport.doConsume(SslTransport.java:104)
> org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:203)
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:185)
> java.lang.Thread.run(Thread.java:595)
> CopyOnWriteArrayList@509c6c30 lock acquired at org.apache.activemq.broker.region.policy.SimpleDispatchPolicy.dispatch(SimpleDispatchPolicy.java:49) below:
> Name: ActiveMQ Transport: tcp:///192.168.32.73:45666
> State: BLOCKED on java.lang.Object@58c1c2f1 owned by: TempQueue: c34c73c1-d2e9-41f3-89a6-82f2c59ca62b:1
> Stack trace:
> org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:39)
> org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:1207)
> org.apache.activemq.broker.TransportConnection.processDispatch(TransportConnection.java:793)
> org.apache.activemq.broker.TransportConnection.dispatchSync(TransportConnection.java:752)
> org.apache.activemq.broker.region.TopicSubscription.dispatch(TopicSubscription.java:427)
> org.apache.activemq.broker.region.TopicSubscription.add(TopicSubscription.java:90)
> org.apache.activemq.broker.region.policy.SimpleDispatchPolicy.dispatch(SimpleDispatchPolicy.java:49)
> org.apache.activemq.broker.region.Topic.dispatch(Topic.java:580)
> org.apache.activemq.broker.region.Topic.doMessageSend(Topic.java:427)
> org.apache.activemq.broker.region.Topic.send(Topic.java:365)
> org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:350)
> org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:437)
> org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:224)
> org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:126)
> org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:95)
> org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:126)
> org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:126)
> org.apache.activemq.security.AuthorizationBroker.send(AuthorizationBroker.java:184)
> org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:133)
> org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:450)
> org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:639)
> org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:305)
> org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:179)
> org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:68)
> org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:143)
> org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:206)
> org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:84)
> org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:203)
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:185)
> java.lang.Thread.run(Thread.java:595)
> Object@58c1c2f1 lock acquired at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40) below:
> Name: TempQueue: c34c73c1-d2e9-41f3-89a6-82f2c59ca62b:1
> State: RUNNABLE
> Total blocked: 55 Total waited: 11
> Stack trace:
> java.net.SocketOutputStream.socketWrite0(Native Method)
> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
> java.net.SocketOutputStream.write(SocketOutputStream.java:136)
> com.sun.net.ssl.internal.ssl.OutputRecord.writeBuffer(OutputRecord.java:295)
> com.sun.net.ssl.internal.ssl.OutputRecord.write(OutputRecord.java:284)
> com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:727)
> com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:715)
> com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:59)
> org.apache.activemq.transport.tcp.TcpBufferedOutputStream.flush(TcpBufferedOutputStream.java:115)
> java.io.DataOutputStream.flush(DataOutputStream.java:106)
> org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:167)
> org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:233)
> org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:83)
> org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:100)
> org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
> org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:1207)
> org.apache.activemq.broker.TransportConnection.processDispatch(TransportConnection.java:793)
> org.apache.activemq.broker.TransportConnection.dispatchSync(TransportConnection.java:752)
> org.apache.activemq.broker.region.PrefetchSubscription.dispatch(PrefetchSubscription.java:640)
> org.apache.activemq.broker.region.PrefetchSubscription.dispatchPending(PrefetchSubscription.java:588)
> org.apache.activemq.broker.region.PrefetchSubscription.add(PrefetchSubscription.java:150)
> org.apache.activemq.broker.region.Queue.doActualDispatch(Queue.java:1266)
> org.apache.activemq.broker.region.Queue.doDispatch(Queue.java:1223)
> org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:1308)
> org.apache.activemq.broker.region.Queue.iterate(Queue.java:1011)
> org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:98)
> org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:36)
> Let me know if I can do anything else to help.
> --
> Mats Henrikson
> Software Engineer
> SunGard Kiodex
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.