You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by DraCzech <dr...@gmail.com> on 2016/11/21 10:22:20 UTC
Qpid JMS client JmsConnection.close() hangs
Hi, I'm currently using ActiveMQ 5.14.1 embedded broker connected from
several local queues to MS Azure IoT Hub using JmsConnectionFactory as
foreignConnectionFactory in SimpleJmsQueueConnector. (Using dependency to
qpid jms client 0.11.1) Each queue is connected to the server via its
respective bridge and everything seems to be working fine until I want to
shut down the connection.
If I have network connection, everything is stopped in few seconds. However
if I lose network connection and call SimpleJmsQueueConnector.stop() method,
it hangs on its foreignConnection.close() (JmsConnection). It remains stuck
indefinitely and I have to interrupt the thread responsible for closing of
the connection. Which unfortunately leads to more problems later as there
are remains of the connection in the system.
*Part of the thread dump:*
2016-11-16 09:35:13.275227 - parking to wait for <0x42e6bc08> (a
java.util.concurrent.CountDownLatch$Sync)
2016-11-16 09:35:13.275654 at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
2016-11-16 09:35:13.276143 at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
2016-11-16 09:35:13.276631 at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
2016-11-16 09:35:13.277058 at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
2016-11-16 09:35:13.277577 at
java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
2016-11-16 09:35:13.278676 at
org.apache.qpid.jms.provider.ProviderFuture.sync(ProviderFuture.java:97)
2016-11-16 09:35:13.279103 at
org.apache.qpid.jms.JmsConnection.close(JmsConnection.java:187)
2016-11-16 09:35:13.279622 - locked <0x487b1cb8> (a
org.apache.qpid.jms.JmsQueueConnection)
2016-11-16 09:35:13.280018 at
org.apache.activemq.network.jms.JmsConnector.stop(JmsConnector.java:179)
I believe this problem is tied to the JMS specification and is caused by
MessageConsumer.receive() waiting on some pending receive. I've found some
description of the same or similar issue online, but no specific solution
for it. From what I've read I think the problem might be related to:
http://qpid.2158936.n2.nabble.com/Qpid-JMS-0-9-0-receive-hanging-issue-relates-to-Artemis-works-on-ActiveMQ-td7642585.html
<http://qpid.2158936.n2.nabble.com/Qpid-JMS-0-9-0-receive-hanging-issue-relates-to-Artemis-works-on-ActiveMQ-td7642585.html>
and
https://issues.apache.org/jira/browse/QPIDJMS-221
<https://issues.apache.org/jira/browse/QPIDJMS-221>
Could you please tell me if it's possible to set up my messaging in such a
way which prevents this kind of freeze? How should I defend against this
kind of problems?
Thanks!
--
View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-JMS-client-JmsConnection-close-hangs-tp7654565.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid JMS client JmsConnection.close() hangs
Posted by Robbie Gemmell <ro...@gmail.com>.
I've made some further changes to resolve an issue that might have
been something you hit in your scenario, where if you have the JMS 1.1
and JMS2 APIs both on the classpath it could fail during send, so I've
tweaked the client to tolerate that situation. Regarding the
thread/heap usage, thats likely related to something we are currently
looking into, but yes creating lots of connections would be needed for
it to come into play.
On 13 December 2016 at 16:19, Milano Nicolum <dr...@gmail.com> wrote:
> Unfortunately the latest update is not working for me either. Though it
> seems that connection is established to the IoT hub, the messages cannot be
> delivered. If I look at the AMQPS frames, I can see only Begin and Empty
> frames being exchanged.
> Also AmqpProvider and QpidJMS Connection Executor threads are somehow
> created in such extent that my JVM ran out of heap and my CPU is
> overloaded. Connection restarting policy might be responsible for this,
> because it is checking messages waiting for acknowledgement and tries to
> reconnect if any are detected.
>
> I decided I'll try different approach and discard ActiveMQ bridges entirely
> and try to pass all messages via single JmsConnection wrapped by my code.
> That way the whole process will be more likely under my control than under
> ActiveMQ's control. And while it surely requires some additional coding, I
> hope the result is worth it.
>
>
> 2016-12-13 15:15 GMT+01:00 Robbie Gemmell <ro...@gmail.com>:
>
>> I've made some changes to handle sending foreign messages from JMS 1.1
>> providers and published a new snapshot, you should be able to give
>> that a try now.
>>
>> Robbie
>>
>> On 13 December 2016 at 10:47, Robbie Gemmell <ro...@gmail.com>
>> wrote:
>> > Yes, that appears to be the client trying to call setJMSDeliveryTime
>> > on the Message, only to find the Message object impl doesnt support
>> > that due to being from a JMS 1.1 impl. Will need to have a think about
>> > how best to handle that. As a workaround currently you'd need to
>> > either modify the client not to make that call, or the ActiveMQ bridge
>> > to wrap the ActiveMQ Message and no-op the JMS 2 method before its
>> > passed to the client.
>> >
>> > To your previous question, there isnt a fixed date for the release, we
>> > will do it when we think its ready. If only because I'll be taking
>> > heading on vacation soon, some point in January is my current
>> > thinking.
>> >
>> > Robbie
>> >
>> > On 13 December 2016 at 08:18, Milano Nicolum <dr...@gmail.com> wrote:
>> >> Seems to me that Qpid JMS client is not compatible with my setup. I
>> tried
>> >> to use latest snapshot as dependency, but at the moment I'm getting:
>> >>
>> >> 2016-12-13 07:59:11,815 | ERROR | Error in thread 'ActiveMQ Session
>> Task-1'
>> >> | org.apache.activemq.thread.TaskRunnerFactory | ActiveMQ Session
>> Task-1
>> >> java.lang.AbstractMethodError:
>> >> org.apache.activemq.command.ActiveMQBytesMessage.setJMSDeliveryTime(J)V
>> >> at org.apache.qpid.jms.JmsSession.send(JmsSession.java:772)
>> >> at org.apache.qpid.jms.JmsSession.send(JmsSession.java:742)
>> >> at
>> >> org.apache.qpid.jms.JmsMessageProducer.sendMessage(
>> JmsMessageProducer.java:240)
>> >> at
>> >> org.apache.qpid.jms.JmsMessageProducer.send(
>> JmsMessageProducer.java:188)
>> >> at
>> >> org.apache.qpid.jms.JmsMessageProducer.send(
>> JmsMessageProducer.java:175)
>> >> at org.apache.qpid.jms.JmsQueueSender.send(JmsQueueSender.java:50)
>> >> at
>> >> org.apache.activemq.network.jms.QueueBridge.sendMessage(
>> QueueBridge.java:83)
>> >> at
>> >> org.apache.activemq.network.jms.DestinationBridge.
>> onMessage(DestinationBridge.java:135)
>> >> at
>> >> org.apache.activemq.ActiveMQMessageConsumer.dispatch(
>> ActiveMQMessageConsumer.java:1401)
>> >> at
>> >> org.apache.activemq.ActiveMQSessionExecutor.dispatch(
>> ActiveMQSessionExecutor.java:131)
>> >> at
>> >> org.apache.activemq.ActiveMQSessionExecutor.iterate(
>> ActiveMQSessionExecutor.java:202)
>> >> at
>> >> org.apache.activemq.thread.PooledTaskRunner.runTask(
>> PooledTaskRunner.java:133)
>> >> at
>> >> org.apache.activemq.thread.PooledTaskRunner$1.run(
>> PooledTaskRunner.java:48)
>> >> at
>> >> java.util.concurrent.ThreadPoolExecutor.runWorker(
>> ThreadPoolExecutor.java:1145)
>> >> at
>> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(
>> ThreadPoolExecutor.java:615)
>> >> at java.lang.Thread.run(Thread.java:745)
>> >>
>> >>
>> >> So every time a message is to be sent, it fails on this error. I guess
>> it
>> >> is some kind of incompatibility between JMS 1.1 (used by ActiveMQ) and
>> JMS
>> >> 2.0 (used in latest Qpid JMS client). But it might as well be anything
>> else.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid JMS client JmsConnection.close() hangs
Posted by Milano Nicolum <dr...@gmail.com>.
Unfortunately the latest update is not working for me either. Though it
seems that connection is established to the IoT hub, the messages cannot be
delivered. If I look at the AMQPS frames, I can see only Begin and Empty
frames being exchanged.
Also AmqpProvider and QpidJMS Connection Executor threads are somehow
created in such extent that my JVM ran out of heap and my CPU is
overloaded. Connection restarting policy might be responsible for this,
because it is checking messages waiting for acknowledgement and tries to
reconnect if any are detected.
I decided I'll try different approach and discard ActiveMQ bridges entirely
and try to pass all messages via single JmsConnection wrapped by my code.
That way the whole process will be more likely under my control than under
ActiveMQ's control. And while it surely requires some additional coding, I
hope the result is worth it.
2016-12-13 15:15 GMT+01:00 Robbie Gemmell <ro...@gmail.com>:
> I've made some changes to handle sending foreign messages from JMS 1.1
> providers and published a new snapshot, you should be able to give
> that a try now.
>
> Robbie
>
> On 13 December 2016 at 10:47, Robbie Gemmell <ro...@gmail.com>
> wrote:
> > Yes, that appears to be the client trying to call setJMSDeliveryTime
> > on the Message, only to find the Message object impl doesnt support
> > that due to being from a JMS 1.1 impl. Will need to have a think about
> > how best to handle that. As a workaround currently you'd need to
> > either modify the client not to make that call, or the ActiveMQ bridge
> > to wrap the ActiveMQ Message and no-op the JMS 2 method before its
> > passed to the client.
> >
> > To your previous question, there isnt a fixed date for the release, we
> > will do it when we think its ready. If only because I'll be taking
> > heading on vacation soon, some point in January is my current
> > thinking.
> >
> > Robbie
> >
> > On 13 December 2016 at 08:18, Milano Nicolum <dr...@gmail.com> wrote:
> >> Seems to me that Qpid JMS client is not compatible with my setup. I
> tried
> >> to use latest snapshot as dependency, but at the moment I'm getting:
> >>
> >> 2016-12-13 07:59:11,815 | ERROR | Error in thread 'ActiveMQ Session
> Task-1'
> >> | org.apache.activemq.thread.TaskRunnerFactory | ActiveMQ Session
> Task-1
> >> java.lang.AbstractMethodError:
> >> org.apache.activemq.command.ActiveMQBytesMessage.setJMSDeliveryTime(J)V
> >> at org.apache.qpid.jms.JmsSession.send(JmsSession.java:772)
> >> at org.apache.qpid.jms.JmsSession.send(JmsSession.java:742)
> >> at
> >> org.apache.qpid.jms.JmsMessageProducer.sendMessage(
> JmsMessageProducer.java:240)
> >> at
> >> org.apache.qpid.jms.JmsMessageProducer.send(
> JmsMessageProducer.java:188)
> >> at
> >> org.apache.qpid.jms.JmsMessageProducer.send(
> JmsMessageProducer.java:175)
> >> at org.apache.qpid.jms.JmsQueueSender.send(JmsQueueSender.java:50)
> >> at
> >> org.apache.activemq.network.jms.QueueBridge.sendMessage(
> QueueBridge.java:83)
> >> at
> >> org.apache.activemq.network.jms.DestinationBridge.
> onMessage(DestinationBridge.java:135)
> >> at
> >> org.apache.activemq.ActiveMQMessageConsumer.dispatch(
> ActiveMQMessageConsumer.java:1401)
> >> at
> >> org.apache.activemq.ActiveMQSessionExecutor.dispatch(
> ActiveMQSessionExecutor.java:131)
> >> at
> >> org.apache.activemq.ActiveMQSessionExecutor.iterate(
> ActiveMQSessionExecutor.java:202)
> >> at
> >> org.apache.activemq.thread.PooledTaskRunner.runTask(
> PooledTaskRunner.java:133)
> >> at
> >> org.apache.activemq.thread.PooledTaskRunner$1.run(
> PooledTaskRunner.java:48)
> >> at
> >> java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1145)
> >> at
> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:615)
> >> at java.lang.Thread.run(Thread.java:745)
> >>
> >>
> >> So every time a message is to be sent, it fails on this error. I guess
> it
> >> is some kind of incompatibility between JMS 1.1 (used by ActiveMQ) and
> JMS
> >> 2.0 (used in latest Qpid JMS client). But it might as well be anything
> else.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Qpid JMS client JmsConnection.close() hangs
Posted by Robbie Gemmell <ro...@gmail.com>.
I've made some changes to handle sending foreign messages from JMS 1.1
providers and published a new snapshot, you should be able to give
that a try now.
Robbie
On 13 December 2016 at 10:47, Robbie Gemmell <ro...@gmail.com> wrote:
> Yes, that appears to be the client trying to call setJMSDeliveryTime
> on the Message, only to find the Message object impl doesnt support
> that due to being from a JMS 1.1 impl. Will need to have a think about
> how best to handle that. As a workaround currently you'd need to
> either modify the client not to make that call, or the ActiveMQ bridge
> to wrap the ActiveMQ Message and no-op the JMS 2 method before its
> passed to the client.
>
> To your previous question, there isnt a fixed date for the release, we
> will do it when we think its ready. If only because I'll be taking
> heading on vacation soon, some point in January is my current
> thinking.
>
> Robbie
>
> On 13 December 2016 at 08:18, Milano Nicolum <dr...@gmail.com> wrote:
>> Seems to me that Qpid JMS client is not compatible with my setup. I tried
>> to use latest snapshot as dependency, but at the moment I'm getting:
>>
>> 2016-12-13 07:59:11,815 | ERROR | Error in thread 'ActiveMQ Session Task-1'
>> | org.apache.activemq.thread.TaskRunnerFactory | ActiveMQ Session Task-1
>> java.lang.AbstractMethodError:
>> org.apache.activemq.command.ActiveMQBytesMessage.setJMSDeliveryTime(J)V
>> at org.apache.qpid.jms.JmsSession.send(JmsSession.java:772)
>> at org.apache.qpid.jms.JmsSession.send(JmsSession.java:742)
>> at
>> org.apache.qpid.jms.JmsMessageProducer.sendMessage(JmsMessageProducer.java:240)
>> at
>> org.apache.qpid.jms.JmsMessageProducer.send(JmsMessageProducer.java:188)
>> at
>> org.apache.qpid.jms.JmsMessageProducer.send(JmsMessageProducer.java:175)
>> at org.apache.qpid.jms.JmsQueueSender.send(JmsQueueSender.java:50)
>> at
>> org.apache.activemq.network.jms.QueueBridge.sendMessage(QueueBridge.java:83)
>> at
>> org.apache.activemq.network.jms.DestinationBridge.onMessage(DestinationBridge.java:135)
>> at
>> org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1401)
>> at
>> org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131)
>> at
>> org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202)
>> at
>> org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)
>> at
>> org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> at java.lang.Thread.run(Thread.java:745)
>>
>>
>> So every time a message is to be sent, it fails on this error. I guess it
>> is some kind of incompatibility between JMS 1.1 (used by ActiveMQ) and JMS
>> 2.0 (used in latest Qpid JMS client). But it might as well be anything else.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid JMS client JmsConnection.close() hangs
Posted by Robbie Gemmell <ro...@gmail.com>.
Yes, that appears to be the client trying to call setJMSDeliveryTime
on the Message, only to find the Message object impl doesnt support
that due to being from a JMS 1.1 impl. Will need to have a think about
how best to handle that. As a workaround currently you'd need to
either modify the client not to make that call, or the ActiveMQ bridge
to wrap the ActiveMQ Message and no-op the JMS 2 method before its
passed to the client.
To your previous question, there isnt a fixed date for the release, we
will do it when we think its ready. If only because I'll be taking
heading on vacation soon, some point in January is my current
thinking.
Robbie
On 13 December 2016 at 08:18, Milano Nicolum <dr...@gmail.com> wrote:
> Seems to me that Qpid JMS client is not compatible with my setup. I tried
> to use latest snapshot as dependency, but at the moment I'm getting:
>
> 2016-12-13 07:59:11,815 | ERROR | Error in thread 'ActiveMQ Session Task-1'
> | org.apache.activemq.thread.TaskRunnerFactory | ActiveMQ Session Task-1
> java.lang.AbstractMethodError:
> org.apache.activemq.command.ActiveMQBytesMessage.setJMSDeliveryTime(J)V
> at org.apache.qpid.jms.JmsSession.send(JmsSession.java:772)
> at org.apache.qpid.jms.JmsSession.send(JmsSession.java:742)
> at
> org.apache.qpid.jms.JmsMessageProducer.sendMessage(JmsMessageProducer.java:240)
> at
> org.apache.qpid.jms.JmsMessageProducer.send(JmsMessageProducer.java:188)
> at
> org.apache.qpid.jms.JmsMessageProducer.send(JmsMessageProducer.java:175)
> at org.apache.qpid.jms.JmsQueueSender.send(JmsQueueSender.java:50)
> at
> org.apache.activemq.network.jms.QueueBridge.sendMessage(QueueBridge.java:83)
> at
> org.apache.activemq.network.jms.DestinationBridge.onMessage(DestinationBridge.java:135)
> at
> org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1401)
> at
> org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131)
> at
> org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202)
> at
> org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)
> at
> org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
>
>
> So every time a message is to be sent, it fails on this error. I guess it
> is some kind of incompatibility between JMS 1.1 (used by ActiveMQ) and JMS
> 2.0 (used in latest Qpid JMS client). But it might as well be anything else.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid JMS client JmsConnection.close() hangs
Posted by Milano Nicolum <dr...@gmail.com>.
Seems to me that Qpid JMS client is not compatible with my setup. I tried
to use latest snapshot as dependency, but at the moment I'm getting:
2016-12-13 07:59:11,815 | ERROR | Error in thread 'ActiveMQ Session Task-1'
| org.apache.activemq.thread.TaskRunnerFactory | ActiveMQ Session Task-1
java.lang.AbstractMethodError:
org.apache.activemq.command.ActiveMQBytesMessage.setJMSDeliveryTime(J)V
at org.apache.qpid.jms.JmsSession.send(JmsSession.java:772)
at org.apache.qpid.jms.JmsSession.send(JmsSession.java:742)
at
org.apache.qpid.jms.JmsMessageProducer.sendMessage(JmsMessageProducer.java:240)
at
org.apache.qpid.jms.JmsMessageProducer.send(JmsMessageProducer.java:188)
at
org.apache.qpid.jms.JmsMessageProducer.send(JmsMessageProducer.java:175)
at org.apache.qpid.jms.JmsQueueSender.send(JmsQueueSender.java:50)
at
org.apache.activemq.network.jms.QueueBridge.sendMessage(QueueBridge.java:83)
at
org.apache.activemq.network.jms.DestinationBridge.onMessage(DestinationBridge.java:135)
at
org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1401)
at
org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131)
at
org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202)
at
org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)
at
org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
So every time a message is to be sent, it fails on this error. I guess it
is some kind of incompatibility between JMS 1.1 (used by ActiveMQ) and JMS
2.0 (used in latest Qpid JMS client). But it might as well be anything else.
Re: Qpid JMS client JmsConnection.close() hangs
Posted by Milano Nicolum <dr...@gmail.com>.
Tomorow I plan to do some testing with 0.20.0-SNAPSHOT. The thing is I can
hardly use snapshots in my company's software (next release version of the
OSGi bundle I'm currently working on). So even if the fix made in
QPIDJMS-228 solves all my problems I have to wait for a release or find a
workaround. So I wanted to ask if there are any plans for 0.20.0 release at
the moment. Is it scheduled for January 2017, Q1 of next year or still not
decided at the moment?
Thanks for any information on this!
2016-12-01 17:35 GMT+01:00 Timothy Bish <ta...@gmail.com>:
> Close timeout handling has been fixed up a fair bit with issue:
> https://issues.apache.org/jira/browse/QPIDJMS-228
>
>
> On 11/25/2016 11:04 AM, Robbie Gemmell wrote:
>
>> "I also think I see an issue with the closeTimeout handling in that in
>> one place it seems likely to be using the requestTimeout value
>> instead. Need to investigate that further."
>>
>> Though in my attemps that didnt actually matter and the close
>> completed fine if the network dropped and no response could be
>> received (or was ever sent). As I say, needs more investigation.
>>
>> On 25 November 2016 at 15:49, DraCzech <dr...@gmail.com> wrote:
>>
>>> I'll try to find a way of preventing this problem from happening.
>>> (Though I'm
>>> not sure how to do that properly at the moment.)
>>>
>>> But how do you explain unlimited waiting for Close frame response as
>>> shown
>>> in my previous post?
>>>
>>>
>>> DraCzech wrote
>>>
>>>> 2016-11-24 16:41:45.202589 [19105288:0] -> Close{error=null}
>>>>
>>>> on network lost vs.
>>>>
>>>> 2016-11-24 16:51:31.027968 [18734493:0] -> Close{error=null}
>>>> 2016-11-24 16:51:31.977722 [18734493:0] <- Close{error=null}
>>>>
>>> Shouldn't some timeout throw an exception or so?
>>>
>>>
>>>
>>> --
>>> View this message in context: http://qpid.2158936.n2.nabble.
>>> com/Qpid-JMS-client-JmsConnection-close-hangs-tp7654565p7654872.html
>>> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>>> For additional commands, e-mail: users-help@qpid.apache.org
>>>
>>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
>>
>>
>
> --
> Tim Bish
> twitter: @tabish121
> blog: http://timbish.blogspot.com/
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: Qpid JMS client JmsConnection.close() hangs
Posted by DraCzech <dr...@gmail.com>.
Thanks! I'll try that as soon as 0.20.0 is released!
This might also help with another issue I'm experiencing - The
ThreadPoolExecutor in JmsConnection is producing non-daemon threads
intentionally (I can see the comment, but still don't get the reason why to
do it in your code.) Which means in case any QpidJMS Connection Executor
thread gets stuck (on lock, indefinite timeout etc.) it prevents the device
I'm using from clean shutdown. As I have no means of killing those threads
from my code.
Any chances this might become configurable with default value set to
nondaemon?
--
View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-JMS-client-JmsConnection-close-hangs-tp7654565p7655299.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid JMS client JmsConnection.close() hangs
Posted by Timothy Bish <ta...@gmail.com>.
Close timeout handling has been fixed up a fair bit with issue:
https://issues.apache.org/jira/browse/QPIDJMS-228
On 11/25/2016 11:04 AM, Robbie Gemmell wrote:
> "I also think I see an issue with the closeTimeout handling in that in
> one place it seems likely to be using the requestTimeout value
> instead. Need to investigate that further."
>
> Though in my attemps that didnt actually matter and the close
> completed fine if the network dropped and no response could be
> received (or was ever sent). As I say, needs more investigation.
>
> On 25 November 2016 at 15:49, DraCzech <dr...@gmail.com> wrote:
>> I'll try to find a way of preventing this problem from happening. (Though I'm
>> not sure how to do that properly at the moment.)
>>
>> But how do you explain unlimited waiting for Close frame response as shown
>> in my previous post?
>>
>>
>> DraCzech wrote
>>> 2016-11-24 16:41:45.202589 [19105288:0] -> Close{error=null}
>>>
>>> on network lost vs.
>>>
>>> 2016-11-24 16:51:31.027968 [18734493:0] -> Close{error=null}
>>> 2016-11-24 16:51:31.977722 [18734493:0] <- Close{error=null}
>> Shouldn't some timeout throw an exception or so?
>>
>>
>>
>> --
>> View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-JMS-client-JmsConnection-close-hangs-tp7654565p7654872.html
>> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
>> For additional commands, e-mail: users-help@qpid.apache.org
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
--
Tim Bish
twitter: @tabish121
blog: http://timbish.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid JMS client JmsConnection.close() hangs
Posted by Robbie Gemmell <ro...@gmail.com>.
"I also think I see an issue with the closeTimeout handling in that in
one place it seems likely to be using the requestTimeout value
instead. Need to investigate that further."
Though in my attemps that didnt actually matter and the close
completed fine if the network dropped and no response could be
received (or was ever sent). As I say, needs more investigation.
On 25 November 2016 at 15:49, DraCzech <dr...@gmail.com> wrote:
> I'll try to find a way of preventing this problem from happening. (Though I'm
> not sure how to do that properly at the moment.)
>
> But how do you explain unlimited waiting for Close frame response as shown
> in my previous post?
>
>
> DraCzech wrote
>> 2016-11-24 16:41:45.202589 [19105288:0] -> Close{error=null}
>>
>> on network lost vs.
>>
>> 2016-11-24 16:51:31.027968 [18734493:0] -> Close{error=null}
>> 2016-11-24 16:51:31.977722 [18734493:0] <- Close{error=null}
>
> Shouldn't some timeout throw an exception or so?
>
>
>
> --
> View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-JMS-client-JmsConnection-close-hangs-tp7654565p7654872.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid JMS client JmsConnection.close() hangs
Posted by DraCzech <dr...@gmail.com>.
I'll try to find a way of preventing this problem from happening. (Though I'm
not sure how to do that properly at the moment.)
But how do you explain unlimited waiting for Close frame response as shown
in my previous post?
DraCzech wrote
> 2016-11-24 16:41:45.202589 [19105288:0] -> Close{error=null}
>
> on network lost vs.
>
> 2016-11-24 16:51:31.027968 [18734493:0] -> Close{error=null}
> 2016-11-24 16:51:31.977722 [18734493:0] <- Close{error=null}
Shouldn't some timeout throw an exception or so?
--
View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-JMS-client-JmsConnection-close-hangs-tp7654565p7654872.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid JMS client JmsConnection.close() hangs
Posted by Robbie Gemmell <ro...@gmail.com>.
Yes that looks like its trying to get a lock which the onMessage
handler is holding while firing, which is ongoing due to a send within
it.
On 25 November 2016 at 07:37, DraCzech <dr...@gmail.com> wrote:
> When going through logs I also found this piece of thread dump. Which shows
> that my cleaning thread is waiting for a lock held by ActiveMQ thread. A
> state which is definitely not desired, but I'm not sure if can be prevented
> completely...
>
> 2016-11-24 17:46:10.016523 "JMS Connector cleaner" daemon prio=10
> tid=0x0005bf70 nid=0x9d7 waiting for monitor entry [0x555bb000]
> 2016-11-24 17:46:10.016829 java.lang.Thread.State: BLOCKED (on object
> monitor)
> 2016-11-24 17:46:10.017286 at
> org.apache.activemq.FifoMessageDispatchChannel.stop(FifoMessageDispatchChannel.java:124)
> 2016-11-24 17:46:10.018599 - waiting to lock <0x489bcd68> (a
> java.lang.Object)
> 2016-11-24 17:46:10.018996 at
> org.apache.activemq.ActiveMQMessageConsumer.stop(ActiveMQMessageConsumer.java:1551)
> 2016-11-24 17:46:10.019423 at
> org.apache.activemq.ActiveMQSession.stop(ActiveMQSession.java:1856)
> 2016-11-24 17:46:10.019942 at
> org.apache.activemq.ActiveMQConnection.doStop(ActiveMQConnection.java:586)
> 2016-11-24 17:46:10.020338 - locked <0x4832e608> (a
> java.util.concurrent.CopyOnWriteArrayList)
> 2016-11-24 17:46:10.020827 at
> org.apache.activemq.ActiveMQConnection.close(ActiveMQConnection.java:642)
> 2016-11-24 17:46:10.021193 at
> org.apache.activemq.network.jms.JmsConnector.stop(JmsConnector.java:186)
> 2016-11-24 17:46:10.022109 at
> com.openmatics.box.iotdatabroker.internal.connection.ServerConnector$JMSConnectorCleaner.run(ServerConnector.java:645)
> 2016-11-24 17:46:10.022536 at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> 2016-11-24 17:46:10.023024 at
> java.util.concurrent.FutureTask.run(FutureTask.java:262)
> 2016-11-24 17:46:10.024306 at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 2016-11-24 17:46:10.024764 at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 2016-11-24 17:46:10.025313 at java.lang.Thread.run(Thread.java:745)
> 2016-11-24 17:46:10.038101 "ActiveMQ Session Task-17" prio=10
> tid=0x00271920 nid=0x995 waiting on condition [0x56072000]
> 2016-11-24 17:46:10.038681 java.lang.Thread.State: WAITING (parking)
> 2016-11-24 17:46:10.039749 at sun.misc.Unsafe.park(Native Method)
> 2016-11-24 17:46:10.040024 - parking to wait for <0x44080548> (a
> java.util.concurrent.CountDownLatch$Sync)
> 2016-11-24 17:46:10.040238 at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> 2016-11-24 17:46:10.040421 at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> 2016-11-24 17:46:10.040756 at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> 2016-11-24 17:46:10.040970 at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> 2016-11-24 17:46:10.043106 at
> java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
> 2016-11-24 17:46:10.043351 at
> org.apache.qpid.jms.provider.ProviderFuture.sync(ProviderFuture.java:97)
> 2016-11-24 17:46:10.043625 at
> org.apache.qpid.jms.JmsConnection.send(JmsConnection.java:630)
> 2016-11-24 17:46:10.043839 at
> org.apache.qpid.jms.JmsConnection.send(JmsConnection.java:619)
> 2016-11-24 17:46:10.044144 at
> org.apache.qpid.jms.JmsNoTxTransactionContext.send(JmsNoTxTransactionContext.java:37)
> 2016-11-24 17:46:10.044358 at
> org.apache.qpid.jms.JmsSession.send(JmsSession.java:717)
> 2016-11-24 17:46:10.044571 at
> org.apache.qpid.jms.JmsSession.send(JmsSession.java:634)
> 2016-11-24 17:46:10.044785 at
> org.apache.qpid.jms.JmsMessageProducer.sendMessage(JmsMessageProducer.java:185)
> 2016-11-24 17:46:10.045121 at
> org.apache.qpid.jms.JmsMessageProducer.send(JmsMessageProducer.java:177)
> 2016-11-24 17:46:10.045365 at
> org.apache.qpid.jms.JmsMessageProducer.send(JmsMessageProducer.java:166)
> 2016-11-24 17:46:10.045609 at
> org.apache.qpid.jms.JmsQueueSender.send(JmsQueueSender.java:50)
> 2016-11-24 17:46:10.046403 at
> org.apache.activemq.network.jms.QueueBridge.sendMessage(QueueBridge.java:83)
> 2016-11-24 17:46:10.048692 - locked <0x48366908> (a
> org.apache.activemq.network.jms.OutboundQueueBridge)
> 2016-11-24 17:46:10.048936 at
> org.apache.activemq.network.jms.DestinationBridge.onMessage(DestinationBridge.java:135)
> 2016-11-24 17:46:10.049149 at
> org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1401)
> 2016-11-24 17:46:10.049363 - locked <0x489bcd68> (a java.lang.Object)
> 2016-11-24 17:46:10.049668 at
> org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131)
> 2016-11-24 17:46:10.049882 at
> org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202)
> 2016-11-24 17:46:10.050126 at
> org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)
> 2016-11-24 17:46:10.050340 at
> org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
> 2016-11-24 17:46:10.050553 at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 2016-11-24 17:46:10.050859 at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 2016-11-24 17:46:10.051988 at java.lang.Thread.run(Thread.java:745)
>
>
>
> --
> View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-JMS-client-JmsConnection-close-hangs-tp7654565p7654835.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid JMS client JmsConnection.close() hangs
Posted by DraCzech <dr...@gmail.com>.
When going through logs I also found this piece of thread dump. Which shows
that my cleaning thread is waiting for a lock held by ActiveMQ thread. A
state which is definitely not desired, but I'm not sure if can be prevented
completely...
2016-11-24 17:46:10.016523 "JMS Connector cleaner" daemon prio=10
tid=0x0005bf70 nid=0x9d7 waiting for monitor entry [0x555bb000]
2016-11-24 17:46:10.016829 java.lang.Thread.State: BLOCKED (on object
monitor)
2016-11-24 17:46:10.017286 at
org.apache.activemq.FifoMessageDispatchChannel.stop(FifoMessageDispatchChannel.java:124)
2016-11-24 17:46:10.018599 - waiting to lock <0x489bcd68> (a
java.lang.Object)
2016-11-24 17:46:10.018996 at
org.apache.activemq.ActiveMQMessageConsumer.stop(ActiveMQMessageConsumer.java:1551)
2016-11-24 17:46:10.019423 at
org.apache.activemq.ActiveMQSession.stop(ActiveMQSession.java:1856)
2016-11-24 17:46:10.019942 at
org.apache.activemq.ActiveMQConnection.doStop(ActiveMQConnection.java:586)
2016-11-24 17:46:10.020338 - locked <0x4832e608> (a
java.util.concurrent.CopyOnWriteArrayList)
2016-11-24 17:46:10.020827 at
org.apache.activemq.ActiveMQConnection.close(ActiveMQConnection.java:642)
2016-11-24 17:46:10.021193 at
org.apache.activemq.network.jms.JmsConnector.stop(JmsConnector.java:186)
2016-11-24 17:46:10.022109 at
com.openmatics.box.iotdatabroker.internal.connection.ServerConnector$JMSConnectorCleaner.run(ServerConnector.java:645)
2016-11-24 17:46:10.022536 at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
2016-11-24 17:46:10.023024 at
java.util.concurrent.FutureTask.run(FutureTask.java:262)
2016-11-24 17:46:10.024306 at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
2016-11-24 17:46:10.024764 at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
2016-11-24 17:46:10.025313 at java.lang.Thread.run(Thread.java:745)
2016-11-24 17:46:10.038101 "ActiveMQ Session Task-17" prio=10
tid=0x00271920 nid=0x995 waiting on condition [0x56072000]
2016-11-24 17:46:10.038681 java.lang.Thread.State: WAITING (parking)
2016-11-24 17:46:10.039749 at sun.misc.Unsafe.park(Native Method)
2016-11-24 17:46:10.040024 - parking to wait for <0x44080548> (a
java.util.concurrent.CountDownLatch$Sync)
2016-11-24 17:46:10.040238 at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
2016-11-24 17:46:10.040421 at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
2016-11-24 17:46:10.040756 at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
2016-11-24 17:46:10.040970 at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
2016-11-24 17:46:10.043106 at
java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
2016-11-24 17:46:10.043351 at
org.apache.qpid.jms.provider.ProviderFuture.sync(ProviderFuture.java:97)
2016-11-24 17:46:10.043625 at
org.apache.qpid.jms.JmsConnection.send(JmsConnection.java:630)
2016-11-24 17:46:10.043839 at
org.apache.qpid.jms.JmsConnection.send(JmsConnection.java:619)
2016-11-24 17:46:10.044144 at
org.apache.qpid.jms.JmsNoTxTransactionContext.send(JmsNoTxTransactionContext.java:37)
2016-11-24 17:46:10.044358 at
org.apache.qpid.jms.JmsSession.send(JmsSession.java:717)
2016-11-24 17:46:10.044571 at
org.apache.qpid.jms.JmsSession.send(JmsSession.java:634)
2016-11-24 17:46:10.044785 at
org.apache.qpid.jms.JmsMessageProducer.sendMessage(JmsMessageProducer.java:185)
2016-11-24 17:46:10.045121 at
org.apache.qpid.jms.JmsMessageProducer.send(JmsMessageProducer.java:177)
2016-11-24 17:46:10.045365 at
org.apache.qpid.jms.JmsMessageProducer.send(JmsMessageProducer.java:166)
2016-11-24 17:46:10.045609 at
org.apache.qpid.jms.JmsQueueSender.send(JmsQueueSender.java:50)
2016-11-24 17:46:10.046403 at
org.apache.activemq.network.jms.QueueBridge.sendMessage(QueueBridge.java:83)
2016-11-24 17:46:10.048692 - locked <0x48366908> (a
org.apache.activemq.network.jms.OutboundQueueBridge)
2016-11-24 17:46:10.048936 at
org.apache.activemq.network.jms.DestinationBridge.onMessage(DestinationBridge.java:135)
2016-11-24 17:46:10.049149 at
org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1401)
2016-11-24 17:46:10.049363 - locked <0x489bcd68> (a java.lang.Object)
2016-11-24 17:46:10.049668 at
org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131)
2016-11-24 17:46:10.049882 at
org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202)
2016-11-24 17:46:10.050126 at
org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)
2016-11-24 17:46:10.050340 at
org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
2016-11-24 17:46:10.050553 at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
2016-11-24 17:46:10.050859 at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
2016-11-24 17:46:10.051988 at java.lang.Thread.run(Thread.java:745)
--
View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-JMS-client-JmsConnection-close-hangs-tp7654565p7654835.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid JMS client JmsConnection.close() hangs
Posted by Robbie Gemmell <ro...@gmail.com>.
I had a look at this on master and though I did see a missing guard
(diff below) on the connection close that could be related, I haven't
reproduced what you are seeing even without that. It may still be
worth you giving it a try out.
Your logging snippets only include protons protocol trace, it might
also help to turn on some of the other client logging.
I also think I see an issue with the closeTimeout handling in that in
one place it seems likely to be using the requestTimeout value
instead. Need to investigate that further.
I'm heading on vacation so itll be late next week before I get back to
having a further look.
+++ b/qpid-jms-client/src/main/java/org/apache/qpid/jms/JmsConnection.java
@@ -185,6 +185,7 @@ public class JmsConnection implements
AutoCloseable, Connection, TopicConnection
if (isConnected() && !failed.get()) {
ProviderFuture request = new ProviderFuture();
try {
+ requests.put(request, request);
provider.destroy(connectionInfo, request);
try {
@@ -200,6 +201,8 @@ public class JmsConnection implements
AutoCloseable, Connection, TopicConnection
}
} catch(ProviderClosedException pce) {
LOG.debug("Ignoring provider closed exception
during connection close");
+ } finally {
+ requests.remove(request);
}
}
On 24 November 2016 at 17:08, DraCzech <dr...@gmail.com> wrote:
> So I tried to turn on logging on qpid and this is what I found so far.
>
> In case I shut down the messaging bundle, it sends *Close*, receives
> *Close*, everything goes fine.
>
> But In case the device loses network connection it sends *Close* and waits
> for response which never comes.
>
> I'm using factory.setCloseTimeout(8000) and amqp.idleTimeout=120000 on my
> URL. I'd expect one of these parameters to fire some exception after
> respective period of time. But every time I lose network the cleaning thread
> hangs and I have to kill it manually (currently timeout is 25s).
>
> *Network lost:*
> 2016-11-24 16:41:38.315759 [19105288:4] ->
> Attach{name='qpid-jms:sender:ID:7fa560e5-9f45-43ab-9215-2b6021fec6d0:3:-1:15:/devices/7C9763001961/messages/events',
> handle=0, role=SENDER, sndSettleMode=UNSETTLED, rcvSettleMode=FIRST,
> source=Source{address='ID:7fa560e5-9f45-43ab-9215-2b6021fec6d0:3:-1:15',
> durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false,
> dynamicNodeProperties=null, distributionMode=null, filter=null,
> defaultOutcome=null, outcomes=[amqp:accepted:list, amqp:rejected:list],
> capabilities=null},
> target=Target{address='/devices/7C9763001961/messages/events', durable=NONE,
> expiryPolicy=SESSION_END, timeout=0, dynamic=false,
> dynamicNodeProperties=null, capabilities=[queue]}, unsettled=null,
> incompleteUnsettled=false, initialDeliveryCount=0, maxMessageSize=null,
> offeredCapabilities=null, desiredCapabilities=null, properties=null}
> 2016-11-24 16:41:38.991236 [19105288:4] <-
> Attach{name='qpid-jms:sender:ID:7fa560e5-9f45-43ab-9215-2b6021fec6d0:3:-1:15:/devices/7C9763001961/messages/events',
> handle=0, role=RECEIVER, sndSettleMode=UNSETTLED, rcvSettleMode=FIRST,
> source=Source{address='ID:7fa560e5-9f45-43ab-9215-2b6021fec6d0:3:-1:15',
> durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false,
> dynamicNodeProperties=null, distributionMode=null, filter=null,
> defaultOutcome=null, outcomes=[amqp:accepted:list, amqp:rejected:list],
> capabilities=null},
> target=Target{address='/devices/7C9763001961/messages/events', durable=NONE,
> expiryPolicy=SESSION_END, timeout=0, dynamic=false,
> dynamicNodeProperties=null, capabilities=[queue]}, unsettled=null,
> incompleteUnsettled=false, initialDeliveryCount=null,
> maxMessageSize=1048576, offeredCapabilities=null, desiredCapabilities=null,
> properties=null}
> 2016-11-24 16:41:39.150984 [19105288:4] <- Flow{nextIncomingId=16,
> incomingWindow=5000, nextOutgoingId=1, outgoingWindow=2047, handle=0,
> deliveryCount=0, linkCredit=1000, available=0, drain=false, echo=false,
> properties=null}
> 2016-11-24 16:41:39.182738 [19105288:4] -> Transfer{handle=0,
> deliveryId=15, deliveryTag=0, messageFormat=0, settled=null, more=false,
> rcvSettleMode=null, state=null, resume=false, aborted=false,
> batchable=false} (10823)
> "\x00Sp\xc0\x02\x01A\x00Sr\xc1)\x04\xa3\x12x-opt-jms-msg-typeQ\x03\xa3\x0ex-opt-jms-destQ\x00\x00Ss\xc0\x83\x0a\xa10ID:ec766fb9-8779-4bbd-afe2-604752e8b480:1:6:1-16@\xa1%/devices/7C9763001961/messages/events@@@\xa3\x18application/octet-stream@@\x83\x00\x00\x01X\x976\xc2\xe6\x00St\xc1\xac\x10\xa1\x05boxId\xa1\x0c7C9763001961\xa1\x06scopes\xa1\x06SERVER\xa1\x08priority\xa1\x012\xa1\x04name\xa13com.openmatics.system.fw.config.localchange.request\xa1\x0bcomponentId\xa1\x02-1\xa1\x0atimeToLive\xa1\x02-1\xa1\x07version\xa1\x051.0.0\xa1\x06appKey\xa1\x03-14\x00Su\xb0\x00\x00(\xd1\x0b\x01\x1b\x01\x02com.openmatics.system.fw.config.localchange.reques\xf4\x0c\x03\x00$\x04\x01\x0e\x05\x04\x06\x02\x071.0.\xb0&\x01\x03\x03\x02LOCAL_VARIANT_TIMESTAM\xd0\x09\xe4\x8a\xb6\xf3\x92V\x03\x03LOCAL_VARIANT_ID\xd3&\x04\x01\x03\x05/sbx/config/lcr.xm\xec\x03\x067C9763001961_0148000569814\xb1\x03\x07BOX_CONFIGURATIO\xce\x01\x00\xc3\x01com.openmatics.shared.messaging.box.configuration.BoxConfiguration\x08\x00$\x09\x01\x01\x01com.openmat"...(truncated)
> 2016-11-24 16:41:39.745306 [19105288:4] <- Disposition{role=RECEIVER,
> first=15, last=null, settled=true, state=Accepted{}, batchable=false}
> 2016-11-24 16:41:39.747107 [19105288:4] -> Detach{handle=0, closed=true,
> error=null}
> 2016-11-24 16:41:39.919648 [19105288:4] <- Detach{handle=0, closed=true,
> error=null}
> 2016-11-24 16:41:45.202589 [19105288:0] -> Close{error=null}
>
> *Normal bundle shutdown:*
> 2016-11-24 16:51:11.517312 [18734493:6] ->
> Attach{name='qpid-jms:sender:ID:4c47ee6a-8f04-46bd-9a81-ac1be65b2b5e:15:-1:9:/devices/7C9763001961/messages/events',
> handle=0, role=SENDER, sndSettleMode=UNSETTLED, rcvSettleMode=FIRST,
> source=Source{address='ID:4c47ee6a-8f04-46bd-9a81-ac1be65b2b5e:15:-1:9',
> durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false,
> dynamicNodeProperties=null, distributionMode=null, filter=null,
> defaultOutcome=null, outcomes=[amqp:accepted:list, amqp:rejected:list],
> capabilities=null},
> target=Target{address='/devices/7C9763001961/messages/events', durable=NONE,
> expiryPolicy=SESSION_END, timeout=0, dynamic=false,
> dynamicNodeProperties=null, capabilities=[queue]}, unsettled=null,
> incompleteUnsettled=false, initialDeliveryCount=0, maxMessageSize=null,
> offeredCapabilities=null, desiredCapabilities=null, properties=null}
> 2016-11-24 16:51:11.690158 [18734493:6] <-
> Attach{name='qpid-jms:sender:ID:4c47ee6a-8f04-46bd-9a81-ac1be65b2b5e:15:-1:9:/devices/7C9763001961/messages/events',
> handle=0, role=RECEIVER, sndSettleMode=UNSETTLED, rcvSettleMode=FIRST,
> source=Source{address='ID:4c47ee6a-8f04-46bd-9a81-ac1be65b2b5e:15:-1:9',
> durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false,
> dynamicNodeProperties=null, distributionMode=null, filter=null,
> defaultOutcome=null, outcomes=[amqp:accepted:list, amqp:rejected:list],
> capabilities=null},
> target=Target{address='/devices/7C9763001961/messages/events', durable=NONE,
> expiryPolicy=SESSION_END, timeout=0, dynamic=false,
> dynamicNodeProperties=null, capabilities=[queue]}, unsettled=null,
> incompleteUnsettled=false, initialDeliveryCount=null,
> maxMessageSize=1048576, offeredCapabilities=null, desiredCapabilities=null,
> properties=null}
> 2016-11-24 16:51:11.842212 [18734493:6] <- Flow{nextIncomingId=10,
> incomingWindow=5000, nextOutgoingId=1, outgoingWindow=2047, handle=0,
> deliveryCount=0, linkCredit=1000, available=0, drain=false, echo=false,
> properties=null}
> 2016-11-24 16:51:11.861722 [18734493:6] -> Transfer{handle=0, deliveryId=9,
> deliveryTag=0, messageFormat=0, settled=null, more=false,
> rcvSettleMode=null, state=null, resume=false, aborted=false,
> batchable=false} (443)
> "\x00Sp\xc0\x02\x01A\x00Sr\xc1)\x04\xa3\x12x-opt-jms-msg-typeQ\x03\xa3\x0ex-opt-jms-destQ\x00\x00Ss\xc0\x83\x0a\xa10ID:15ec0e45-968b-4028-a0ac-80cf3e7a62fc:1:8:1-10@\xa1%/devices/7C9763001961/messages/events@@@\xa3\x18application/octet-stream@@\x83\x00\x00\x01X\x97?\x82\x0b\x00St\xc1\x93\x10\xa1\x05boxId\xa1\x0c7C9763001961\xa1\x06scopes\xa1\x06SERVER\xa1\x08priority\xa1\x014\xa1\x04name\xa1\x1bcom.openmatics.system.alive\xa1\x0bcomponentId\xa1\x02-1\xa1\x0atimeToLive\xa1\x02-1\xa1\x07version\xa1\x051.0.0\xa1\x06appKey\xa1\x02-3\x00Su\xa0a\x0b\x01\x05\x01\x02com.openmatics.system.aliv\xe5\x0c\x03\x00$\x04\x01\x0e\x05\x04\x06\x04\x071.0.\xb0&\x01\x03\x03\x02lo\xee\x1a\x03@*\xbc\x82C\x80w#\x03\x04Aliv\xe5\x1c\x05\xb4\xf7\xfb\xf3\x92V\x03\x06la\xf4\x1a\x07@H\xdfd\xf6\xb3\xbf\xcf"
> 2016-11-24 16:51:12.075635 [18734493:6] <- Disposition{role=RECEIVER,
> first=9, last=null, settled=true, state=Accepted{}, batchable=false}
> 2016-11-24 16:51:12.085039 [18734493:6] -> Detach{handle=0, closed=true,
> error=null}
> 2016-11-24 16:51:12.254863 [18734493:6] <- Detach{handle=0, closed=true,
> error=null}
> 2016-11-24 16:51:31.027968 [18734493:0] -> Close{error=null}
> 2016-11-24 16:51:31.977722 [18734493:0] <- Close{error=null}
>
>
>
> --
> View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-JMS-client-JmsConnection-close-hangs-tp7654565p7654821.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid JMS client JmsConnection.close() hangs
Posted by DraCzech <dr...@gmail.com>.
So I tried to turn on logging on qpid and this is what I found so far.
In case I shut down the messaging bundle, it sends *Close*, receives
*Close*, everything goes fine.
But In case the device loses network connection it sends *Close* and waits
for response which never comes.
I'm using factory.setCloseTimeout(8000) and amqp.idleTimeout=120000 on my
URL. I'd expect one of these parameters to fire some exception after
respective period of time. But every time I lose network the cleaning thread
hangs and I have to kill it manually (currently timeout is 25s).
*Network lost:*
2016-11-24 16:41:38.315759 [19105288:4] ->
Attach{name='qpid-jms:sender:ID:7fa560e5-9f45-43ab-9215-2b6021fec6d0:3:-1:15:/devices/7C9763001961/messages/events',
handle=0, role=SENDER, sndSettleMode=UNSETTLED, rcvSettleMode=FIRST,
source=Source{address='ID:7fa560e5-9f45-43ab-9215-2b6021fec6d0:3:-1:15',
durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false,
dynamicNodeProperties=null, distributionMode=null, filter=null,
defaultOutcome=null, outcomes=[amqp:accepted:list, amqp:rejected:list],
capabilities=null},
target=Target{address='/devices/7C9763001961/messages/events', durable=NONE,
expiryPolicy=SESSION_END, timeout=0, dynamic=false,
dynamicNodeProperties=null, capabilities=[queue]}, unsettled=null,
incompleteUnsettled=false, initialDeliveryCount=0, maxMessageSize=null,
offeredCapabilities=null, desiredCapabilities=null, properties=null}
2016-11-24 16:41:38.991236 [19105288:4] <-
Attach{name='qpid-jms:sender:ID:7fa560e5-9f45-43ab-9215-2b6021fec6d0:3:-1:15:/devices/7C9763001961/messages/events',
handle=0, role=RECEIVER, sndSettleMode=UNSETTLED, rcvSettleMode=FIRST,
source=Source{address='ID:7fa560e5-9f45-43ab-9215-2b6021fec6d0:3:-1:15',
durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false,
dynamicNodeProperties=null, distributionMode=null, filter=null,
defaultOutcome=null, outcomes=[amqp:accepted:list, amqp:rejected:list],
capabilities=null},
target=Target{address='/devices/7C9763001961/messages/events', durable=NONE,
expiryPolicy=SESSION_END, timeout=0, dynamic=false,
dynamicNodeProperties=null, capabilities=[queue]}, unsettled=null,
incompleteUnsettled=false, initialDeliveryCount=null,
maxMessageSize=1048576, offeredCapabilities=null, desiredCapabilities=null,
properties=null}
2016-11-24 16:41:39.150984 [19105288:4] <- Flow{nextIncomingId=16,
incomingWindow=5000, nextOutgoingId=1, outgoingWindow=2047, handle=0,
deliveryCount=0, linkCredit=1000, available=0, drain=false, echo=false,
properties=null}
2016-11-24 16:41:39.182738 [19105288:4] -> Transfer{handle=0,
deliveryId=15, deliveryTag=0, messageFormat=0, settled=null, more=false,
rcvSettleMode=null, state=null, resume=false, aborted=false,
batchable=false} (10823)
"\x00Sp\xc0\x02\x01A\x00Sr\xc1)\x04\xa3\x12x-opt-jms-msg-typeQ\x03\xa3\x0ex-opt-jms-destQ\x00\x00Ss\xc0\x83\x0a\xa10ID:ec766fb9-8779-4bbd-afe2-604752e8b480:1:6:1-16@\xa1%/devices/7C9763001961/messages/events@@@\xa3\x18application/octet-stream@@\x83\x00\x00\x01X\x976\xc2\xe6\x00St\xc1\xac\x10\xa1\x05boxId\xa1\x0c7C9763001961\xa1\x06scopes\xa1\x06SERVER\xa1\x08priority\xa1\x012\xa1\x04name\xa13com.openmatics.system.fw.config.localchange.request\xa1\x0bcomponentId\xa1\x02-1\xa1\x0atimeToLive\xa1\x02-1\xa1\x07version\xa1\x051.0.0\xa1\x06appKey\xa1\x03-14\x00Su\xb0\x00\x00(\xd1\x0b\x01\x1b\x01\x02com.openmatics.system.fw.config.localchange.reques\xf4\x0c\x03\x00$\x04\x01\x0e\x05\x04\x06\x02\x071.0.\xb0&\x01\x03\x03\x02LOCAL_VARIANT_TIMESTAM\xd0\x09\xe4\x8a\xb6\xf3\x92V\x03\x03LOCAL_VARIANT_ID\xd3&\x04\x01\x03\x05/sbx/config/lcr.xm\xec\x03\x067C9763001961_0148000569814\xb1\x03\x07BOX_CONFIGURATIO\xce\x01\x00\xc3\x01com.openmatics.shared.messaging.box.configuration.BoxConfiguration\x08\x00$\x09\x01\x01\x01com.openmat"...(truncated)
2016-11-24 16:41:39.745306 [19105288:4] <- Disposition{role=RECEIVER,
first=15, last=null, settled=true, state=Accepted{}, batchable=false}
2016-11-24 16:41:39.747107 [19105288:4] -> Detach{handle=0, closed=true,
error=null}
2016-11-24 16:41:39.919648 [19105288:4] <- Detach{handle=0, closed=true,
error=null}
2016-11-24 16:41:45.202589 [19105288:0] -> Close{error=null}
*Normal bundle shutdown:*
2016-11-24 16:51:11.517312 [18734493:6] ->
Attach{name='qpid-jms:sender:ID:4c47ee6a-8f04-46bd-9a81-ac1be65b2b5e:15:-1:9:/devices/7C9763001961/messages/events',
handle=0, role=SENDER, sndSettleMode=UNSETTLED, rcvSettleMode=FIRST,
source=Source{address='ID:4c47ee6a-8f04-46bd-9a81-ac1be65b2b5e:15:-1:9',
durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false,
dynamicNodeProperties=null, distributionMode=null, filter=null,
defaultOutcome=null, outcomes=[amqp:accepted:list, amqp:rejected:list],
capabilities=null},
target=Target{address='/devices/7C9763001961/messages/events', durable=NONE,
expiryPolicy=SESSION_END, timeout=0, dynamic=false,
dynamicNodeProperties=null, capabilities=[queue]}, unsettled=null,
incompleteUnsettled=false, initialDeliveryCount=0, maxMessageSize=null,
offeredCapabilities=null, desiredCapabilities=null, properties=null}
2016-11-24 16:51:11.690158 [18734493:6] <-
Attach{name='qpid-jms:sender:ID:4c47ee6a-8f04-46bd-9a81-ac1be65b2b5e:15:-1:9:/devices/7C9763001961/messages/events',
handle=0, role=RECEIVER, sndSettleMode=UNSETTLED, rcvSettleMode=FIRST,
source=Source{address='ID:4c47ee6a-8f04-46bd-9a81-ac1be65b2b5e:15:-1:9',
durable=NONE, expiryPolicy=SESSION_END, timeout=0, dynamic=false,
dynamicNodeProperties=null, distributionMode=null, filter=null,
defaultOutcome=null, outcomes=[amqp:accepted:list, amqp:rejected:list],
capabilities=null},
target=Target{address='/devices/7C9763001961/messages/events', durable=NONE,
expiryPolicy=SESSION_END, timeout=0, dynamic=false,
dynamicNodeProperties=null, capabilities=[queue]}, unsettled=null,
incompleteUnsettled=false, initialDeliveryCount=null,
maxMessageSize=1048576, offeredCapabilities=null, desiredCapabilities=null,
properties=null}
2016-11-24 16:51:11.842212 [18734493:6] <- Flow{nextIncomingId=10,
incomingWindow=5000, nextOutgoingId=1, outgoingWindow=2047, handle=0,
deliveryCount=0, linkCredit=1000, available=0, drain=false, echo=false,
properties=null}
2016-11-24 16:51:11.861722 [18734493:6] -> Transfer{handle=0, deliveryId=9,
deliveryTag=0, messageFormat=0, settled=null, more=false,
rcvSettleMode=null, state=null, resume=false, aborted=false,
batchable=false} (443)
"\x00Sp\xc0\x02\x01A\x00Sr\xc1)\x04\xa3\x12x-opt-jms-msg-typeQ\x03\xa3\x0ex-opt-jms-destQ\x00\x00Ss\xc0\x83\x0a\xa10ID:15ec0e45-968b-4028-a0ac-80cf3e7a62fc:1:8:1-10@\xa1%/devices/7C9763001961/messages/events@@@\xa3\x18application/octet-stream@@\x83\x00\x00\x01X\x97?\x82\x0b\x00St\xc1\x93\x10\xa1\x05boxId\xa1\x0c7C9763001961\xa1\x06scopes\xa1\x06SERVER\xa1\x08priority\xa1\x014\xa1\x04name\xa1\x1bcom.openmatics.system.alive\xa1\x0bcomponentId\xa1\x02-1\xa1\x0atimeToLive\xa1\x02-1\xa1\x07version\xa1\x051.0.0\xa1\x06appKey\xa1\x02-3\x00Su\xa0a\x0b\x01\x05\x01\x02com.openmatics.system.aliv\xe5\x0c\x03\x00$\x04\x01\x0e\x05\x04\x06\x04\x071.0.\xb0&\x01\x03\x03\x02lo\xee\x1a\x03@*\xbc\x82C\x80w#\x03\x04Aliv\xe5\x1c\x05\xb4\xf7\xfb\xf3\x92V\x03\x06la\xf4\x1a\x07@H\xdfd\xf6\xb3\xbf\xcf"
2016-11-24 16:51:12.075635 [18734493:6] <- Disposition{role=RECEIVER,
first=9, last=null, settled=true, state=Accepted{}, batchable=false}
2016-11-24 16:51:12.085039 [18734493:6] -> Detach{handle=0, closed=true,
error=null}
2016-11-24 16:51:12.254863 [18734493:6] <- Detach{handle=0, closed=true,
error=null}
2016-11-24 16:51:31.027968 [18734493:0] -> Close{error=null}
2016-11-24 16:51:31.977722 [18734493:0] <- Close{error=null}
--
View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-JMS-client-JmsConnection-close-hangs-tp7654565p7654821.html
Sent from the Apache Qpid users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: Qpid JMS client JmsConnection.close() hangs
Posted by Robbie Gemmell <ro...@gmail.com>.
I'm not sure it would relate to those bits, and could well just be a
bug. Can you give a bit more detail around e.g. your uri settings etc.
You might also want to turn up the logging and see if anything stands
out, see http://qpid.apache.org/releases/qpid-jms-0.11.1/docs/index.html#logging.
Robbie
On 21 November 2016 at 10:22, DraCzech <dr...@gmail.com> wrote:
> Hi, I'm currently using ActiveMQ 5.14.1 embedded broker connected from
> several local queues to MS Azure IoT Hub using JmsConnectionFactory as
> foreignConnectionFactory in SimpleJmsQueueConnector. (Using dependency to
> qpid jms client 0.11.1) Each queue is connected to the server via its
> respective bridge and everything seems to be working fine until I want to
> shut down the connection.
>
> If I have network connection, everything is stopped in few seconds. However
> if I lose network connection and call SimpleJmsQueueConnector.stop() method,
> it hangs on its foreignConnection.close() (JmsConnection). It remains stuck
> indefinitely and I have to interrupt the thread responsible for closing of
> the connection. Which unfortunately leads to more problems later as there
> are remains of the connection in the system.
>
> *Part of the thread dump:*
> 2016-11-16 09:35:13.275227 - parking to wait for <0x42e6bc08> (a
> java.util.concurrent.CountDownLatch$Sync)
> 2016-11-16 09:35:13.275654 at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> 2016-11-16 09:35:13.276143 at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> 2016-11-16 09:35:13.276631 at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> 2016-11-16 09:35:13.277058 at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> 2016-11-16 09:35:13.277577 at
> java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
> 2016-11-16 09:35:13.278676 at
> org.apache.qpid.jms.provider.ProviderFuture.sync(ProviderFuture.java:97)
> 2016-11-16 09:35:13.279103 at
> org.apache.qpid.jms.JmsConnection.close(JmsConnection.java:187)
> 2016-11-16 09:35:13.279622 - locked <0x487b1cb8> (a
> org.apache.qpid.jms.JmsQueueConnection)
> 2016-11-16 09:35:13.280018 at
> org.apache.activemq.network.jms.JmsConnector.stop(JmsConnector.java:179)
>
> I believe this problem is tied to the JMS specification and is caused by
> MessageConsumer.receive() waiting on some pending receive. I've found some
> description of the same or similar issue online, but no specific solution
> for it. From what I've read I think the problem might be related to:
>
> http://qpid.2158936.n2.nabble.com/Qpid-JMS-0-9-0-receive-hanging-issue-relates-to-Artemis-works-on-ActiveMQ-td7642585.html
> <http://qpid.2158936.n2.nabble.com/Qpid-JMS-0-9-0-receive-hanging-issue-relates-to-Artemis-works-on-ActiveMQ-td7642585.html>
>
> and
>
> https://issues.apache.org/jira/browse/QPIDJMS-221
> <https://issues.apache.org/jira/browse/QPIDJMS-221>
>
> Could you please tell me if it's possible to set up my messaging in such a
> way which prevents this kind of freeze? How should I defend against this
> kind of problems?
> Thanks!
>
>
>
> --
> View this message in context: http://qpid.2158936.n2.nabble.com/Qpid-JMS-client-JmsConnection-close-hangs-tp7654565.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org