You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by IBeaumont <IB...@categoric.com> on 2009/01/08 16:51:38 UTC
Client threads hang if ActiveMQ dies
I'm running ActiveMQ on another machine and connecting via TCP. If it dies
(I press CTRL-C, or may get an out of memory error) while I'm processing
messages then the consume gets stuck waiting for a message and never returns
(I'm calling receiveNoWait). This is the stack trace where it is stuck in
jconsole...
Name: WorkerThread for PIQ Server manager 39
State: WAITING on java.lang.Object@47eb38
Total blocked: 0 Total waited: 43
Stack trace:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:485)
org.apache.activemq.MessageDispatchChannel.dequeue(MessageDispatchChannel.java:75)
org.apache.activemq.ActiveMQMessageConsumer.dequeue(ActiveMQMessageConsumer.java:412)
org.apache.activemq.ActiveMQMessageConsumer.receiveNoWait(ActiveMQMessageConsumer.java:560)
….
At the same time I may see another ActiveMQ thread waiting (and not sure if
this is related or the cause)...
Name: ActiveMQ Connection Worker: tcp://2000-serve-base/192.168.1.9:61616
State: WAITING on
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16372aa
Total blocked: 0 Total waited: 3
Stack trace:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:946)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:906)
java.lang.Thread.run(Thread.java:619)
This causes me 2 problems -
1. The client application can't cleanly shutdown as it is waiting for my
consumer thread to finish.
2. If ActiveMQ restarts my client doesn't automatically reconnect becasue
the consumer thread is still stuck.
Any ideas?
Ian
This means that when active MQ is restarted my client cannot automatically
--
View this message in context: http://www.nabble.com/Client-threads-hang-if-ActiveMQ-dies-tp21354487p21354487.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Client threads hang if ActiveMQ dies
Posted by IBeaumont <IB...@categoric.com>.
Thanks for the suggestion. I tried it and it made no difference. I also
found that the memory usage on the client goes up a lot when using
transactions with many messages (I guess because it has to store the whole
transaction on the client in-case it needs to failover which potentially
causes me another problem).
I found this setting though "soTimeout" - which can be set for the socket
timeout and defaults to 0. So I changed this to 10000 (10 secs) and
everything now behaves a lot better.
bsnyder wrote:
>
> On Thu, Jan 8, 2009 at 8:51 AM, IBeaumont <IB...@categoric.com> wrote:
>>
>> I'm running ActiveMQ on another machine and connecting via TCP. If it
>> dies
>> (I press CTRL-C, or may get an out of memory error) while I'm processing
>> messages then the consume gets stuck waiting for a message and never
>> returns
>> (I'm calling receiveNoWait). This is the stack trace where it is stuck
>> in
>> jconsole...
>>
>> Name: WorkerThread for PIQ Server manager 39
>> State: WAITING on java.lang.Object@47eb38
>> Total blocked: 0 Total waited: 43
>>
>> Stack trace:
>> java.lang.Object.wait(Native Method)
>> java.lang.Object.wait(Object.java:485)
>> org.apache.activemq.MessageDispatchChannel.dequeue(MessageDispatchChannel.java:75)
>> org.apache.activemq.ActiveMQMessageConsumer.dequeue(ActiveMQMessageConsumer.java:412)
>> org.apache.activemq.ActiveMQMessageConsumer.receiveNoWait(ActiveMQMessageConsumer.java:560)
>> ….
>>
>>
>> At the same time I may see another ActiveMQ thread waiting (and not sure
>> if
>> this is related or the cause)...
>>
>> Name: ActiveMQ Connection Worker: tcp://2000-serve-base/192.168.1.9:61616
>> State: WAITING on
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16372aa
>> Total blocked: 0 Total waited: 3
>>
>> Stack trace:
>> sun.misc.Unsafe.park(Native Method)
>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
>> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:946)
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:906)
>> java.lang.Thread.run(Thread.java:619)
>>
>>
>> This causes me 2 problems -
>> 1. The client application can't cleanly shutdown as it is waiting for my
>> consumer thread to finish.
>> 2. If ActiveMQ restarts my client doesn't automatically reconnect becasue
>> the consumer thread is still stuck.
>>
>> Any ideas?
>>
>> Ian
>> This means that when active MQ is restarted my client cannot
>> automatically
>
> Are your clients using the failover transport to connect to the broker?
>
> http://activemq.apache.org/failover-transport-reference.html
>
> The failover transport provides automatic reconnection for clients
> when a broker cycles.
>
> Bruce
> --
> perl -e 'print
> unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
> );'
>
> Apache ActiveMQ - http://activemq.org/
> Apache Camel - http://activemq.org/camel/
> Apache ServiceMix - http://servicemix.org/
>
> Blog: http://bruceblog.org/
>
>
--
View this message in context: http://www.nabble.com/Client-threads-hang-if-ActiveMQ-dies-tp21354487p21357557.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Client threads hang if ActiveMQ dies
Posted by Bruce Snyder <br...@gmail.com>.
On Thu, Jan 8, 2009 at 8:51 AM, IBeaumont <IB...@categoric.com> wrote:
>
> I'm running ActiveMQ on another machine and connecting via TCP. If it dies
> (I press CTRL-C, or may get an out of memory error) while I'm processing
> messages then the consume gets stuck waiting for a message and never returns
> (I'm calling receiveNoWait). This is the stack trace where it is stuck in
> jconsole...
>
> Name: WorkerThread for PIQ Server manager 39
> State: WAITING on java.lang.Object@47eb38
> Total blocked: 0 Total waited: 43
>
> Stack trace:
> java.lang.Object.wait(Native Method)
> java.lang.Object.wait(Object.java:485)
> org.apache.activemq.MessageDispatchChannel.dequeue(MessageDispatchChannel.java:75)
> org.apache.activemq.ActiveMQMessageConsumer.dequeue(ActiveMQMessageConsumer.java:412)
> org.apache.activemq.ActiveMQMessageConsumer.receiveNoWait(ActiveMQMessageConsumer.java:560)
> ….
>
>
> At the same time I may see another ActiveMQ thread waiting (and not sure if
> this is related or the cause)...
>
> Name: ActiveMQ Connection Worker: tcp://2000-serve-base/192.168.1.9:61616
> State: WAITING on
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@16372aa
> Total blocked: 0 Total waited: 3
>
> Stack trace:
> sun.misc.Unsafe.park(Native Method)
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:946)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:906)
> java.lang.Thread.run(Thread.java:619)
>
>
> This causes me 2 problems -
> 1. The client application can't cleanly shutdown as it is waiting for my
> consumer thread to finish.
> 2. If ActiveMQ restarts my client doesn't automatically reconnect becasue
> the consumer thread is still stuck.
>
> Any ideas?
>
> Ian
> This means that when active MQ is restarted my client cannot automatically
Are your clients using the failover transport to connect to the broker?
http://activemq.apache.org/failover-transport-reference.html
The failover transport provides automatic reconnection for clients
when a broker cycles.
Bruce
--
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'
Apache ActiveMQ - http://activemq.org/
Apache Camel - http://activemq.org/camel/
Apache ServiceMix - http://servicemix.org/
Blog: http://bruceblog.org/