activeMQ seems stalled

[David Delbecq <de...@oma.be>]

Hello,

we are using an activeMQ server here, as a standalone application and
used by tomcat, which connects to server using tcp:// protocol.
Sometimes, the tomcat client is stalled trying to connect to a queue,
and i have no idea why. Here is a stack dump from tomcat jvm when this
happen

   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <3f543bf8> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1889)
        at
java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
        at
org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
        at
org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80)
        at
org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233)
        at
org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1659)
        - locked <3f5425b8> (a java.lang.Object)
        at
org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:227)
        at
org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:241)
        at be.rmi.shark.JMSEventManager.sendEvent(JMSEventManager.java:146)

It seems that client is waiting forever to receive and answer from MQ
server. I have several threads in this situation and the only way to
unlock de situation is to kill and restart the activeMQ server. However,
the server itself does not loag any error. It looks like the srver
"forgot" to send response and client is waiting forever. Any idea what
can cause this? In current situation, we have noticed this happens when
another client is making bad connection (it connect to activeMQ queues,
receive an item but neither close connection, nor acknowledge or reject
the message). Could it be that activeMQ is stalled waiting for
acknoledge befor allowing other client to add items to queue. Should
those 2 operations be desynchronzed?

-- 
David Delbecq
ICT
Institut Royal Météorologique
Ext:557

Re: activeMQ seems stalled

[Gary Tully <ga...@gmail.com>]

sounds like http://activemq.apache.org/producer-flow-control.html - increase
your q usage limits or decrease the system usage to ensure pending message
references are paged to disk. see also :
http://activemq.apache.org/message-cursors.html

2009/12/11 David Delbecq <de...@oma.be>

> Hello,
>
> we are using an activeMQ server here, as a standalone application and
> used by tomcat, which connects to server using tcp:// protocol.
> Sometimes, the tomcat client is stalled trying to connect to a queue,
> and i have no idea why. Here is a stack dump from tomcat jvm when this
> happen
>
>   java.lang.Thread.State: WAITING (parking)
>        at sun.misc.Unsafe.park(Native Method)
>        - parking to wait for  <3f543bf8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>        at
>
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1889)
>        at
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
>        at
>
> org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
>        at
>
> org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:80)
>        at
>
> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233)
>        at
> org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1659)
>        - locked <3f5425b8> (a java.lang.Object)
>        at
>
> org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:227)
>        at
>
> org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:241)
>        at be.rmi.shark.JMSEventManager.sendEvent(JMSEventManager.java:146)
>
> It seems that client is waiting forever to receive and answer from MQ
> server. I have several threads in this situation and the only way to
> unlock de situation is to kill and restart the activeMQ server. However,
> the server itself does not loag any error. It looks like the srver
> "forgot" to send response and client is waiting forever. Any idea what
> can cause this? In current situation, we have noticed this happens when
> another client is making bad connection (it connect to activeMQ queues,
> receive an item but neither close connection, nor acknowledge or reject
> the message). Could it be that activeMQ is stalled waiting for
> acknoledge befor allowing other client to add items to queue. Should
> those 2 operations be desynchronzed?
>
> --
> David Delbecq
> ICT
> Institut Royal Météorologique
> Ext:557
>
>
>


-- 
http://blog.garytully.com

Open Source Integration
http://fusesource.com