Consumer is hung and Heap Memory limit is reahcing Maximum

[superman <sa...@gmail.com>]

It would be good if Apache ActiveMQ developer/user can share their
experience/resolution which I am facing in Production environment now.

We are using 4 network of brokers in the provider end. They are all using
ActiveMQ 5.5.1 (along with Fuse 7.0.2). All these brokers are running in
Ubuntu OS.

There are 6 Glassfish 3.1.1 app server instances are running under CentOS.
These consumers are using Spring-jms producer template along with
ActiveMQPooledConnectionFactory. Camel DMLC is used for request-reply
paradigm while connecting to Active MQ and hence all the Request Queues are
Permanent and Reply Queues are temporary (i.e. created on the fly). This
implementation is configured via camel route.

Once in a week one of the consumer (6 GF App server nodes) is reaching
maximum heap memory allocated(around 34 GB) and blocked by threads. On the
whole the server node is not responsive for any request, but the server is
keep-alive. The only resolution is to restart the server node for freeing up
the resources such as Memory and Threads.


Following are the version of each tool used in our PROD:

Fuse Enterprise (7.0.2) distribution includes
Apache ActiveMQ 5.5.1
Camel 2.9.0
karaf 2.2.5
spring 3.0.7
cxf 2.5.0 





--
View this message in context: http://activemq.2283324.n4.nabble.com/Consumer-is-hung-and-Heap-Memory-limit-is-reahcing-Maximum-tp4684926.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Consumer is hung and Heap Memory limit is reaching Maximum

[superman <sa...@gmail.com>]

Thanks much your reply!!
We also thought the same, but quiet interestingly there is no
OutOfMemoryError in any of the Glassfish application server(which are all
typical consumers) logs and there is enough memory (-Xmx is set to 34 GB)
for each of the node.

However, I will try to analyze the dump and post the findings.

 



--
View this message in context: http://activemq.2283324.n4.nabble.com/Consumer-is-hung-and-Heap-Memory-limit-is-reaching-Maximum-tp4684926p4684989.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Consumer is hung and Heap Memory limit is reaching Maximum

[artnaseef <ar...@artnaseef.com>]

That sounds like the typical Java memory leak.  If so, it should be easy to
track down since it sounds easily reproduced.

I recommend grabbing a couple of heap dumps a few hours, or days, apart, and
comparing.

JVisualVM can help analyze the heap dump.  YourKit is even better.



--
View this message in context: http://activemq.2283324.n4.nabble.com/Consumer-is-hung-and-Heap-Memory-limit-is-reaching-Maximum-tp4684926p4684955.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.