OutOfMemoryException - a solution

[jepries <je...@gmail.com>]

HI everyone.  First of all, thanks for the great application, it works for me
very well now that I've figured my problem.  It took me a while, so here's
hoping this nugget finds someone else who might be fighting the same thing
someday.  The problem is specific to me and the size of my messages.It was
very easy for me to get working, I've migrated from two previous message
services and still maintain those connections while also publishing to
activemq.My set up is quite simple.  Non durable topicsjava publisher
application publishes to one or more non durable topicsa different java
subscriber subscribes to one or more non durable topics.Using
DUPS_OK_ACKNOWLEDGE policy when I create my session.Everything worked great
until about 2 hours into it:2016-05-20 13:27:04,757 | WARN  | Transport
Connection to: tcp://xxx.xxx.xxx.xxx:43932 failed: java.io.IOException:
Unexpected error occurred: java.lang.OutOfMemoryError: Java heap space |
org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ
Transport: tcp:///xxxx.xxx.xxxx.xxxx:43932@61616I searched high and low,
followed every example, played with GC settings, opened up the memory (just
made it last longer till it died), searched the ends of the earth and
incorporated all the fixes I could find.Finally I found a hint in a Redhat
forum.  When you use DUPS_OK_ACKNOWLEDGE, your messages aren't ACKED away
until about 50% to 65% of your PREFETCH POLICY are received. (ActiveMQ is
50%). When you use a non durable topic, default PREFETCH POLICY is SHORT -1
(32727)!  This may be great for small number of topics and small message
sizes.  My messages are averaging about 200K each, and when you stack 16K of
them up on multiple topics, ouch! :).I tried putting the jms.prefetchPolicy
into the transport in activemq.xml, but I probably did it wrong as it didn't
change may actual prefetch policy when I looked at the topic from jmx, it
was still default.So, I implemented it in my client and voila.  It works
perfectly now.ActiveMQConnectionFactory topicConnectionFactory 	                         
= new ActiveMQConnectionFactory(				                        		  username,				                        		 
password,				                        		 
"tcp://"+hostname+":"+port);ActiveMQPrefetchPolicy prefetchPolicy = new
ActiveMQPrefetchPolicy();prefetchPolicy.setTopicPrefetch(500);topicConnectionFactory.setPrefetchPolicy(prefetchPolicy);topicConnectionFactory.setTrustAllPackages(true);topicConnection
=
topicConnectionFactory.createTopicConnection();topicConnection.start();session
=
topicConnection.createTopicSession(false,Session.DUPS_OK_ACKNOWLEDGE);Thanks
all, hope that helps.  It was a riddle wrapped in a how do I fix this!



--
View this message in context: http://activemq.2283324.n4.nabble.com/OutOfMemoryException-a-solution-tp4712395.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: OutOfMemoryException - a solution

[jepries <je...@gmail.com>]

OMG, that formatted TERRIBLY!

let me try again:

HI everyone.  First of all, thanks for the great application, it works for
me very well now that I've figured my problem.  It took me a while, so
here's hoping this nugget finds someone else who might be fighting the same
thing someday.  The problem is specific to me and the size of my messages.

It was very easy for me to get working, I've migrated from two previous
message services and still maintain those connections while also publishing
to activemq.

My set up is quite simple.  Non durable topics
java publisher application publishes to one or more non durable topics
a different java subscriber subscribes to one or more non durable topics.
Using DUPS_OK_ACKNOWLEDGE policy when I create my session.

Everything worked great until about 2 hours into it:

2016-05-20 13:27:04,757 | WARN  | Transport Connection to:
tcp://xxx.xxx.xxx.xxx:43932 failed: java.io.IOException: Unexpected error
occurred: java.lang.OutOfMemoryError: Java heap space |
org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ
Transport: tcp:///xxxx.xxx.xxxx.xxxx:43932@61616

I searched high and low, followed every example, played with GC settings,
opened up the memory (just made it last longer till it died), searched the
ends of the earth and incorporated all the fixes I could find.

Finally I found a hint in a Redhat forum.  When you use DUPS_OK_ACKNOWLEDGE,
your messages aren't ACKED away until about 50% to 65% of your PREFETCH
POLICY are received. (ActiveMQ is 50%). 

When you use a non durable topic, default PREFETCH POLICY is SHORT -1
(32727)!  

This may be great for small number of topics and small message sizes.  My
messages are averaging about 200K each, and when you stack 16K of them up on
multiple topics, ouch! :).

I tried putting the jms.prefetchPolicy into the transport in activemq.xml,
but I probably did it wrong as it didn't change may actual prefetch policy
when I looked at the topic from jmx, it was still default.

So, I implemented it in my client and voila.  It works perfectly now.

ActiveMQConnectionFactory topicConnectionFactory 
	                          = new ActiveMQConnectionFactory(
				                        		  username,
				                        		  password,
				                        		  "tcp://"+hostname+":"+port);
ActiveMQPrefetchPolicy prefetchPolicy = new ActiveMQPrefetchPolicy();
prefetchPolicy.setTopicPrefetch(500);
topicConnectionFactory.setPrefetchPolicy(prefetchPolicy);
topicConnectionFactory.setTrustAllPackages(true);
topicConnection = topicConnectionFactory.createTopicConnection();
topicConnection.start();
session =
topicConnection.createTopicSession(false,Session.DUPS_OK_ACKNOWLEDGE);



--
View this message in context: http://activemq.2283324.n4.nabble.com/OutOfMemoryException-a-solution-tp4712395p4712396.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.