Out of memory with 14K large messages

["Eichberger, German" <ge...@soe.sony.com>]

Hi,

 

I am new to activeMQ and I have configured a network of brokers (two
servers) with two slow consumers and a durable queue. We are
approximately adding 13 messages/second and taking about one out per
second. 

 

To deal with that asymmetry I have also added some of the performance
tips on the producer: 

connection.setCopyMessageOnSend(false);

            connection.setOptimizeAcknowledge(true);

            connection.setUseAsyncSend(true);

 

 

We are sending 36,173 byte long messages to the system and after about
14,000 we get a java.lang.OutOfMemoryError - activeMQ is running in a
tomcat container which has only 512MB heap space.

 

I was wondering if there is any way (besides increasing the heap space)
to make activeMQ recover when it runs out of memory or if there is a
setting to enable it to run on a small memory footprint.

 

The stack trace looks like:

 

at
org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.
java:46)

        at
org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection
.java:1191)

        at
org.apache.activemq.AdvisoryConsumer.<init>(AdvisoryConsumer.java:46)

        at
org.apache.activemq.ActiveMQConnection.ensureConnectionInfoSent(ActiveMQ
Connection.java:1281)

        at
org.apache.activemq.ActiveMQConnection.start(ActiveMQConnection.java:449
)

 

 

 

 

and the configuration file:

 

 

<!--

    Licensed to the Apache Software Foundation (ASF) under one or more

    contributor license agreements.  See the NOTICE file distributed
with

    this work for additional information regarding copyright ownership.

    The ASF licenses this file to You under the Apache License, Version
2.0

    (the "License"); you may not use this file except in compliance with

    the License.  You may obtain a copy of the License at

   

    http://www.apache.org/licenses/LICENSE-2.0

   

    Unless required by applicable law or agreed to in writing, software

    distributed under the License is distributed on an "AS IS" BASIS,

    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied.

    See the License for the specific language governing permissions and

    limitations under the License.

-->

<!-- START SNIPPET: example -->

<beans xmlns="http://activemq.org/config/1.0">

 

  <!-- Allows us to use system properties as variables in this
configuration file -->

  <bean
class="org.springframework.beans.factory.config.PropertyPlaceholderConfi
gurer"/>

  

  <broker useJmx="true">

  

    <!--  Use the following to set the broker memory limit (in bytes)
(test only has 512 MB)-->

    <memoryManager>  

        <usageManager id="memory-manager" limit="400000000"/>

    </memoryManager>

   

    

    <!-- Use the following to configure how ActiveMQ is exposed in JMX

    <managementContext>

       <managementContext connectorPort="1099"
jmxDomainName="org.apache.activemq"/>

    </managementContext>

    -->

 

    <!-- In ActiveMQ 4, you can setup destination policies -->  

    <!--<destinationPolicy>

      <policyMap><policyEntries>

        

          <policyEntry topic="FOO.>">

            <dispatchPolicy>

              <strictOrderDispatchPolicy />

            </dispatchPolicy>

            <subscriptionRecoveryPolicy>

              <lastImageSubscriptionRecoveryPolicy />

            </subscriptionRecoveryPolicy>

          </policyEntry>

 

      </policyEntries></policyMap>

    </destinationPolicy>-->

  

  

    <persistenceAdapter>

                        <!-- UNIX OPS was kind enough to create links
for each broker to the filer-->

        <!--<journaledJDBC journalLogFiles="5"
dataDirectory="/m2/tomcat5.5-temp/playersqueue/4209/q"/>-->

                        <!--<kahaPersistenceAdapter dir =
"/m2/tomcat5.5-temp/playersqueue/4209/q" maxDataFileLength =
"33554432"/>-->

                        <jdbcPersistenceAdapter
dataDirectory="/m2/tomcat5.5-temp/playersqueue/4209/q"/>

      <!-- To use a different datasource, use th following syntax : -->

      <!-- 

      <journaledJDBC journalLogFiles="5"
dataDirectory="../activemq-data" dataSource="#postgres-ds"/>

       -->

    </persistenceAdapter>

  

    <transportConnectors>

       <transportConnector name="default" uri="tcp://localhost:61616"/>

               <transportConnector name="local" uri="vm://localhost"/>

    </transportConnectors>

    

    <networkConnectors>

      <!-- by default just auto discover the other brokers -->

      <!--<networkConnector name="default"
uri="multicast://default"/>-->

      <networkConnector name="bridge1"
uri="static://(tcp://sdt-plattestt1.station.sony.com:61616)"
failover="true"/>

              <networkConnector name="bridge2"
uri="static://(tcp://sdt-plattestt2.station.sony.com:61616)"
failover="true"/>

    </networkConnectors>

    

  </broker>

  

  <!--  This xbean configuration file supports all the standard spring
xml configuration options -->

  

  <!-- Postgres DataSource Sample Setup -->

  <!-- 

  <bean id="postgres-ds" class="org.postgresql.ds.PGPoolingDataSource">

    <property name="serverName" value="localhost"/>

    <property name="databaseName" value="activemq"/>

    <property name="portNumber" value="0"/>

    <property name="user" value="activemq"/>

    <property name="password" value="activemq"/>

    <property name="dataSourceName" value="postgres"/>

    <property name="initialConnections" value="1"/>

    <property name="maxConnections" value="10"/>

  </bean>

  -->

  

  <!-- MySql DataSource Sample Setup -->

  <!-- 

  <bean id="mysql-ds" class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close">

    <property name="driverClassName" value="com.mysql.jdbc.Driver"/>

    <property name="url"
value="jdbc:mysql://localhost/activemq?relaxAutoCommit=true"/>

    <property name="username" value="activemq"/>

    <property name="password" value="activemq"/>

    <property name="poolPreparedStatements" value="true"/>

  </bean>

  -->  

   

  <!-- Embedded Derby DataSource Sample Setup -->

  <!-- 

  <bean id="derby-ds" class="org.apache.derby.jdbc.EmbeddedDataSource">

    <property name="databaseName" value="derbydb"/>

    <property name="createDatabase" value="create"/>

  </bean>

  -->  

 

</beans>

<!-- END SNIPPET: example -->

 

 

 

I know we are exploring some kind of edge case but we would like to
buffer a huge amount of messages during peak hours and catch up with our
consumers during non-peak...

 

Thanks,

German

 

---

German Eichberger - geichberger@soe.sony.com