You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by "Justin C. van Vorst" <pr...@gmail.com> on 2007/04/23 18:01:37 UTC

OutOfMemoryError using 4.1.1

On our production server this weekend, activemq-4.1.1 hit a
java.lang.OutOfMemoryError exception and then proceeded to stop functioning. 
The process was using about 3GB of RAM.

I've seen some other posts about OOME, but none that quite match the stack
of ours.  Does anyone have a place for me to start investigating this issue?

Below are the error logs, command line start-up, and activemq.xml config:


Error log:

2007-04-23 02:43:10,286 [localhost:61613] WARN  ManagedTransportConnection    
- Failed to register MBean:
org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_54813
2007-04-23 02:49:10,415 [localhost:61613] WARN  ManagedTransportConnection    
- Failed to register MBean:
org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_58365
2007-04-23 03:11:10,301 [localhost:61613] WARN  ManagedTransportConnection    
- Failed to register MBean:
org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_42109
2007-04-23 03:15:10,056 [localhost:61613] WARN  ManagedTransportConnection    
- Failed to register MBean:
org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_44595
2007-04-23 03:27:10,120 [localhost:61613] WARN  ManagedTransportConnection    
- Failed to register MBean:
org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_51338
2007-04-23 03:44:31,858 [eckpoint Worker] ERROR JournalPersistenceAdapter     
- Failed to checkpoint a message store:
edu.emory.mathcs.backport.java.util.concurrent.ExecutionException:
java.lang.OutOfMemoryError: unable to create new native thread
edu.emory.mathcs.backport.java.util.concurrent.ExecutionException:
java.lang.OutOfMemoryError: unable to create new native thread
        at
edu.emory.mathcs.backport.java.util.concurrent.FutureTask.getResult(FutureTask.java:299)
        at
edu.emory.mathcs.backport.java.util.concurrent.FutureTask.get(FutureTask.java:118)
        at
org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:386)
        at
org.apache.activemq.store.journal.JournalPersistenceAdapter$2.iterate(JournalPersistenceAdapter.java:129)
        at
org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:101)
        at
org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:25)
        at
org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:39)
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:574)
        at java.util.Timer.<init>(Timer.java:154)
        at java.util.Timer.<init>(Timer.java:122)
        at com.mysql.jdbc.Connection.<init>(Connection.java:1441)
        at
com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:266)
        at
org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:37)
        at
org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:290)
        at
org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:771)
        at
org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:95)
        at
org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:544)
        at
org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:55)
        at
org.apache.activemq.store.jdbc.TransactionContext.begin(TransactionContext.java:147)
        at
org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.beginTransaction(JDBCPersistenceAdapter.java:358)
        at
org.apache.activemq.store.journal.JournalPersistenceAdapter.beginTransaction(JournalPersistenceAdapter.java:189)
        at
org.apache.activemq.util.TransactionTemplate.run(TransactionTemplate.java:41)
        at
org.apache.activemq.store.journal.JournalMessageStore.checkpoint(JournalMessageStore.java:247)
        at
org.apache.activemq.store.journal.JournalMessageStore.checkpoint(JournalMessageStore.java:221)
        at
org.apache.activemq.store.journal.JournalPersistenceAdapter$4.call(JournalPersistenceAdapter.java:356)
        at
edu.emory.mathcs.backport.java.util.concurrent.FutureTask.run(FutureTask.java:176)
        at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
        at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
        at java.lang.Thread.run(Thread.java:595)
2007-04-23 03:44:31,858 [eckpoint Worker] ERROR JournalPersistenceAdapter     
- Failed to mark the Journal:
org.apache.activeio.journal.InvalidRecordLocationException: The location is
less than the last mark.
org.apache.activeio.journal.InvalidRecordLocationException: The location is
less than the last mark.
        at
org.apache.activeio.journal.active.JournalImpl.setMark(JournalImpl.java:340)
        at
org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:403)
        at
org.apache.activemq.store.journal.JournalPersistenceAdapter$2.iterate(JournalPersistenceAdapter.java:129)
        at
org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:101)
        at
org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:25)
        at
org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:39)
2007-04-23 03:47:10,105 [localhost:61613] WARN  ManagedTransportConnection    
- Failed to register MBean:
org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_34348
2007-04-23 03:50:01,821 [eckpoint Worker] ERROR JournalPersistenceAdapter     
- Failed to checkpoint a message store:
edu.emory.mathcs.backport.java.util.concurrent.ExecutionException:
java.io.IOException: Already started.
edu.emory.mathcs.backport.java.util.concurrent.ExecutionException:
java.io.IOException: Already started.
        at
edu.emory.mathcs.backport.java.util.concurrent.FutureTask.getResult(FutureTask.java:299)
        at
edu.emory.mathcs.backport.java.util.concurrent.FutureTask.get(FutureTask.java:118)
        at
org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:386)
        at
org.apache.activemq.store.journal.JournalPersistenceAdapter$2.iterate(JournalPersistenceAdapter.java:129)
        at
org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:101)
        at
org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:25)
        at
org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:39)
Caused by: java.io.IOException: Already started.
        at
org.apache.activemq.store.jdbc.TransactionContext.begin(TransactionContext.java:145)
        at
org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.beginTransaction(JDBCPersistenceAdapter.java:358)
        at
org.apache.activemq.store.journal.JournalPersistenceAdapter.beginTransaction(JournalPersistenceAdapter.java:189)
        at
org.apache.activemq.util.TransactionTemplate.run(TransactionTemplate.java:41)
        at
org.apache.activemq.store.journal.JournalMessageStore.checkpoint(JournalMessageStore.java:247)
        at
org.apache.activemq.store.journal.JournalMessageStore.checkpoint(JournalMessageStore.java:221)
        at
org.apache.activemq.store.journal.JournalPersistenceAdapter$4.call(JournalPersistenceAdapter.java:356)
        at
edu.emory.mathcs.backport.java.util.concurrent.FutureTask.run(FutureTask.java:176)
        at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
        at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
        at java.lang.Thread.run(Thread.java:595)
2007-04-23 03:50:01,821 [eckpoint Worker] ERROR JournalPersistenceAdapter     
- Failed to mark the Journal:
org.apache.activeio.journal.InvalidRecordLocationException: The location is
less than the last mark.
org.apache.activeio.journal.InvalidRecordLocationException: The location is
less than the last mark.
        at
org.apache.activeio.journal.active.JournalImpl.setMark(JournalImpl.java:340)
        at
org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:403)
        at
org.apache.activemq.store.journal.JournalPersistenceAdapter$2.iterate(JournalPersistenceAdapter.java:129)
        at
org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:101)
        at
org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:25)
        at
org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:39)
2007-04-23 03:51:10,188 [localhost:61613] WARN  ManagedTransportConnection    
- Failed to register MBean:
org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_36436
2007-04-23 04:13:10,134 [localhost:61613] WARN  ManagedTransportConnection    
- Failed to register MBean:
org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_49022
2007-04-23 04:17:10,094 [localhost:61613] WARN  ManagedTransportConnection    
- Failed to register MBean:
org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_51068



Command line:

usr/java/current/bin/java -Xmx512M
-Dorg.apache.activemq.UseDedicatedTaskRunner=true
-Dderby.system.home=/opt/activemq-4.1.1/data
-Dderby.storage.fileSyncTransactionLog=true -Dcom.sun.management.jmxremote
-Djavax.net.ssl.keyStorePassword=password
-Djavax.net.ssl.trustStorePassword=password
-Djavax.net.ssl.keyStore=/opt/activemq-4.1.1/conf/broker.ks
-Djavax.net.ssl.trustStore=/opt/activemq-4.1.1/conf/broker.ts -classpath 
-Dactivemq.home=/opt/rhg/activemq-4.1.1 -Dactivemq.base=/opt/activemq-4.1.1
-jar /opt/activemq-4.1.1/bin/run.jar



activemq.xml:

<beans>

  <bean
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"/>

  <broker brokerName="localhost" useJmx="true"
xmlns="http://activemq.org/config/1.0">

    <destinationPolicy>
      <policyMap><policyEntries>
          <policyEntry topic="FOO.>">
            <dispatchPolicy>
              <strictOrderDispatchPolicy />
            </dispatchPolicy>
            <subscriptionRecoveryPolicy>
              <lastImageSubscriptionRecoveryPolicy />
            </subscriptionRecoveryPolicy>
          </policyEntry>
      </policyEntries></policyMap>
    </destinationPolicy>

    <persistenceAdapter>
      <journaledJDBC journalLogFiles="5" dataDirectory="../activemq-data"
dataSource="#mysql-ds"/>
    </persistenceAdapter>

    <transportConnectors>
       <transportConnector name="openwire" uri="tcp://localhost:61616"
discoveryUri="multicast://default"/>
       <transportConnector name="ssl"     uri="ssl://localhost:61617"/>
       <transportConnector name="stomp"   uri="stomp://localhost:61613"/>
    </transportConnectors>

    <networkConnectors>
      <networkConnector name="default-nc" uri="multicast://default"/>
    </networkConnectors>

  </broker>

  <bean id="mysql-ds" class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close">
    <property name="driverClassName" value="com.mysql.jdbc.Driver"/>
    <property name="url"
value="jdbc:mysql://x.x.x.x/activemq?relaxAutoCommit=true"/>
    <property name="username" value="username"/>
    <property name="password" value="password"/>
    <property name="poolPreparedStatements" value="true"/>
  </bean>

</beans>
-- 
View this message in context: http://www.nabble.com/OutOfMemoryError-using-4.1.1-tf3632613s2354.html#a10143322
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Re: OutOfMemoryError using 4.1.1

Posted by "Christopher G. Stach II" <cg...@ldsys.net>.
Justin C. van Vorst wrote:
> Please don't hijack my thread.  Now I'm not sure whether Christopher's
> comment in any way applies to my problem.

If you run out of threads, it could be for a few reasons.  You could
have lingering threads in the JVM somewhere (use jstack, kill -QUIT, or
ctrl-\ to get a thread dump), you could hit the process' thread limit as
James said, the entire OS could be out of processes, or for other less
obvious reasons like heap fragmentation or thread stack size.

-- 
Christopher G. Stach II


Re: OutOfMemoryError using 4.1.1

Posted by "Justin C. van Vorst" <pr...@gmail.com>.
Please don't hijack my thread.  Now I'm not sure whether Christopher's
comment in any way applies to my problem.



avin98 wrote:
> 
> I have hit an OOMError without using persistent messages, and even if AMQ
> is inactive for a period of about a day or so.
> 
> [4/22/07 11:17:44:727 PDT]  [WARN] - [GeronimoConnectionEventListener] -
> connectionErrorOccurred called with null
> javax.jms.JMSException: Channel was inactive for too long.
> 	at
> org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:46)
> 	at
> org.apache.activemq.ActiveMQConnection.onAsyncException(ActiveMQConnection.java:1519)
> 	at
> org.apache.activemq.ActiveMQConnection.onException(ActiveMQConnection.java:1535)
> 	at
> org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:96)
> 	at
> org.apache.activemq.transport.ResponseCorrelator.onException(ResponseCorrelator.java:114)
> 	at
> org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:96)
> 	at
> org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:96)
> 	at
> org.apache.activemq.transport.WireFormatNegotiator.onException(WireFormatNegotiator.java:147)
> 	at
> org.apache.activemq.transport.InactivityMonitor.onException(InactivityMonitor.java:150)
> 	at
> org.apache.activemq.transport.InactivityMonitor.readCheck(InactivityMonitor.java:101)
> 	at
> org.apache.activemq.transport.InactivityMonitor.access$000(InactivityMonitor.java:35)
> 	at
> org.apache.activemq.transport.InactivityMonitor$1.run(InactivityMonitor.java:51)
> 	at
> edu.emory.mathcs.backport.java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:442)
> 	at
> edu.emory.mathcs.backport.java.util.concurrent.FutureTask.runAndReset(FutureTask.java:198)
> 	at
> edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:102)
> 	at
> edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:189)
> 	at
> edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:213)
> 	at
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:665)
> 	at
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:690)
> 	at java.lang.Thread.run(Thread.java:799)
> Caused by: 
> org.apache.activemq.transport.InactivityIOException: Channel was inactive
> for too long.
> 	... 11 more
> JVMDUMP006I Processing Dump Event "uncaught", detail
> "java/lang/OutOfMemoryError" - Please Wait.
> JVMDUMP007I JVM Requesting Snap Dump using
> '/opt/WebSphere/AppServerCommunityEdition/bin/Snap0001.20070422.112429.8876.trc'
> JVMDUMP010I Snap Dump written to
> /opt/WebSphere/AppServerCommunityEdition/bin/Snap0001.20070422.112429.8876.trc
> JVMDUMP007I JVM Requesting Heap Dump using
> '/opt/WebSphere/AppServerCommunityEdition/bin/heapdump.20070422.112429.8876.phd'
> JVMDUMP010I Heap Dump written to
> /opt/WebSphere/AppServerCommunityEdition/bin/heapdump.20070422.112429.8876.phd
> JVMDUMP007I JVM Requesting Java Dump using
> '/opt/WebSphere/AppServerCommunityEdition/bin/javacore.20070422.112429.8876.txt'
> UTE503: utsGetTracePointIterator cannot allocate iterator's buffer
> JVMDUMP010I Java Dump written to
> /opt/WebSphere/AppServerCommunityEdition/bin/javacore.20070422.112429.8876.txt
> 
> 

-- 
View this message in context: http://www.nabble.com/OutOfMemoryError-using-4.1.1-tf3632613s2354.html#a10165073
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Re: OutOfMemoryError using 4.1.1

Posted by "Christopher G. Stach II" <cg...@ldsys.net>.
avin98 wrote:
> I have hit an OOMError without using persistent messages, and even if AMQ is
> inactive for a period of about a day or so.

Did you take a baseline dump after the app reaches a steady state, turn
on heap dumps on memory errors, and then compare the two dumps and see
where the memory is going?  If not, nobody is really going to be able to
help you.

-- 
Christopher G. Stach II


Re: OutOfMemoryError using 4.1.1

Posted by avin98 <av...@yahoo.com>.
I have hit an OOMError without using persistent messages, and even if AMQ is
inactive for a period of about a day or so.

[4/22/07 11:17:44:727 PDT]  [WARN] - [GeronimoConnectionEventListener] -
connectionErrorOccurred called with null
javax.jms.JMSException: Channel was inactive for too long.
	at
org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:46)
	at
org.apache.activemq.ActiveMQConnection.onAsyncException(ActiveMQConnection.java:1519)
	at
org.apache.activemq.ActiveMQConnection.onException(ActiveMQConnection.java:1535)
	at
org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:96)
	at
org.apache.activemq.transport.ResponseCorrelator.onException(ResponseCorrelator.java:114)
	at
org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:96)
	at
org.apache.activemq.transport.TransportFilter.onException(TransportFilter.java:96)
	at
org.apache.activemq.transport.WireFormatNegotiator.onException(WireFormatNegotiator.java:147)
	at
org.apache.activemq.transport.InactivityMonitor.onException(InactivityMonitor.java:150)
	at
org.apache.activemq.transport.InactivityMonitor.readCheck(InactivityMonitor.java:101)
	at
org.apache.activemq.transport.InactivityMonitor.access$000(InactivityMonitor.java:35)
	at
org.apache.activemq.transport.InactivityMonitor$1.run(InactivityMonitor.java:51)
	at
edu.emory.mathcs.backport.java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:442)
	at
edu.emory.mathcs.backport.java.util.concurrent.FutureTask.runAndReset(FutureTask.java:198)
	at
edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:102)
	at
edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:189)
	at
edu.emory.mathcs.backport.java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:213)
	at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:665)
	at
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:690)
	at java.lang.Thread.run(Thread.java:799)
Caused by: 
org.apache.activemq.transport.InactivityIOException: Channel was inactive
for too long.
	... 11 more
JVMDUMP006I Processing Dump Event "uncaught", detail
"java/lang/OutOfMemoryError" - Please Wait.
JVMDUMP007I JVM Requesting Snap Dump using
'/opt/WebSphere/AppServerCommunityEdition/bin/Snap0001.20070422.112429.8876.trc'
JVMDUMP010I Snap Dump written to
/opt/WebSphere/AppServerCommunityEdition/bin/Snap0001.20070422.112429.8876.trc
JVMDUMP007I JVM Requesting Heap Dump using
'/opt/WebSphere/AppServerCommunityEdition/bin/heapdump.20070422.112429.8876.phd'
JVMDUMP010I Heap Dump written to
/opt/WebSphere/AppServerCommunityEdition/bin/heapdump.20070422.112429.8876.phd
JVMDUMP007I JVM Requesting Java Dump using
'/opt/WebSphere/AppServerCommunityEdition/bin/javacore.20070422.112429.8876.txt'
UTE503: utsGetTracePointIterator cannot allocate iterator's buffer
JVMDUMP010I Java Dump written to
/opt/WebSphere/AppServerCommunityEdition/bin/javacore.20070422.112429.8876.txt

-- 
View this message in context: http://www.nabble.com/OutOfMemoryError-using-4.1.1-tf3632613s2354.html#a10149412
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Re: OutOfMemoryError using 4.1.1

Posted by James Strachan <ja...@gmail.com>.
On 07/05/2007, Justin C. van Vorst <pr...@gmail.com> wrote:
>
> What is the correct number of file descriptors necessary to not have ActiveMQ
> fail in a production environment?

It depends on how many connections you're gonna have - I'd set it
pretty high just in case; say to a few thousand.

-- 
James
-------
http://macstrac.blogspot.com/

Open Source SOA
http://open.iona.com

Re: OutOfMemoryError using 4.1.1

Posted by "Justin C. van Vorst" <pr...@gmail.com>.
What is the correct number of file descriptors necessary to not have ActiveMQ
fail in a production environment?



James.Strachan wrote:
> 
> It might be you are just running out of threads; maybe your setting of
> file descriptors per process is too low?
> 
> 
> On 4/23/07, Justin C. van Vorst <pr...@gmail.com> wrote:
>>
>> On our production server this weekend, activemq-4.1.1 hit a
>> java.lang.OutOfMemoryError exception and then proceeded to stop
>> functioning.
>> The process was using about 3GB of RAM.
>>
>> I've seen some other posts about OOME, but none that quite match the
>> stack
>> of ours.  Does anyone have a place for me to start investigating this
>> issue?
>>
>> Below are the error logs, command line start-up, and activemq.xml config:
>>
>>
>> Error log:
>>
>> 2007-04-23 02:43:10,286 [localhost:61613] WARN 
>> ManagedTransportConnection
>> - Failed to register MBean:
>> org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_54813
>> 2007-04-23 02:49:10,415 [localhost:61613] WARN 
>> ManagedTransportConnection
>> - Failed to register MBean:
>> org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_58365
>> 2007-04-23 03:11:10,301 [localhost:61613] WARN 
>> ManagedTransportConnection
>> - Failed to register MBean:
>> org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_42109
>> 2007-04-23 03:15:10,056 [localhost:61613] WARN 
>> ManagedTransportConnection
>> - Failed to register MBean:
>> org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_44595
>> 2007-04-23 03:27:10,120 [localhost:61613] WARN 
>> ManagedTransportConnection
>> - Failed to register MBean:
>> org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_51338
>> 2007-04-23 03:44:31,858 [eckpoint Worker] ERROR JournalPersistenceAdapter
>> - Failed to checkpoint a message store:
>> edu.emory.mathcs.backport.java.util.concurrent.ExecutionException:
>> java.lang.OutOfMemoryError: unable to create new native thread
>> edu.emory.mathcs.backport.java.util.concurrent.ExecutionException:
>> java.lang.OutOfMemoryError: unable to create new native thread
>>         at
>> edu.emory.mathcs.backport.java.util.concurrent.FutureTask.getResult(FutureTask.java:299)
>>         at
>> edu.emory.mathcs.backport.java.util.concurrent.FutureTask.get(FutureTask.java:118)
>>         at
>> org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:386)
>>         at
>> org.apache.activemq.store.journal.JournalPersistenceAdapter$2.iterate(JournalPersistenceAdapter.java:129)
>>         at
>> org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:101)
>>         at
>> org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:25)
>>         at
>> org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:39)
>> Caused by: java.lang.OutOfMemoryError: unable to create new native thread
>>         at java.lang.Thread.start0(Native Method)
>>         at java.lang.Thread.start(Thread.java:574)
>>         at java.util.Timer.<init>(Timer.java:154)
>>         at java.util.Timer.<init>(Timer.java:122)
>>         at com.mysql.jdbc.Connection.<init>(Connection.java:1441)
>>         at
>> com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:266)
>>         at
>> org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:37)
>>         at
>> org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:290)
>>         at
>> org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:771)
>>         at
>> org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:95)
>>         at
>> org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:544)
>>         at
>> org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:55)
>>         at
>> org.apache.activemq.store.jdbc.TransactionContext.begin(TransactionContext.java:147)
>>         at
>> org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.beginTransaction(JDBCPersistenceAdapter.java:358)
>>         at
>> org.apache.activemq.store.journal.JournalPersistenceAdapter.beginTransaction(JournalPersistenceAdapter.java:189)
>>         at
>> org.apache.activemq.util.TransactionTemplate.run(TransactionTemplate.java:41)
>>         at
>> org.apache.activemq.store.journal.JournalMessageStore.checkpoint(JournalMessageStore.java:247)
>>         at
>> org.apache.activemq.store.journal.JournalMessageStore.checkpoint(JournalMessageStore.java:221)
>>         at
>> org.apache.activemq.store.journal.JournalPersistenceAdapter$4.call(JournalPersistenceAdapter.java:356)
>>         at
>> edu.emory.mathcs.backport.java.util.concurrent.FutureTask.run(FutureTask.java:176)
>>         at
>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
>>         at
>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
>>         at java.lang.Thread.run(Thread.java:595)
>> 2007-04-23 03:44:31,858 [eckpoint Worker] ERROR JournalPersistenceAdapter
>> - Failed to mark the Journal:
>> org.apache.activeio.journal.InvalidRecordLocationException: The location
>> is
>> less than the last mark.
>> org.apache.activeio.journal.InvalidRecordLocationException: The location
>> is
>> less than the last mark.
>>         at
>> org.apache.activeio.journal.active.JournalImpl.setMark(JournalImpl.java:340)
>>         at
>> org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:403)
>>         at
>> org.apache.activemq.store.journal.JournalPersistenceAdapter$2.iterate(JournalPersistenceAdapter.java:129)
>>         at
>> org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:101)
>>         at
>> org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:25)
>>         at
>> org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:39)
>> 2007-04-23 03:47:10,105 [localhost:61613] WARN 
>> ManagedTransportConnection
>> - Failed to register MBean:
>> org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_34348
>> 2007-04-23 03:50:01,821 [eckpoint Worker] ERROR JournalPersistenceAdapter
>> - Failed to checkpoint a message store:
>> edu.emory.mathcs.backport.java.util.concurrent.ExecutionException:
>> java.io.IOException: Already started.
>> edu.emory.mathcs.backport.java.util.concurrent.ExecutionException:
>> java.io.IOException: Already started.
>>         at
>> edu.emory.mathcs.backport.java.util.concurrent.FutureTask.getResult(FutureTask.java:299)
>>         at
>> edu.emory.mathcs.backport.java.util.concurrent.FutureTask.get(FutureTask.java:118)
>>         at
>> org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:386)
>>         at
>> org.apache.activemq.store.journal.JournalPersistenceAdapter$2.iterate(JournalPersistenceAdapter.java:129)
>>         at
>> org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:101)
>>         at
>> org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:25)
>>         at
>> org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:39)
>> Caused by: java.io.IOException: Already started.
>>         at
>> org.apache.activemq.store.jdbc.TransactionContext.begin(TransactionContext.java:145)
>>         at
>> org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.beginTransaction(JDBCPersistenceAdapter.java:358)
>>         at
>> org.apache.activemq.store.journal.JournalPersistenceAdapter.beginTransaction(JournalPersistenceAdapter.java:189)
>>         at
>> org.apache.activemq.util.TransactionTemplate.run(TransactionTemplate.java:41)
>>         at
>> org.apache.activemq.store.journal.JournalMessageStore.checkpoint(JournalMessageStore.java:247)
>>         at
>> org.apache.activemq.store.journal.JournalMessageStore.checkpoint(JournalMessageStore.java:221)
>>         at
>> org.apache.activemq.store.journal.JournalPersistenceAdapter$4.call(JournalPersistenceAdapter.java:356)
>>         at
>> edu.emory.mathcs.backport.java.util.concurrent.FutureTask.run(FutureTask.java:176)
>>         at
>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
>>         at
>> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
>>         at java.lang.Thread.run(Thread.java:595)
>> 2007-04-23 03:50:01,821 [eckpoint Worker] ERROR JournalPersistenceAdapter
>> - Failed to mark the Journal:
>> org.apache.activeio.journal.InvalidRecordLocationException: The location
>> is
>> less than the last mark.
>> org.apache.activeio.journal.InvalidRecordLocationException: The location
>> is
>> less than the last mark.
>>         at
>> org.apache.activeio.journal.active.JournalImpl.setMark(JournalImpl.java:340)
>>         at
>> org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:403)
>>         at
>> org.apache.activemq.store.journal.JournalPersistenceAdapter$2.iterate(JournalPersistenceAdapter.java:129)
>>         at
>> org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:101)
>>         at
>> org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:25)
>>         at
>> org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:39)
>> 2007-04-23 03:51:10,188 [localhost:61613] WARN 
>> ManagedTransportConnection
>> - Failed to register MBean:
>> org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_36436
>> 2007-04-23 04:13:10,134 [localhost:61613] WARN 
>> ManagedTransportConnection
>> - Failed to register MBean:
>> org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_49022
>> 2007-04-23 04:17:10,094 [localhost:61613] WARN 
>> ManagedTransportConnection
>> - Failed to register MBean:
>> org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_51068
>>
>>
>>
>> Command line:
>>
>> usr/java/current/bin/java -Xmx512M
>> -Dorg.apache.activemq.UseDedicatedTaskRunner=true
>> -Dderby.system.home=/opt/activemq-4.1.1/data
>> -Dderby.storage.fileSyncTransactionLog=true
>> -Dcom.sun.management.jmxremote
>> -Djavax.net.ssl.keyStorePassword=password
>> -Djavax.net.ssl.trustStorePassword=password
>> -Djavax.net.ssl.keyStore=/opt/activemq-4.1.1/conf/broker.ks
>> -Djavax.net.ssl.trustStore=/opt/activemq-4.1.1/conf/broker.ts -classpath
>> -Dactivemq.home=/opt/rhg/activemq-4.1.1
>> -Dactivemq.base=/opt/activemq-4.1.1
>> -jar /opt/activemq-4.1.1/bin/run.jar
>>
>>
>>
>> activemq.xml:
>>
>> <beans>
>>
>>   <bean
>> class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"/>
>>
>>   <broker brokerName="localhost" useJmx="true"
>> xmlns="http://activemq.org/config/1.0">
>>
>>     <destinationPolicy>
>>       <policyMap><policyEntries>
>>           <policyEntry topic="FOO.>">
>>             <dispatchPolicy>
>>               <strictOrderDispatchPolicy />
>>             </dispatchPolicy>
>>             <subscriptionRecoveryPolicy>
>>               <lastImageSubscriptionRecoveryPolicy />
>>             </subscriptionRecoveryPolicy>
>>           </policyEntry>
>>       </policyEntries></policyMap>
>>     </destinationPolicy>
>>
>>     <persistenceAdapter>
>>       <journaledJDBC journalLogFiles="5" dataDirectory="../activemq-data"
>> dataSource="#mysql-ds"/>
>>     </persistenceAdapter>
>>
>>     <transportConnectors>
>>        <transportConnector name="openwire" uri="tcp://localhost:61616"
>> discoveryUri="multicast://default"/>
>>        <transportConnector name="ssl"     uri="ssl://localhost:61617"/>
>>        <transportConnector name="stomp"   uri="stomp://localhost:61613"/>
>>     </transportConnectors>
>>
>>     <networkConnectors>
>>       <networkConnector name="default-nc" uri="multicast://default"/>
>>     </networkConnectors>
>>
>>   </broker>
>>
>>   <bean id="mysql-ds" class="org.apache.commons.dbcp.BasicDataSource"
>> destroy-method="close">
>>     <property name="driverClassName" value="com.mysql.jdbc.Driver"/>
>>     <property name="url"
>> value="jdbc:mysql://x.x.x.x/activemq?relaxAutoCommit=true"/>
>>     <property name="username" value="username"/>
>>     <property name="password" value="password"/>
>>     <property name="poolPreparedStatements" value="true"/>
>>   </bean>
>>
>> </beans>
>> --
>> View this message in context:
>> http://www.nabble.com/OutOfMemoryError-using-4.1.1-tf3632613s2354.html#a10143322
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>
> 
> 
> -- 
> James
> -------
> http://macstrac.blogspot.com/
> 
> 

-- 
View this message in context: http://www.nabble.com/OutOfMemoryError-using-4.1.1-tf3632613s2354.html#a10359823
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Re: OutOfMemoryError using 4.1.1

Posted by James Strachan <ja...@gmail.com>.
It might be you are just running out of threads; maybe your setting of
file descriptors per process is too low?


On 4/23/07, Justin C. van Vorst <pr...@gmail.com> wrote:
>
> On our production server this weekend, activemq-4.1.1 hit a
> java.lang.OutOfMemoryError exception and then proceeded to stop functioning.
> The process was using about 3GB of RAM.
>
> I've seen some other posts about OOME, but none that quite match the stack
> of ours.  Does anyone have a place for me to start investigating this issue?
>
> Below are the error logs, command line start-up, and activemq.xml config:
>
>
> Error log:
>
> 2007-04-23 02:43:10,286 [localhost:61613] WARN  ManagedTransportConnection
> - Failed to register MBean:
> org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_54813
> 2007-04-23 02:49:10,415 [localhost:61613] WARN  ManagedTransportConnection
> - Failed to register MBean:
> org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_58365
> 2007-04-23 03:11:10,301 [localhost:61613] WARN  ManagedTransportConnection
> - Failed to register MBean:
> org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_42109
> 2007-04-23 03:15:10,056 [localhost:61613] WARN  ManagedTransportConnection
> - Failed to register MBean:
> org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_44595
> 2007-04-23 03:27:10,120 [localhost:61613] WARN  ManagedTransportConnection
> - Failed to register MBean:
> org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_51338
> 2007-04-23 03:44:31,858 [eckpoint Worker] ERROR JournalPersistenceAdapter
> - Failed to checkpoint a message store:
> edu.emory.mathcs.backport.java.util.concurrent.ExecutionException:
> java.lang.OutOfMemoryError: unable to create new native thread
> edu.emory.mathcs.backport.java.util.concurrent.ExecutionException:
> java.lang.OutOfMemoryError: unable to create new native thread
>         at
> edu.emory.mathcs.backport.java.util.concurrent.FutureTask.getResult(FutureTask.java:299)
>         at
> edu.emory.mathcs.backport.java.util.concurrent.FutureTask.get(FutureTask.java:118)
>         at
> org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:386)
>         at
> org.apache.activemq.store.journal.JournalPersistenceAdapter$2.iterate(JournalPersistenceAdapter.java:129)
>         at
> org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:101)
>         at
> org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:25)
>         at
> org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:39)
> Caused by: java.lang.OutOfMemoryError: unable to create new native thread
>         at java.lang.Thread.start0(Native Method)
>         at java.lang.Thread.start(Thread.java:574)
>         at java.util.Timer.<init>(Timer.java:154)
>         at java.util.Timer.<init>(Timer.java:122)
>         at com.mysql.jdbc.Connection.<init>(Connection.java:1441)
>         at
> com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:266)
>         at
> org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:37)
>         at
> org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:290)
>         at
> org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:771)
>         at
> org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:95)
>         at
> org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:544)
>         at
> org.apache.activemq.store.jdbc.TransactionContext.getConnection(TransactionContext.java:55)
>         at
> org.apache.activemq.store.jdbc.TransactionContext.begin(TransactionContext.java:147)
>         at
> org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.beginTransaction(JDBCPersistenceAdapter.java:358)
>         at
> org.apache.activemq.store.journal.JournalPersistenceAdapter.beginTransaction(JournalPersistenceAdapter.java:189)
>         at
> org.apache.activemq.util.TransactionTemplate.run(TransactionTemplate.java:41)
>         at
> org.apache.activemq.store.journal.JournalMessageStore.checkpoint(JournalMessageStore.java:247)
>         at
> org.apache.activemq.store.journal.JournalMessageStore.checkpoint(JournalMessageStore.java:221)
>         at
> org.apache.activemq.store.journal.JournalPersistenceAdapter$4.call(JournalPersistenceAdapter.java:356)
>         at
> edu.emory.mathcs.backport.java.util.concurrent.FutureTask.run(FutureTask.java:176)
>         at
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
>         at
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
>         at java.lang.Thread.run(Thread.java:595)
> 2007-04-23 03:44:31,858 [eckpoint Worker] ERROR JournalPersistenceAdapter
> - Failed to mark the Journal:
> org.apache.activeio.journal.InvalidRecordLocationException: The location is
> less than the last mark.
> org.apache.activeio.journal.InvalidRecordLocationException: The location is
> less than the last mark.
>         at
> org.apache.activeio.journal.active.JournalImpl.setMark(JournalImpl.java:340)
>         at
> org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:403)
>         at
> org.apache.activemq.store.journal.JournalPersistenceAdapter$2.iterate(JournalPersistenceAdapter.java:129)
>         at
> org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:101)
>         at
> org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:25)
>         at
> org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:39)
> 2007-04-23 03:47:10,105 [localhost:61613] WARN  ManagedTransportConnection
> - Failed to register MBean:
> org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_34348
> 2007-04-23 03:50:01,821 [eckpoint Worker] ERROR JournalPersistenceAdapter
> - Failed to checkpoint a message store:
> edu.emory.mathcs.backport.java.util.concurrent.ExecutionException:
> java.io.IOException: Already started.
> edu.emory.mathcs.backport.java.util.concurrent.ExecutionException:
> java.io.IOException: Already started.
>         at
> edu.emory.mathcs.backport.java.util.concurrent.FutureTask.getResult(FutureTask.java:299)
>         at
> edu.emory.mathcs.backport.java.util.concurrent.FutureTask.get(FutureTask.java:118)
>         at
> org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:386)
>         at
> org.apache.activemq.store.journal.JournalPersistenceAdapter$2.iterate(JournalPersistenceAdapter.java:129)
>         at
> org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:101)
>         at
> org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:25)
>         at
> org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:39)
> Caused by: java.io.IOException: Already started.
>         at
> org.apache.activemq.store.jdbc.TransactionContext.begin(TransactionContext.java:145)
>         at
> org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.beginTransaction(JDBCPersistenceAdapter.java:358)
>         at
> org.apache.activemq.store.journal.JournalPersistenceAdapter.beginTransaction(JournalPersistenceAdapter.java:189)
>         at
> org.apache.activemq.util.TransactionTemplate.run(TransactionTemplate.java:41)
>         at
> org.apache.activemq.store.journal.JournalMessageStore.checkpoint(JournalMessageStore.java:247)
>         at
> org.apache.activemq.store.journal.JournalMessageStore.checkpoint(JournalMessageStore.java:221)
>         at
> org.apache.activemq.store.journal.JournalPersistenceAdapter$4.call(JournalPersistenceAdapter.java:356)
>         at
> edu.emory.mathcs.backport.java.util.concurrent.FutureTask.run(FutureTask.java:176)
>         at
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
>         at
> edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
>         at java.lang.Thread.run(Thread.java:595)
> 2007-04-23 03:50:01,821 [eckpoint Worker] ERROR JournalPersistenceAdapter
> - Failed to mark the Journal:
> org.apache.activeio.journal.InvalidRecordLocationException: The location is
> less than the last mark.
> org.apache.activeio.journal.InvalidRecordLocationException: The location is
> less than the last mark.
>         at
> org.apache.activeio.journal.active.JournalImpl.setMark(JournalImpl.java:340)
>         at
> org.apache.activemq.store.journal.JournalPersistenceAdapter.doCheckpoint(JournalPersistenceAdapter.java:403)
>         at
> org.apache.activemq.store.journal.JournalPersistenceAdapter$2.iterate(JournalPersistenceAdapter.java:129)
>         at
> org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:101)
>         at
> org.apache.activemq.thread.DedicatedTaskRunner.access$000(DedicatedTaskRunner.java:25)
>         at
> org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:39)
> 2007-04-23 03:51:10,188 [localhost:61613] WARN  ManagedTransportConnection
> - Failed to register MBean:
> org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_36436
> 2007-04-23 04:13:10,134 [localhost:61613] WARN  ManagedTransportConnection
> - Failed to register MBean:
> org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_49022
> 2007-04-23 04:17:10,094 [localhost:61613] WARN  ManagedTransportConnection
> - Failed to register MBean:
> org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=stomp,ViewType=address,Name=/x.x.x.x_51068
>
>
>
> Command line:
>
> usr/java/current/bin/java -Xmx512M
> -Dorg.apache.activemq.UseDedicatedTaskRunner=true
> -Dderby.system.home=/opt/activemq-4.1.1/data
> -Dderby.storage.fileSyncTransactionLog=true -Dcom.sun.management.jmxremote
> -Djavax.net.ssl.keyStorePassword=password
> -Djavax.net.ssl.trustStorePassword=password
> -Djavax.net.ssl.keyStore=/opt/activemq-4.1.1/conf/broker.ks
> -Djavax.net.ssl.trustStore=/opt/activemq-4.1.1/conf/broker.ts -classpath
> -Dactivemq.home=/opt/rhg/activemq-4.1.1 -Dactivemq.base=/opt/activemq-4.1.1
> -jar /opt/activemq-4.1.1/bin/run.jar
>
>
>
> activemq.xml:
>
> <beans>
>
>   <bean
> class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"/>
>
>   <broker brokerName="localhost" useJmx="true"
> xmlns="http://activemq.org/config/1.0">
>
>     <destinationPolicy>
>       <policyMap><policyEntries>
>           <policyEntry topic="FOO.>">
>             <dispatchPolicy>
>               <strictOrderDispatchPolicy />
>             </dispatchPolicy>
>             <subscriptionRecoveryPolicy>
>               <lastImageSubscriptionRecoveryPolicy />
>             </subscriptionRecoveryPolicy>
>           </policyEntry>
>       </policyEntries></policyMap>
>     </destinationPolicy>
>
>     <persistenceAdapter>
>       <journaledJDBC journalLogFiles="5" dataDirectory="../activemq-data"
> dataSource="#mysql-ds"/>
>     </persistenceAdapter>
>
>     <transportConnectors>
>        <transportConnector name="openwire" uri="tcp://localhost:61616"
> discoveryUri="multicast://default"/>
>        <transportConnector name="ssl"     uri="ssl://localhost:61617"/>
>        <transportConnector name="stomp"   uri="stomp://localhost:61613"/>
>     </transportConnectors>
>
>     <networkConnectors>
>       <networkConnector name="default-nc" uri="multicast://default"/>
>     </networkConnectors>
>
>   </broker>
>
>   <bean id="mysql-ds" class="org.apache.commons.dbcp.BasicDataSource"
> destroy-method="close">
>     <property name="driverClassName" value="com.mysql.jdbc.Driver"/>
>     <property name="url"
> value="jdbc:mysql://x.x.x.x/activemq?relaxAutoCommit=true"/>
>     <property name="username" value="username"/>
>     <property name="password" value="password"/>
>     <property name="poolPreparedStatements" value="true"/>
>   </bean>
>
> </beans>
> --
> View this message in context: http://www.nabble.com/OutOfMemoryError-using-4.1.1-tf3632613s2354.html#a10143322
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>


-- 
James
-------
http://macstrac.blogspot.com/

Re: OutOfMemoryError using 4.1.1

Posted by Yohan Liyanage <yo...@gmail.com>.
I came across this issue notice :

https://issues.apache.org/activemq/browse/AMQ-1044
https://issues.apache.org/activemq/browse/AMQ-1044 


"If async error such as a inactivity IO exception is detected while a
subscription is being created, then that subscription is not fully cleaned
up when the connection is disposed. If the client is using failover and
reconnects, then it gets a "Failed to register MBean error." Since the
subscription is still registered in JMX. This is also a source of a memory
leak since subscriptions are are left registered in JMX." - ActiveMQ Issue
List


-- 
View this message in context: http://www.nabble.com/OutOfMemoryError-using-4.1.1-tf3632613s2354.html#a13583188
Sent from the ActiveMQ - User mailing list archive at Nabble.com.