You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Sandeep Chayapathi <sa...@wssource.com> on 2007/02/05 16:20:59 UTC
jdbcPersistenceAdapter enlightenment
Hi all,
As a followup to these questions:
http://http://www.nabble.com/how-to-set-the-message-of-the-broker-will-store-in-the-mySQL-only-tf2957449.html#a8294916
how to set the message...
http://www.nabble.com/Queue-Memory-Limits-tf1857084.html Queue Memory limit
http://www.nabble.com/Async-Exception%3A-Broken-Pipe-tf3160498.html#a8765808
Async exception
My bane of existence has been activemq, for a while now. The single issue,
is activemq running out of heap memory, when I use offline durable
subscribers. I wondered if by using db persistence, instead of caching
messages, would solve this proble. To that end, I modified my activemq.xml,
thusly:
------
<beans>
<bean
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"/>
<broker brokerName="localhost" useJmx="true"
xmlns="http://activemq.org/config/1.0" persistent="true">
<memoryManager>
<usageManager id="memory-manager" limit="1024 MB"/>
</memoryManager>
<persistenceAdapter>
<jdbcPersistenceAdapter dataSource="#mysql-ds"/>
</persistenceAdapter>
</broker>
<!-- MySql DataSource Sample Setup -->
<bean id="mysql-ds" class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver"/>
<property name="url"
value="jdbc:mysql://localhost/activemq?relaxAutoCommit=true"/>
<property name="username" value="sandeep"/>
<property name="password" value="password"/>
<property name="poolPreparedStatements" value="false"/>
</bean>
</beans>
-----
I had 3 offline durable subscribers, to a topic (tired with queues also) and
1 producer, which pumped in messages to the topic. The jvm was started with
-Xmx1024M.
I found that, although the messages were supposed to be persisted to the
mysql db, all the tables were empty. Once the memory usage hit the jvm's max
allocated memory, I got the dreaded
"java.lang.OutOfMemoryError: Java heap space", yet no messages were found in
the datastore.
Can anyone pls enlighten me why the un-acked messages were not being
persisted to the datastore and if there is a way to force activemq to
persist messages ? Thanks
- Sandeep
--
View this message in context: http://www.nabble.com/jdbcPersistenceAdapter-enlightenment-tf3174884.html#a8808214
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: jdbcPersistenceAdapter enlightenment
Posted by Sandeep Chayapathi <sa...@wssource.com>.
Hi Rob,
does that mean the persistence is broken in 4.1 ? will you be updating the
4.2 release time line ? For now, I want to measure the performance impact of
using persistence and also see if it would solve the out of memory problem.
- Sandeep
--
View this message in context: http://www.nabble.com/jdbcPersistenceAdapter-enlightenment-tf3174884.html#a8810963
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: jdbcPersistenceAdapter enlightenment
Posted by Rob Davies <ra...@gmail.com>.
Hi Peter,
This means there is a known issue with very high volume Queues/
durable subscribers in 4.x that is being addressed in the next release.
I wouldn't recommend you go with the next release in production
however, not all the functionality is tied down yet.
You could either help contribute to the stability of the Apache
ActiveMQ (by either submitting user junit test cases, contributing
fixes etc.) - or you could investigate commercial support - see:
http://www.logicblaze.com/services.jsp
cheers,
Rob Davies
http://rajdavies.blogspot.com/
On 5 Feb 2007, at 23:25, Peter Leonard wrote:
>
> Does this mean "this is a known bug in 4.1 and has been fixed", or
> "take 4.2-snapshot for a spin, and maybe it helps?"
>
> We're running 4.1 in production, and this and other memory leaks
> (e.g. the whole issue revolving around disconnect() not working
> with durable topics) are causing major problems for us - there
> doesn't seem to be any solution other than to monitor memory usage,
> bring everything to a halt, and restart services in a maintenance
> window. We're unable to run unreleased code in production, what's
> the current wisdom on 4.2 coming out? Or a 4.1.1? I am somewhat
> scared of 4.2.0 introducing a whole new set of unknown headaches -
> at least we seem to know our issues with 4.1 at this time.
>
> Thank you,
>
> Peter
>
>
> Rob Davies wrote:
>> Hi Sandeep,
>> I think using the latest snapshot for 4.2 will help:
>> http://people.apache.org/repo/m2-snapshot-repository/org/apache/
>> activemq/apache-activemq/4.2-incubator-SNAPSHOT/ cheers,
>> Rob
>> On 5 Feb 2007, at 15:20, Sandeep Chayapathi wrote:
>>>
>>> Hi all,
>>>
>>> As a followup to these questions:
>>>
>>> http://http://www.nabble.com/how-to-set-the-message-of-the-broker-
>>> will-store-in-the-mySQL-only-tf2957449.html#a8294916
>>> how to set the message...
>>> http://www.nabble.com/Queue-Memory-Limits-tf1857084.html Queue
>>> Memory limit
>>> http://www.nabble.com/Async-Exception%3A-Broken-Pipe-
>>> tf3160498.html#a8765808
>>> Async exception
>>>
>>> My bane of existence has been activemq, for a while now. The
>>> single issue,
>>> is activemq running out of heap memory, when I use offline durable
>>> subscribers. I wondered if by using db persistence, instead of
>>> caching
>>> messages, would solve this proble. To that end, I modified my
>>> activemq.xml,
>>> thusly:
>>>
>>> ------
>>> <beans>
>>> <bean
>>> class="org.springframework.beans.factory.config.PropertyPlaceholderC
>>> onfigurer"/>
>>>
>>> <broker brokerName="localhost" useJmx="true"
>>> xmlns="http://activemq.org/config/1.0" persistent="true">
>>>
>>> <memoryManager>
>>> <usageManager id="memory-manager" limit="1024 MB"/>
>>> </memoryManager>
>>> <persistenceAdapter>
>>> <jdbcPersistenceAdapter dataSource="#mysql-ds"/>
>>> </persistenceAdapter>
>>> </broker>
>>> <!-- MySql DataSource Sample Setup -->
>>> <bean id="mysql-ds"
>>> class="org.apache.commons.dbcp.BasicDataSource"
>>> destroy-method="close">
>>> <property name="driverClassName" value="com.mysql.jdbc.Driver"/>
>>> <property name="url"
>>> value="jdbc:mysql://localhost/activemq?relaxAutoCommit=true"/>
>>> <property name="username" value="sandeep"/>
>>> <property name="password" value="password"/>
>>> <property name="poolPreparedStatements" value="false"/>
>>> </bean>
>>> </beans>
>>>
>>> -----
>>>
>>> I had 3 offline durable subscribers, to a topic (tired with
>>> queues also) and
>>> 1 producer, which pumped in messages to the topic. The jvm was
>>> started with
>>> -Xmx1024M.
>>>
>>> I found that, although the messages were supposed to be persisted
>>> to the
>>> mysql db, all the tables were empty. Once the memory usage hit
>>> the jvm's max
>>> allocated memory, I got the dreaded
>>> "java.lang.OutOfMemoryError: Java heap space", yet no messages
>>> were found in
>>> the datastore.
>>>
>>> Can anyone pls enlighten me why the un-acked messages were not being
>>> persisted to the datastore and if there is a way to force
>>> activemq to
>>> persist messages ? Thanks
>>>
>>> - Sandeep
>>> --View this message in context: http://www.nabble.com/
>>> jdbcPersistenceAdapter-enlightenment-tf3174884.html#a8808214
>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>
Re: jdbcPersistenceAdapter enlightenment
Posted by Peter Leonard <pe...@peteleonard.com>.
Does this mean "this is a known bug in 4.1 and has been fixed", or "take
4.2-snapshot for a spin, and maybe it helps?"
We're running 4.1 in production, and this and other memory leaks (e.g.
the whole issue revolving around disconnect() not working with durable
topics) are causing major problems for us - there doesn't seem to be any
solution other than to monitor memory usage, bring everything to a halt,
and restart services in a maintenance window. We're unable to run
unreleased code in production, what's the current wisdom on 4.2 coming
out? Or a 4.1.1? I am somewhat scared of 4.2.0 introducing a whole new
set of unknown headaches - at least we seem to know our issues with 4.1
at this time.
Thank you,
Peter
Rob Davies wrote:
> Hi Sandeep,
>
> I think using the latest snapshot for 4.2 will help:
>
> http://people.apache.org/repo/m2-snapshot-repository/org/apache/activemq/apache-activemq/4.2-incubator-SNAPSHOT/
>
>
> cheers,
>
> Rob
>
> On 5 Feb 2007, at 15:20, Sandeep Chayapathi wrote:
>
>>
>> Hi all,
>>
>> As a followup to these questions:
>>
>> http://http://www.nabble.com/how-to-set-the-message-of-the-broker-will-store-in-the-mySQL-only-tf2957449.html#a8294916
>>
>> how to set the message...
>> http://www.nabble.com/Queue-Memory-Limits-tf1857084.html Queue Memory
>> limit
>> http://www.nabble.com/Async-Exception%3A-Broken-Pipe-tf3160498.html#a8765808
>>
>> Async exception
>>
>> My bane of existence has been activemq, for a while now. The single
>> issue,
>> is activemq running out of heap memory, when I use offline durable
>> subscribers. I wondered if by using db persistence, instead of caching
>> messages, would solve this proble. To that end, I modified my
>> activemq.xml,
>> thusly:
>>
>> ------
>> <beans>
>> <bean
>> class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"/>
>>
>>
>> <broker brokerName="localhost" useJmx="true"
>> xmlns="http://activemq.org/config/1.0" persistent="true">
>>
>> <memoryManager>
>> <usageManager id="memory-manager" limit="1024 MB"/>
>> </memoryManager>
>> <persistenceAdapter>
>> <jdbcPersistenceAdapter dataSource="#mysql-ds"/>
>> </persistenceAdapter>
>> </broker>
>> <!-- MySql DataSource Sample Setup -->
>> <bean id="mysql-ds" class="org.apache.commons.dbcp.BasicDataSource"
>> destroy-method="close">
>> <property name="driverClassName" value="com.mysql.jdbc.Driver"/>
>> <property name="url"
>> value="jdbc:mysql://localhost/activemq?relaxAutoCommit=true"/>
>> <property name="username" value="sandeep"/>
>> <property name="password" value="password"/>
>> <property name="poolPreparedStatements" value="false"/>
>> </bean>
>> </beans>
>>
>> -----
>>
>> I had 3 offline durable subscribers, to a topic (tired with queues
>> also) and
>> 1 producer, which pumped in messages to the topic. The jvm was started
>> with
>> -Xmx1024M.
>>
>> I found that, although the messages were supposed to be persisted to the
>> mysql db, all the tables were empty. Once the memory usage hit the
>> jvm's max
>> allocated memory, I got the dreaded
>> "java.lang.OutOfMemoryError: Java heap space", yet no messages were
>> found in
>> the datastore.
>>
>> Can anyone pls enlighten me why the un-acked messages were not being
>> persisted to the datastore and if there is a way to force activemq to
>> persist messages ? Thanks
>>
>> - Sandeep
>> --View this message in context:
>> http://www.nabble.com/jdbcPersistenceAdapter-enlightenment-tf3174884.html#a8808214
>>
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
Re: jdbcPersistenceAdapter enlightenment
Posted by Rob Davies <ra...@gmail.com>.
Hi Sandeep,
I think using the latest snapshot for 4.2 will help:
http://people.apache.org/repo/m2-snapshot-repository/org/apache/
activemq/apache-activemq/4.2-incubator-SNAPSHOT/
cheers,
Rob
On 5 Feb 2007, at 15:20, Sandeep Chayapathi wrote:
>
> Hi all,
>
> As a followup to these questions:
>
> http://http://www.nabble.com/how-to-set-the-message-of-the-broker-
> will-store-in-the-mySQL-only-tf2957449.html#a8294916
> how to set the message...
> http://www.nabble.com/Queue-Memory-Limits-tf1857084.html Queue
> Memory limit
> http://www.nabble.com/Async-Exception%3A-Broken-Pipe-
> tf3160498.html#a8765808
> Async exception
>
> My bane of existence has been activemq, for a while now. The single
> issue,
> is activemq running out of heap memory, when I use offline durable
> subscribers. I wondered if by using db persistence, instead of caching
> messages, would solve this proble. To that end, I modified my
> activemq.xml,
> thusly:
>
> ------
> <beans>
> <bean
> class="org.springframework.beans.factory.config.PropertyPlaceholderCon
> figurer"/>
>
> <broker brokerName="localhost" useJmx="true"
> xmlns="http://activemq.org/config/1.0" persistent="true">
>
> <memoryManager>
> <usageManager id="memory-manager" limit="1024 MB"/>
> </memoryManager>
> <persistenceAdapter>
> <jdbcPersistenceAdapter dataSource="#mysql-ds"/>
> </persistenceAdapter>
> </broker>
> <!-- MySql DataSource Sample Setup -->
> <bean id="mysql-ds" class="org.apache.commons.dbcp.BasicDataSource"
> destroy-method="close">
> <property name="driverClassName" value="com.mysql.jdbc.Driver"/>
> <property name="url"
> value="jdbc:mysql://localhost/activemq?relaxAutoCommit=true"/>
> <property name="username" value="sandeep"/>
> <property name="password" value="password"/>
> <property name="poolPreparedStatements" value="false"/>
> </bean>
> </beans>
>
> -----
>
> I had 3 offline durable subscribers, to a topic (tired with queues
> also) and
> 1 producer, which pumped in messages to the topic. The jvm was
> started with
> -Xmx1024M.
>
> I found that, although the messages were supposed to be persisted
> to the
> mysql db, all the tables were empty. Once the memory usage hit the
> jvm's max
> allocated memory, I got the dreaded
> "java.lang.OutOfMemoryError: Java heap space", yet no messages were
> found in
> the datastore.
>
> Can anyone pls enlighten me why the un-acked messages were not being
> persisted to the datastore and if there is a way to force activemq to
> persist messages ? Thanks
>
> - Sandeep
> --
> View this message in context: http://www.nabble.com/
> jdbcPersistenceAdapter-enlightenment-tf3174884.html#a8808214
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>