You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Gary Tully <ga...@gmail.com> on 2011/06/01 15:49:13 UTC
Re: Persistence messages and StoreUsage Memory
did you get sorted?
you need to remove the store limit, replace:
<storeUsage>
<storeUsage limit="1 gb" name="foo"/>
</storeUsage>
with the following, to be limited by available disk space:
<storeUsage>
<storeUsage limit="0"/>
</storeUsage>
On 26 May 2011 11:50, Suneel Papineni <Su...@mettoni.com> wrote:
> Hi,
>
>
>
> I am using Network of brokers and using ActiveMQ5.5.0 and
> ActiveMQ-CPP3.4.0 client application.
>
>
>
> 1. Application 'A' registered to Broker1 and application 'B' is
> registered to Broker2 (Both 'A' and 'B' are registered as Durable
> Consumers). A sends messages to a topic and B receives all messages and
> vice-versa.
>
> 2. Now 'B' is disconnected for almost 1 day, but 'A' is keep on
> sending messages (around 25 messages per second and each message with a
> size of 10kb).
>
> 3. All the messages sent are "PERSISTANT" messages and having
> Time-to-Live as 5 hours.
>
> 4. I am using default configuration....
>
> <persistenceAdapter>
>
> <kahaDB directory="${activemq.base}/data/static-broker1/kahadb" />
>
> </persistenceAdapter>
>
> and
>
> <systemUsage>
>
> <systemUsage>
>
> <memoryUsage>
>
> <memoryUsage limit="100 mb"/>
>
> </memoryUsage>
>
> <storeUsage>
>
> <storeUsage limit="1 gb" name="foo"/>
>
> </storeUsage>
>
> <tempUsage>
>
> <tempUsage limit="200 mb"/>
>
> </tempUsage>
>
> </systemUsage>
>
> </systemUsage>
>
> Also I configured "producerFlowControl" property for Topics as "false"
> in order to allow 'A' to send maximum number of messages.
>
> 5. After sending many messages, 'A' got hung and Broker1 logs
> shows "Usage Manager Store is Full, 100% of 1073741824. Stopping
> producer......"
>
>
>
>
>
> My questions are:
>
> 1. What are the best values I need to configure to stop 'A'
> getting hung or in order to resolve this issue.
>
> 2. As time-to-live for messages is 5 hours, what happens to these
> messages after this timer expires. I know that it is sent to
> ActiveMQ.DLQ, but is it removed from persistence storage or
> storageUsage?
>
> 3. Is there any way I can remove the limits completely and system
> runs until system's disk memory limit (this is to allow maximum number
> of messages. Not a good ideaJ).
>
>
>
> Please let me know if I need to configure any parameters for "kahaDB".
> Thank you for your help.
>
>
>
> Thanks & Regards
>
> Suneel
>
>
>
>
--
http://fusesource.com
http://blog.garytully.com
Re: Persistence messages and StoreUsage Memory
Posted by Gary Tully <ga...@gmail.com>.
@Suneel
no need for doubts, cleanup and expiry will occur as expected but by
default expired messages are sent to the dead letter queue so they
will use up space until they are consumed.
You need to modify the default SharedDeadLetterStrategy and set
processExpired=false so that they don't consume all of your disk
resources.
see: http://activemq.apache.org/message-redelivery-and-dlq-handling.html#MessageRedeliveryandDLQHandling-AutomaticallyDiscardExpiredMessages
On 1 June 2011 17:44, Suneel Papineni <Su...@mettoni.com> wrote:
> Hi Gary,
>
> Thanks for your reply and I just sorted in the same way as you told temporarily, but I am having couple of doubts.
>
> If I keep limit="0", I think khahadb logs goes on increasing and if Disk space is 40gb, after this Producer is blocked to send messages right. In order to stop this, is there any way to cleanup (like expired messages) and release "storeUsage" space or delete old db log files.
>
> I believe there is a parameter "cleanupInterval" of "khahadb" property with default value as 30000 (ms). Could you please let me know what this parameter for. Will it cleans (i.e. deletes expired messages) and releases storeUsage. Or, do I need to follow any other method.
>
> Thanks & Regards
> Suneel
>
>
>
> -----Original Message-----
> From: Gary Tully [mailto:gary.tully@gmail.com]
> Sent: 01 June 2011 14:49
> To: users@activemq.apache.org
> Subject: Re: Persistence messages and StoreUsage Memory
>
> did you get sorted?
> you need to remove the store limit, replace:
> <storeUsage>
> <storeUsage limit="1 gb" name="foo"/>
> </storeUsage>
>
> with the following, to be limited by available disk space:
>
> <storeUsage>
> <storeUsage limit="0"/>
> </storeUsage>
>
> On 26 May 2011 11:50, Suneel Papineni <Su...@mettoni.com> wrote:
>> Hi,
>>
>>
>>
>> I am using Network of brokers and using ActiveMQ5.5.0 and
>> ActiveMQ-CPP3.4.0 client application.
>>
>>
>>
>> 1. Application 'A' registered to Broker1 and application 'B' is
>> registered to Broker2 (Both 'A' and 'B' are registered as Durable
>> Consumers). A sends messages to a topic and B receives all messages and
>> vice-versa.
>>
>> 2. Now 'B' is disconnected for almost 1 day, but 'A' is keep on
>> sending messages (around 25 messages per second and each message with a
>> size of 10kb).
>>
>> 3. All the messages sent are "PERSISTANT" messages and having
>> Time-to-Live as 5 hours.
>>
>> 4. I am using default configuration....
>>
>> <persistenceAdapter>
>>
>> <kahaDB directory="${activemq.base}/data/static-broker1/kahadb" />
>>
>> </persistenceAdapter>
>>
>> and
>>
>> <systemUsage>
>>
>> <systemUsage>
>>
>> <memoryUsage>
>>
>> <memoryUsage limit="100 mb"/>
>>
>> </memoryUsage>
>>
>> <storeUsage>
>>
>> <storeUsage limit="1 gb" name="foo"/>
>>
>> </storeUsage>
>>
>> <tempUsage>
>>
>> <tempUsage limit="200 mb"/>
>>
>> </tempUsage>
>>
>> </systemUsage>
>>
>> </systemUsage>
>>
>> Also I configured "producerFlowControl" property for Topics as "false"
>> in order to allow 'A' to send maximum number of messages.
>>
>> 5. After sending many messages, 'A' got hung and Broker1 logs
>> shows "Usage Manager Store is Full, 100% of 1073741824. Stopping
>> producer......"
>>
>>
>>
>>
>>
>> My questions are:
>>
>> 1. What are the best values I need to configure to stop 'A'
>> getting hung or in order to resolve this issue.
>>
>> 2. As time-to-live for messages is 5 hours, what happens to these
>> messages after this timer expires. I know that it is sent to
>> ActiveMQ.DLQ, but is it removed from persistence storage or
>> storageUsage?
>>
>> 3. Is there any way I can remove the limits completely and system
>> runs until system's disk memory limit (this is to allow maximum number
>> of messages. Not a good ideaJ).
>>
>>
>>
>> Please let me know if I need to configure any parameters for "kahaDB".
>> Thank you for your help.
>>
>>
>>
>> Thanks & Regards
>>
>> Suneel
>>
>>
>>
>>
>
>
>
> --
> http://fusesource.com
> http://blog.garytully.com
>
--
http://fusesource.com
http://blog.garytully.com
Re: Persistence messages and StoreUsage Memory
Posted by Juan Nin <ju...@gmail.com>.
Just as a suggestion, could be nice idea to better explain on the
online documentation about all these memory seetings, what each ones
means, and the typical things that are usually asked on this list
about them.
The "ActiveMQ in Action" book includes some slight things that are not
mentioned on the online docs, but doesn't really explain too much
about them either.
Just my 2 cents.
Juan
On Wednesday, June 1, 2011, Suneel Papineni <Su...@mettoni.com> wrote:
> Hi Gary,
>
> Thanks for your reply and I just sorted in the same way as you told temporarily, but I am having couple of doubts.
>
> If I keep limit="0", I think khahadb logs goes on increasing and if Disk space is 40gb, after this Producer is blocked to send messages right. In order to stop this, is there any way to cleanup (like expired messages) and release "storeUsage" space or delete old db log files.
>
> I believe there is a parameter "cleanupInterval" of "khahadb" property with default value as 30000 (ms). Could you please let me know what this parameter for. Will it cleans (i.e. deletes expired messages) and releases storeUsage. Or, do I need to follow any other method.
>
> Thanks & Regards
> Suneel
>
>
>
> -----Original Message-----
> From: Gary Tully [mailto:gary.tully@gmail.com]
> Sent: 01 June 2011 14:49
> To: users@activemq.apache.org
> Subject: Re: Persistence messages and StoreUsage Memory
>
> did you get sorted?
> you need to remove the store limit, replace:
> <storeUsage>
> <storeUsage limit="1 gb" name="foo"/>
> </storeUsage>
>
> with the following, to be limited by available disk space:
>
> <storeUsage>
> <storeUsage limit="0"/>
> </storeUsage>
>
> On 26 May 2011 11:50, Suneel Papineni <Su...@mettoni.com> wrote:
>> Hi,
>>
>>
>>
>> I am using Network of brokers and using ActiveMQ5.5.0 and
>> ActiveMQ-CPP3.4.0 client application.
>>
>>
>>
>> 1. Application 'A' registered to Broker1 and application 'B' is
>> registered to Broker2 (Both 'A' and 'B' are registered as Durable
>> Consumers). A sends messages to a topic and B receives all messages and
>> vice-versa.
>>
>> 2. Now 'B' is disconnected for almost 1 day, but 'A' is keep on
>> sending messages (around 25 messages per second and each message with a
>> size of 10kb).
>>
>> 3. All the messages sent are "PERSISTANT" messages and having
>> Time-to-Live as 5 hours.
>>
>> 4. I am using default configuration....
>>
>> <persistenceAdapter>
>>
>> <kahaDB directory="${activemq.base}/data/static-broker1/kahadb" />
>>
>> </persistenceAdapter>
>>
>> and
>>
>> <systemUsage>
>>
>> <systemUsage>
>>
>> <memoryUsage>
>>
>> <memoryUsage limit="100 mb"/>
>>
>> </memoryUsage>
>>
>> <storeUsage>
>>
>> <storeUsage limit="1 gb" name="foo"/>
>>
>> </storeUsage>
>>
>> <tempUsage>
>>
>> <tempUsage limit="200 mb"/>
>>
>> </tempUsage>
>>
>> </systemUsage>
>>
>> </systemUsage>
>>
>> Also I configured "producerFlowControl" property for Topics as "false"
>> in order to allow 'A' to send maximum number of messages.
>>
>> 5. After sending many messages, 'A' got hung and Broker1 logs
>> shows "Usage Manager Store is Full, 100% of 1073741824. Stopping
>> producer......"
>>
>>
>>
>>
>>
>> My questions are:
>>
>> 1. What are the best values I need to configure to stop 'A'
>> getting hung or in order to resolve this issue.
>>
>> 2. As time-to-live for messages is 5 hours, what happens to these
>> messages after this timer expires. I know that it is sent to
>> ActiveMQ.DLQ, but is it removed from persistence storage or
>> storageUsage?
>>
>> 3. Is there any way I can remove the limits completely and system
>> runs until system's disk memory limit (this is to allow maximum number
>> of messages. Not a good ideaJ).
>>
>>
>>
>> Please let me know if I need to configure any parameters for "kahaDB".
>> Thank you for your help.
>>
>>
>>
>> Thanks & Regards
>>
>> Suneel
>>
>>
>>
>>
>
>
>
> --
> http://fusesource.com
> http://blo <http://blog.garytully.com>
RE: Persistence messages and StoreUsage Memory
Posted by Suneel Papineni <Su...@mettoni.com>.
Hi Gary,
Thanks for your reply and I just sorted in the same way as you told temporarily, but I am having couple of doubts.
If I keep limit="0", I think khahadb logs goes on increasing and if Disk space is 40gb, after this Producer is blocked to send messages right. In order to stop this, is there any way to cleanup (like expired messages) and release "storeUsage" space or delete old db log files.
I believe there is a parameter "cleanupInterval" of "khahadb" property with default value as 30000 (ms). Could you please let me know what this parameter for. Will it cleans (i.e. deletes expired messages) and releases storeUsage. Or, do I need to follow any other method.
Thanks & Regards
Suneel
-----Original Message-----
From: Gary Tully [mailto:gary.tully@gmail.com]
Sent: 01 June 2011 14:49
To: users@activemq.apache.org
Subject: Re: Persistence messages and StoreUsage Memory
did you get sorted?
you need to remove the store limit, replace:
<storeUsage>
<storeUsage limit="1 gb" name="foo"/>
</storeUsage>
with the following, to be limited by available disk space:
<storeUsage>
<storeUsage limit="0"/>
</storeUsage>
On 26 May 2011 11:50, Suneel Papineni <Su...@mettoni.com> wrote:
> Hi,
>
>
>
> I am using Network of brokers and using ActiveMQ5.5.0 and
> ActiveMQ-CPP3.4.0 client application.
>
>
>
> 1. Application 'A' registered to Broker1 and application 'B' is
> registered to Broker2 (Both 'A' and 'B' are registered as Durable
> Consumers). A sends messages to a topic and B receives all messages and
> vice-versa.
>
> 2. Now 'B' is disconnected for almost 1 day, but 'A' is keep on
> sending messages (around 25 messages per second and each message with a
> size of 10kb).
>
> 3. All the messages sent are "PERSISTANT" messages and having
> Time-to-Live as 5 hours.
>
> 4. I am using default configuration....
>
> <persistenceAdapter>
>
> <kahaDB directory="${activemq.base}/data/static-broker1/kahadb" />
>
> </persistenceAdapter>
>
> and
>
> <systemUsage>
>
> <systemUsage>
>
> <memoryUsage>
>
> <memoryUsage limit="100 mb"/>
>
> </memoryUsage>
>
> <storeUsage>
>
> <storeUsage limit="1 gb" name="foo"/>
>
> </storeUsage>
>
> <tempUsage>
>
> <tempUsage limit="200 mb"/>
>
> </tempUsage>
>
> </systemUsage>
>
> </systemUsage>
>
> Also I configured "producerFlowControl" property for Topics as "false"
> in order to allow 'A' to send maximum number of messages.
>
> 5. After sending many messages, 'A' got hung and Broker1 logs
> shows "Usage Manager Store is Full, 100% of 1073741824. Stopping
> producer......"
>
>
>
>
>
> My questions are:
>
> 1. What are the best values I need to configure to stop 'A'
> getting hung or in order to resolve this issue.
>
> 2. As time-to-live for messages is 5 hours, what happens to these
> messages after this timer expires. I know that it is sent to
> ActiveMQ.DLQ, but is it removed from persistence storage or
> storageUsage?
>
> 3. Is there any way I can remove the limits completely and system
> runs until system's disk memory limit (this is to allow maximum number
> of messages. Not a good ideaJ).
>
>
>
> Please let me know if I need to configure any parameters for "kahaDB".
> Thank you for your help.
>
>
>
> Thanks & Regards
>
> Suneel
>
>
>
>
--
http://fusesource.com
http://blog.garytully.com