You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by lucious <ma...@arise.pl> on 2010/03/05 16:01:12 UTC
Re: AMQ (5.3.0) with CMS-3.01 -> consumes all memory and stops
clients and producers
Hi Mats,
Doesn't work neither :(
1) running AMQ version SNAPSHOT from march 03
2) config file as follows:
<broker xmlns="http://activemq.apache.org/schema/core"
brokerName="localhost" useJmx="true" dataDirectory="${activemq.base}/data">
<managementContext>
<managementContext createConnector="false"/>
</managementContext>
<persistenceAdapter>
<kahaDB directory="${activemq.base}/data/kahadb"/>
</persistenceAdapter>
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" producerFlowControl="false"
memoryLimit="100mb">
<dispatchPolicy>
<strictOrderDispatchPolicy />
</dispatchPolicy>
<pendingSubscriberPolicy>
<vmCursor />
</pendingSubscriberPolicy>
<pendingDurableSubscriberPolicy>
<vmDurableCursor/>
</pendingDurableSubscriberPolicy>
</policyEntry>
<policyEntry queue=">" producerFlowControl="false"
memoryLimit="100mb" optimizedDispatch="true">
<dispatchPolicy>
<strictOrderDispatchPolicy />
</dispatchPolicy>
<pendingQueuePolicy>
<vmQueueCursor/>
</pendingQueuePolicy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage limit="1024 mb"/>
</memoryUsage>
<storeUsage>
<storeUsage limit="15 gb" name="store"/>
</storeUsage>
<!--
<tempUsage>
<tempUsage limit="10 gb"/>
</tempUsage>
-->
</systemUsage>
</systemUsage>
<transportConnectors>
<transportConnector name="openwire"
uri="tcp://0.0.0.0:61616?wireFormat.maxInactivityDuration=0"/>
</transportConnectors>
</broker>
3) this was in log when MemoryPrecentageUsage reached 100% :
2010-03-05 15:35:56,090 | INFO | Usage Manager Memory Limit reached on
temp-queue://e84351f4-d345-2d5c-44e8-35dc2523aea4:2. Producers will be
throttled to the rate at which messages are removed from this destination to
prevent flooding it. See
http://activemq.apache.org/producer-flow-control.html for more info |
org.apache.activemq.broker.region.Queue | ActiveMQ Transport:
tcp:///127.0.0.1:57096
again temp-queue
Maybe I should remove VMCursor or .... I really don't know what to do with
this ... reopen the same BUG?
Thanks,
Marcin
Mats Henrikson wrote:
>
> Hi Marcin,
>
> This looks very similar to a bug that I have logged:
>
> https://issues.apache.org/activemq/browse/AMQ-2616
>
> Supposed to be fixed in 5.3.1 and later.
>
> Mats
>
>
>
>
> On 4 March 2010 06:50, lucious <ma...@arise.pl> wrote:
>>
>> Hello here is what we have:
>>
>> 1) AMQ -5 3.0 with the following configuration:
>> <broker xmlns="http://activemq.apache.org/schema/core"
>> brokerName="localhost" useJmx="true"
>> dataDirectory="${activemq.base}/data">
>>
>> <managementContext>
>> <managementContext createConnector="false"/>
>> </managementContext>
>> <persistenceAdapter>
>> <kahaDB directory="${activemq.base}/data/kahadb"/>
>> </persistenceAdapter>
>> <destinationPolicy>
>> <policyMap>
>> <policyEntries>
>> <policyEntry topic=">" producerFlowControl="false"
>> memoryLimit="100mb">
>> <dispatchPolicy>
>> <strictOrderDispatchPolicy />
>> </dispatchPolicy>
>> <pendingSubscriberPolicy>
>> <vmCursor />
>> </pendingSubscriberPolicy>
>> <pendingDurableSubscriberPolicy>
>> <vmDurableCursor/>
>> </pendingDurableSubscriberPolicy>
>> </policyEntry>
>> <policyEntry queue=">" producerFlowControl="false"
>> memoryLimit="100mb">
>> <dispatchPolicy>
>> <strictOrderDispatchPolicy />
>> </dispatchPolicy>
>> <pendingQueuePolicy>
>> <vmQueueCursor/>
>> </pendingQueuePolicy>
>> </policyEntry>
>> </policyEntries>
>> </policyMap>
>> </destinationPolicy>
>>
>> <systemUsage>
>> <systemUsage>
>> <memoryUsage>
>> <memoryUsage limit="1024 mb"/>
>> </memoryUsage>
>> <storeUsage>
>> <storeUsage limit="10 gb" name="foo"/>
>> </storeUsage>
>> <tempUsage>
>> <tempUsage limit="10 gb"/>
>> </tempUsage>
>> </systemUsage>
>> </systemUsage>
>>
>> <transportConnectors>
>> <transportConnector name="openwire"
>> uri="tcp://0.0.0.0:61616?wireFormat.maxInactivityDuration=0"/>
>> </transportConnectors>
>>
>> System is being run with Xm=2048M
>>
>> 2) Producers and clients use CMS - 3.0.1 - we don't have any features
>> added
>> to CreateFactory - all defaults are used.
>>
>> Results:
>> 1) during the day watch MQ via JMX I have: totalEnqueueCount=1193470,
>> totaldequeuecount=628130,totalmessagecount=1052834 - this I think is
>> wrong
>> ... shouldn't be like that: totalmessage+totaldequeue = totalenqueue?
>>
>> 2) memorypercentusage=73 and it's counting ... once it gets to 100% all
>> is
>> frozen and in log file I get messages like this: INFO | Usage Manager
>> memory
>> limit reached on temp-queue://42455840-0bb5-0101-49b1-304ecf24dbbd:2.
>> Producers will be throttled to the rate at which messages are removed
>> from
>> this destination to prevent flooding it. See
>> http://activemq.apache.org/producer-flow-control.html for more info
>>
>> quite strange as flow-control is disabled.
>>
>> 3) watching Heap memory Usage I can see "saw chart" but it's going up and
>> up
>> to 1GB and then is saturated (AMQ is frozen at that time).
>>
>> 4) the only thing I can do is to stop AMQ and restart ... however on
>> production is not a solution.
>>
>> Maybe I have something wrong with configuration file? I'm also a little
>> bit
>> confused about memoryLimit in topic/queue policy and memoryUsage. What
>> exactly these options do? I thought that if memoryUsage is 1GB and
>> percentUsage is 70% all messages are dropped into "file" and memory is
>> clean. Maybe I mixed this all up and it's not as I think it's?
>>
>> Please help,
>> thx,
>> M.
>>
>> --
>> View this message in context:
>> http://old.nabble.com/AMQ-%285.3.0%29-with-CMS-3.01--%3E-consumes-all-memory-and-stops-clients-and-producers-tp27771356p27771356.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>
>
>
--
View this message in context: http://old.nabble.com/AMQ-%285.3.0%29-with-CMS-3.01--%3E-consumes-all-memory-and-stops-clients-and-producers-tp27771356p27795010.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: AMQ (5.3.0) with CMS-3.01 -> consumes all memory and stops
clients and producers
Posted by lucious <ma...@arise.pl>.
BTW:
TotalEqueueCount: 1915k
TotalDequeueCount: 984k
TotalMessageCount: 380k
Perhaps this can lead to "solution"
M.
lucious wrote:
>
> Hi Mats,
>
> Doesn't work neither :(
>
> 1) running AMQ version SNAPSHOT from march 03
> 2) config file as follows:
> <broker xmlns="http://activemq.apache.org/schema/core"
> brokerName="localhost" useJmx="true"
> dataDirectory="${activemq.base}/data">
>
> <managementContext>
> <managementContext createConnector="false"/>
> </managementContext>
>
> <persistenceAdapter>
> <kahaDB directory="${activemq.base}/data/kahadb"/>
> </persistenceAdapter>
>
> <destinationPolicy>
> <policyMap>
> <policyEntries>
> <policyEntry topic=">" producerFlowControl="false"
> memoryLimit="100mb">
>
> <dispatchPolicy>
> <strictOrderDispatchPolicy />
> </dispatchPolicy>
>
> <pendingSubscriberPolicy>
> <vmCursor />
> </pendingSubscriberPolicy>
>
> <pendingDurableSubscriberPolicy>
> <vmDurableCursor/>
> </pendingDurableSubscriberPolicy>
>
> </policyEntry>
> <policyEntry queue=">" producerFlowControl="false"
> memoryLimit="100mb" optimizedDispatch="true">
>
> <dispatchPolicy>
> <strictOrderDispatchPolicy />
> </dispatchPolicy>
>
> <pendingQueuePolicy>
> <vmQueueCursor/>
> </pendingQueuePolicy>
>
>
> </policyEntry>
> </policyEntries>
> </policyMap>
> </destinationPolicy>
>
> <systemUsage>
> <systemUsage>
> <memoryUsage>
> <memoryUsage limit="1024 mb"/>
> </memoryUsage>
> <storeUsage>
> <storeUsage limit="15 gb" name="store"/>
> </storeUsage>
> <!--
> <tempUsage>
> <tempUsage limit="10 gb"/>
> </tempUsage>
> -->
> </systemUsage>
> </systemUsage>
> <transportConnectors>
> <transportConnector name="openwire"
> uri="tcp://0.0.0.0:61616?wireFormat.maxInactivityDuration=0"/>
> </transportConnectors>
>
> </broker>
>
> 3) this was in log when MemoryPrecentageUsage reached 100% :
> 2010-03-05 15:35:56,090 | INFO | Usage Manager Memory Limit reached on
> temp-queue://e84351f4-d345-2d5c-44e8-35dc2523aea4:2. Producers will be
> throttled to the rate at which messages are removed from this destination
> to prevent flooding it. See
> http://activemq.apache.org/producer-flow-control.html for more info |
> org.apache.activemq.broker.region.Queue | ActiveMQ Transport:
> tcp:///127.0.0.1:57096
>
> again temp-queue
>
>
> Maybe I should remove VMCursor or .... I really don't know what to do with
> this ... reopen the same BUG?
>
> Thanks,
> Marcin
>
>
>
> Mats Henrikson wrote:
>>
>> Hi Marcin,
>>
>> This looks very similar to a bug that I have logged:
>>
>> https://issues.apache.org/activemq/browse/AMQ-2616
>>
>> Supposed to be fixed in 5.3.1 and later.
>>
>> Mats
>>
>>
>>
>>
>> On 4 March 2010 06:50, lucious <ma...@arise.pl> wrote:
>>>
>>> Hello here is what we have:
>>>
>>> 1) AMQ -5 3.0 with the following configuration:
>>> <broker xmlns="http://activemq.apache.org/schema/core"
>>> brokerName="localhost" useJmx="true"
>>> dataDirectory="${activemq.base}/data">
>>>
>>> <managementContext>
>>> <managementContext createConnector="false"/>
>>> </managementContext>
>>> <persistenceAdapter>
>>> <kahaDB directory="${activemq.base}/data/kahadb"/>
>>> </persistenceAdapter>
>>> <destinationPolicy>
>>> <policyMap>
>>> <policyEntries>
>>> <policyEntry topic=">" producerFlowControl="false"
>>> memoryLimit="100mb">
>>> <dispatchPolicy>
>>> <strictOrderDispatchPolicy />
>>> </dispatchPolicy>
>>> <pendingSubscriberPolicy>
>>> <vmCursor />
>>> </pendingSubscriberPolicy>
>>> <pendingDurableSubscriberPolicy>
>>> <vmDurableCursor/>
>>> </pendingDurableSubscriberPolicy>
>>> </policyEntry>
>>> <policyEntry queue=">" producerFlowControl="false"
>>> memoryLimit="100mb">
>>> <dispatchPolicy>
>>> <strictOrderDispatchPolicy />
>>> </dispatchPolicy>
>>> <pendingQueuePolicy>
>>> <vmQueueCursor/>
>>> </pendingQueuePolicy>
>>> </policyEntry>
>>> </policyEntries>
>>> </policyMap>
>>> </destinationPolicy>
>>>
>>> <systemUsage>
>>> <systemUsage>
>>> <memoryUsage>
>>> <memoryUsage limit="1024 mb"/>
>>> </memoryUsage>
>>> <storeUsage>
>>> <storeUsage limit="10 gb" name="foo"/>
>>> </storeUsage>
>>> <tempUsage>
>>> <tempUsage limit="10 gb"/>
>>> </tempUsage>
>>> </systemUsage>
>>> </systemUsage>
>>>
>>> <transportConnectors>
>>> <transportConnector name="openwire"
>>> uri="tcp://0.0.0.0:61616?wireFormat.maxInactivityDuration=0"/>
>>> </transportConnectors>
>>>
>>> System is being run with Xm=2048M
>>>
>>> 2) Producers and clients use CMS - 3.0.1 - we don't have any features
>>> added
>>> to CreateFactory - all defaults are used.
>>>
>>> Results:
>>> 1) during the day watch MQ via JMX I have: totalEnqueueCount=1193470,
>>> totaldequeuecount=628130,totalmessagecount=1052834 - this I think is
>>> wrong
>>> ... shouldn't be like that: totalmessage+totaldequeue = totalenqueue?
>>>
>>> 2) memorypercentusage=73 and it's counting ... once it gets to 100% all
>>> is
>>> frozen and in log file I get messages like this: INFO | Usage Manager
>>> memory
>>> limit reached on temp-queue://42455840-0bb5-0101-49b1-304ecf24dbbd:2.
>>> Producers will be throttled to the rate at which messages are removed
>>> from
>>> this destination to prevent flooding it. See
>>> http://activemq.apache.org/producer-flow-control.html for more info
>>>
>>> quite strange as flow-control is disabled.
>>>
>>> 3) watching Heap memory Usage I can see "saw chart" but it's going up
>>> and up
>>> to 1GB and then is saturated (AMQ is frozen at that time).
>>>
>>> 4) the only thing I can do is to stop AMQ and restart ... however on
>>> production is not a solution.
>>>
>>> Maybe I have something wrong with configuration file? I'm also a little
>>> bit
>>> confused about memoryLimit in topic/queue policy and memoryUsage. What
>>> exactly these options do? I thought that if memoryUsage is 1GB and
>>> percentUsage is 70% all messages are dropped into "file" and memory is
>>> clean. Maybe I mixed this all up and it's not as I think it's?
>>>
>>> Please help,
>>> thx,
>>> M.
>>>
>>> --
>>> View this message in context:
>>> http://old.nabble.com/AMQ-%285.3.0%29-with-CMS-3.01--%3E-consumes-all-memory-and-stops-clients-and-producers-tp27771356p27771356.html
>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>>
>>>
>>
>>
>
>
--
View this message in context: http://old.nabble.com/AMQ-%285.3.0%29-with-CMS-3.01--%3E-consumes-all-memory-and-stops-clients-and-producers-tp27771356p27795183.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.