[jira] Resolved: (AMQ-2533) OutOfMemory provoked by Advisory messages

["Rob Davies (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/activemq/browse/AMQ-2533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rob Davies resolved AMQ-2533.
-----------------------------

       Resolution: Fixed
    Fix Version/s: 5.4.0
                   5.3.1

Fixed by SVN revision 912497on 5.3.1
and 912496 on trunk

> OutOfMemory provoked by Advisory messages
> -----------------------------------------
>
>                 Key: AMQ-2533
>                 URL: https://issues.apache.org/activemq/browse/AMQ-2533
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.2.0, 5.3.0
>            Reporter: Philippe Mouawad
>            Assignee: Rob Davies
>            Priority: Critical
>             Fix For: 5.3.1, 5.4.0
>
>         Attachments: activemq-OOM-ADVISORY.xml, activemq-Workaround-OOM-ADVISORY.xml, activemq.xml, Image 11.png, ThreadDump.txt, TopicListener.java, TopicPublisher.java
>
>
> Hello,
> We are in an environment where we have 1 Broker and 6 clients listener to one Topic.
> We enabled advisory messages through:
> <policyEntry topic=">" memoryLimit="5mb" producerFlowControl="true"  advisoryWhenFull="true" advisoryForSlowConsumers="true"
>                       sendAdvisoryIfNoConsumers="true">
> And sendFailIfNoSpace:
> <systemUsage>
>             <systemUsage sendFailIfNoSpace="true">
>                 <memoryUsage>
>                     <memoryUsage limit="50 mb"/>
>                 </memoryUsage>
>                 <storeUsage>
>                     <storeUsage limit="1 gb" name="foo" store="#store"/>
>                 </storeUsage>
>                 <tempUsage>
>                     <tempUsage limit="100 mb"/>
>                 </tempUsage>
>             </systemUsage>
>         </systemUsage>
> We also have a monitoring process that listens to advisory messages related to the concerned TOPIC.
> After some run, we end up with OutOfMemory.
> OutOfMemory analysis shows:
> - 3372 instances of org.apache.activemq.command.ActiveMQTopic occupying 23,7%
> Names of Topic contain ActiveMQ.Advisory.FULL.Topic:
> - ActiveMQ.Advisory.FULL.Topic.ActiveMQ.Advisory.FULL.Topic.ActiveMQ.Advisory.FULL.....ActiveMQ.Advisory.Producer.Topic.OFBTopic
> 1069 instances of org.apache.activemq.thread.DedicatedTaskRunner$1 holding 12% through the names of the thread that are soooooo long.
> Debuging ActiveMQ, I noticed that AMQ didn't check when sending advisory messages that advisory concerns an advisory.
> Example, suppose you set a listener for advisory message ActiveMQ.Advisory.FULL.Topic.OFBTopic and this listener becomes slow for a reason, then AMQ will send an advisory on this advisory provoking more and more traffic 
> on the broker only for this advisory.
> I attached a Stack Trace showing that an advisory will be sent for the advisory.
> I will be checking to see if this issue also exists on 5.3.0.
> Thanks in advance.
> Philippe.
> http://www.ubik-ingenierie.com

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.