[jira] Updated: (AMQ-2596) Memory leak in network of brokers - the number of TopicSubscription/ConsumerInfo objects increases

["Mark Wuwer (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/activemq/browse/AMQ-2596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mark Wuwer updated AMQ-2596:
----------------------------

    Attachment: MemLeakSuspectNetworkOfBrokersTest.java
                fe_activemq.xml
                be_activemq.xml

Attached activemq.xms for both brokers and a set of unit tests to reproduce the behavior.

> Memory leak in network of brokers - the number of TopicSubscription/ConsumerInfo objects increases
> --------------------------------------------------------------------------------------------------
>
>                 Key: AMQ-2596
>                 URL: https://issues.apache.org/activemq/browse/AMQ-2596
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 4.1.1, 5.3.0
>         Environment: OS: SLES10 SP2 or Ubuntu 9.10 (my desktop)
> JVM: IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux x86-32 j9vmxi3223-20081129 (JIT enabled)
>            Reporter: Mark Wuwer
>            Priority: Critical
>         Attachments: be_activemq.xml, fe_activemq.xml, MemLeakSuspectNetworkOfBrokersTest.java
>
>
> I'd like to report a reproducible memory leak in network of brokers.
> This is the scenario you can reproduce it (tried out with 4.1.1, 5.3.0 and 5.3.1):
> 1. start two brokers using attached configuration.
>     Lets call one of it BE (for backend) another FE (for frontend) - accordingly: be_activemq.xml and fe_activemq.xml are attached.
>     The BE broker contains following configuration: 
>         <transportConnectors>
>             <transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
>         </transportConnectors>
>         <networkConnectors>
>            <networkConnector name="FE_connector" dynamicOnly="true" uri="static://(tcp://localhost:61617)"/>
>         </networkConnectors>
>     The FE broker contains just:
>         <transportConnectors>
>             <transportConnector name="openwire" uri="tcp://0.0.0.0:61617"/>
>         </transportConnectors>
> 2. now start a client that opens and close permanently topic subscriptions (see attached tests)
>     to the FE broker
> 3. after a while you will see (e.g. verbose gc) that the heap usage of the BE broker will increase,
>     but on FE not
> If you analyze the heap dump of BE you will see that the memory leak consist of a large number of ConsumerInfo
> objects. Easy way to find the objects when you use MAT is to use the OQL query e.g.:
>     SELECT c.destination.physicalName.toString() FROM ".*ConsumerInfo" c WHERE (c.destination.physicalName.toString() LIKE "TEST_TOPIC")
> I noticed that basically for each subscription on FE two ConsumerInfo objects are created on BE, which will never be closed.
> There are however some special cases that only 4, 8 .. 12 objects per connection are created. This happens when the client just opens sessions/subscriptions without closing  (see appropriate junit test with comments in it).
> We have it currently a P1 ticket at our customer site (running 3*FE + BE using 4.1.1) because of OoM that occurs periodically (in fact we now restart  every n-days the system). 
> Another strange issue that happens at our customer site is something that I cannot reproduce at all.
> The ConsumerInfo objects on BE contains there non empty selector strings, which makes the memory leak more critical.
> In my test environment however selector strings are only set in ConsumerInfo on FE, but in BE broker the selectorin ConsumerInfo is always null!
> I compared the customer configuration with my and I could not find any difference.
> Any idea what could influence this behavior?
> Thanks in advance for any feedback and help on this issue.
> Best Regards,
> Mark

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.