[jira] Reopened: (AMQ-2448) Memory leak when there are network connector retries to an unavailable broker

["Sebastien Rodriguez (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/activemq/browse/AMQ-2448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sebastien Rodriguez reopened AMQ-2448:
--------------------------------------

    Regression: [Regression]

Hi,
I'm encountering the exact same behavior in both 5.3 branch and development trunk.
I created a simple network of brokers with all the same configuration:
<networkConnector name="HA Queue" uri="static:(tcp://broker1:61616,tcp://broker2:61616,tcp://broker3:61616)?useExponentialBackOff=false" userName="system" password="manager" duplex="true" />
The disables exponential back off allows me to reproduce the memory leak faster.
I stop activemq in 2 out of the 3 brokers and get a memory leak: there is nearly one thread created per second. After a while, the activeMQ will just crash.

Here is the start of the dump heat, after only 20min of running:

--start--
org.apache.activemq.command.ActiveMQDestination[]	5003
org.apache.activemq.transport.ResponseCorrelator	3750
org.apache.activemq.util.IntSequenceGenerator 3750
org.apache.activemq.transport.MutexTransport 3750
org.apache.activemq.management.CountStatisticImpl 2548
org.apache.activemq.command.DataStructure[] 2512
org.apache.activemq.thread.Valve 2504
org.apache.activemq.command.BrokerId[] 2500
org.apache.activemq.util.DataByteArrayOutputStream 1269
org.apache.activemq.util.DataByteArrayInputStream 1266
org.apache.activemq.thread.DedicatedTaskRunner$1 1259
org.apache.activemq.thread.DedicatedTaskRunner 1259
org.apache.activemq.openwire.OpenWireFormat 1256
org.apache.activemq.util.LongSequenceGenerator 1254
org.apache.activemq.util.IdGenerator 1253
org.apache.activemq.command.WireFormatInfo 1251
org.apache.activemq.broker.TransportConnection 1250
org.apache.activemq.transport.WireFormatNegotiator 1250
org.apache.activemq.transport.InactivityMonitor$2 1250
org.apache.activemq.transport.vm.VMTransportServer$1 1250
org.apache.activemq.broker.TransportConnection$1 1250
org.apache.activemq.transport.InactivityMonitor$1 1250
org.apache.activemq.network.DemandForwardingBridgeSupport$2 1250
org.apache.activemq.transport.InactivityMonitor 1250
org.apache.activemq.transport.tcp.TcpTransport	 1250
org.apache.activemq.broker.region.ConnectionStatistics 1250
org.apache.activemq.network.DurableConduitBridge 1250
org.apache.activemq.network.DiscoveryNetworkConnector$1 1250
org.apache.activemq.network.DemandForwardingBridgeSupport$1 1250
org.apache.activemq.transport.vm.VMTransport 1250
org.apache.activemq.broker.SingleTransportConnectionStateRegister	 1249
org.apache.log4j.CategoryKey 194



> Memory leak when there are network connector retries to an unavailable broker
> -----------------------------------------------------------------------------
>
>                 Key: AMQ-2448
>                 URL: https://issues.apache.org/activemq/browse/AMQ-2448
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.3.0
>         Environment: Debian Linux 5.0 amd64 with included Sun Java 6 JRE
>            Reporter: Alexander Fisher
>            Assignee: Gary Tully
>             Fix For: 5.3.1, 5.4.0
>
>         Attachments: activemq1.xml, activemq2.xml, activemq3.xml, activemq4.xml
>
>
> I've discovered a reproducible memory leak.  After discussion on IRC, it is believed to be related to my network of brokers configuration.
> I have a total of 4 brokers configured on 4 hosts (activemq1,2,3,4).
> activemq1 and activemq2 are a shared filesystem master slave pair.  activemq3 and activemq4 are setup in a similar way as a master/slave pair.
> Only one of activemq1 and 2 will be active at a time, likewise only one of activemq3 and 4.
> Both activemq1 and activemq2 have a config with the following networkConnector.
> <networkConnectors>
>                 <networkConnector
>                         uri="static://(tcp://activemq3:61616,tcp://activemq4:61616)"
>                         name="Connection to 3 and 4"
>                         networkTTL="5"
>                         dynamicOnly="true"/>
> </networkConnectors>
> The broker will only be able to connect to either 3 or 4 as only one can be running at a time.
> For obvious reasons, connecting to the slave will fail, but the connection attempt will be retried every 30 seconds by default (more often on initial startup due to backoff algorithm).
> It is believed that the continuous reconnect attempts are the source of the memory leak.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.