You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Priit <pr...@gmail.com> on 2013/05/14 14:41:17 UTC

Connecting to broker slows down

Hey


I have a problem with the broker itself slowing down over a few days of
uptime. Cpu and memory utilization are both ok. But establishing a
connection to the broker takes longer and longer. I'm using tcp+ssl
connector. Same issue with nio and regular tcp.

I have a setup of two brokers, both on version 5.7.0 (were previously on
5.5.0 same problem). 


Any ideas?


Regards






--
View this message in context: http://activemq.2283324.n4.nabble.com/Connecting-to-broker-slows-down-tp4667017.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Connecting to broker slows down

Posted by Priit <pr...@gmail.com>.
Seems to be a cluster problem - if i take one node down, slowdown doesn't
happen. What i also noticed that sometimes cpu usage would rise out of the
normal range (on both brokers) and would only go down
when i stopped the network bridge (or restarted one of the brokers). 

Network connector settings are:
<networkConnector
               
uri="static://(ssl://brokerb:20561)?group=${activemq.group_id}"
                dynamicOnly="true"
                networkTTL="3"
                prefetchSize="1"
                decreaseNetworkConsumerPriority="true"
/>

I'm using static connector as multicast is not allowed between the networks.
Maybe someone has ideas how to better configure network of brokers.





--
View this message in context: http://activemq.2283324.n4.nabble.com/Connecting-to-broker-slows-down-tp4667017p4667526.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Connecting to broker slows down

Posted by Priit <pr...@gmail.com>.
Brokers were slow at the time. Jvm max heap is set at 2048MB, starting the
brokers clean memory usage usually starts from 200MB and slowly starts
rising as more threads are being used. Usual thread usage will go up to
about 300-400 on brokers together. Right now broker b is running 246 threads
and using 400-500MB and broker a 132 threads and 600-900MB. Gradually memory
usage increases with both brokers over time (could be a memory leak?). When
the slowdown becomes noticeable jvm is not even pushing close to its heap
limit. The producer consumer topology is difficult to explain. We're using
an application that itself uses activemq as a backbone to relay messages to
both internal components and external client application parts.



--
View this message in context: http://activemq.2283324.n4.nabble.com/Connecting-to-broker-slows-down-tp4667017p4667070.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Connecting to broker slows down

Posted by Christian Posta <ch...@gmail.com>.
Are the brokers sluggish at the moment of those thread dumps?
You have 165 connections on broker A, 51 on broker B.
Those numbers don't seem too bad. What does the JVM memory look like?
What does the network look like between producers/consumers and broker?


On Wed, May 15, 2013 at 12:04 AM, Priit <pr...@gmail.com> wrote:

> threaddump-2013-05-08---08-27-09_brokerB.txt
> <
> http://activemq.2283324.n4.nabble.com/file/n4667042/threaddump-2013-05-08---08-27-09_brokerB.txt
> >
>
> threaddump-2013-05-08---08-27-15_brokerA.txt
> <
> http://activemq.2283324.n4.nabble.com/file/n4667042/threaddump-2013-05-08---08-27-15_brokerA.txt
> >
>
>
> Threaddumps from both brokers
>
>
> Regards
>
>
> Priit
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Connecting-to-broker-slows-down-tp4667017p4667042.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>



-- 
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta

Re: Connecting to broker slows down

Posted by Priit <pr...@gmail.com>.
threaddump-2013-05-08---08-27-09_brokerB.txt
<http://activemq.2283324.n4.nabble.com/file/n4667042/threaddump-2013-05-08---08-27-09_brokerB.txt>  

threaddump-2013-05-08---08-27-15_brokerA.txt
<http://activemq.2283324.n4.nabble.com/file/n4667042/threaddump-2013-05-08---08-27-15_brokerA.txt>  


Threaddumps from both brokers


Regards


Priit



--
View this message in context: http://activemq.2283324.n4.nabble.com/Connecting-to-broker-slows-down-tp4667017p4667042.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Connecting to broker slows down

Posted by Gary Tully <ga...@gmail.com>.
maybe post a thread dump of the broker. possibly if connections are
not pooled and are never released you could see this but that would be
reflected in fds and threads in use (so would appear as a memory
leak).

On 14 May 2013 13:41, Priit <pr...@gmail.com> wrote:
> Hey
>
>
> I have a problem with the broker itself slowing down over a few days of
> uptime. Cpu and memory utilization are both ok. But establishing a
> connection to the broker takes longer and longer. I'm using tcp+ssl
> connector. Same issue with nio and regular tcp.
>
> I have a setup of two brokers, both on version 5.7.0 (were previously on
> 5.5.0 same problem).
>
>
> Any ideas?
>
>
> Regards
>
>
>
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/Connecting-to-broker-slows-down-tp4667017.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.



-- 
http://redhat.com
http://blog.garytully.com