You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by ShlomiJ <sh...@gmail.com> on 2012/08/16 16:28:43 UTC
Consumers to ActiveMQ shotdown but not reconnecting, even when
failover is used
hi there,
Having a machine running an ActiveMQ (5.5.1) broker.
Another machine runs our Camel (2.9.0) context.
The connection between the 2 is set exactly as described in *Using
connection pooling* in http://camel.apache.org/activemq.html
http://camel.apache.org/activemq.html , with the following configuration:
*brokerUrl* =
failover:(tcp://BROKER:61616)?jms.useAsyncSend=true&maxReconnectDelay=1000&useExponentialBackOff=false&initialReconnectDelay=1000
*concurrentConsumers* = 200
In Camel context we define a thread pool with
poolSize=maxPoolSize=maxQueueSize=200.
The route is a simple 'from activemq' one:
<from uri="activemq:queue:MyQueue" /> <!-- no more parameters here
-->
<threads executorServiceRef="poolRef" />
... etc.
*_After a while (1 to 2 days) the consumers are gone!_*
During the life-cycle of the application we see many successful
re-connections done by the failover ("/failover.FailoverTransport -
Connection established/").
The "keep alive" I'm familiar with (in order to not have so many
re-connections) is ActiveMQ's wireFormat configuration, but we could not
configure the broker connection with both failover and wireFormat
parameters.
Then after a while, from some point on, it simply stops reconnecting.
We restart the route in order to make it reconnect again.
We might see "/Transport failed: java.net.SocketException: Connection
reset/" in the ActiveMQ's log (I'm not sure that always).
This way or the other, we still expect the consumers to reconnect.
Why does the failover stop reconnecting?
Is there a better configuration to use?
What exactly is getting disconnected? Is it something in ActiveMQ or in
Camel?
thanks for future help
SJ
--
View this message in context: http://camel.465427.n5.nabble.com/Consumers-to-ActiveMQ-shotdown-but-not-reconnecting-even-when-failover-is-used-tp5717552.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Consumers to ActiveMQ shotdown but not reconnecting, even when
failover is used
Posted by ShlomiJ <sh...@gmail.com>.
Miguel, thanks for the reply.
I added it to the configuration this way:
/failover:(tcp://BROKER:61616*?keepAlive=true*)?jms.useAsyncSend=true&maxReconnectDelay=1000&useExponentialBackOff=false&initialReconnectDelay=1000/,
and the application started.
I haven't reproduced the problem again (as I said in the opening post, this
problem happens only on production environment, not on DEV).
regards,
SJ
--
View this message in context: http://camel.465427.n5.nabble.com/Consumers-to-ActiveMQ-shotdown-but-not-reconnecting-even-when-failover-is-used-tp5717552p5717864.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Consumers to ActiveMQ shotdown but not reconnecting, even when
failover is used
Posted by Miguel Ruiz Velasco Sobrino <mi...@fcm.com.mx>.
On 08/21/2012 11:42 AM, ShlomiJ wrote:
> We noticed that *after an idle period* (around 40 minutes), during which no
> new message is put in MyQueue, *when finally a message is sent* over the
> queue, a *SocketException *is reported in the broker's log.
>
> We were able to verify this by the second, from examining the producer log,
> the broker log and the messages pending in the queue.
> We suspect it now to be a firewall issue, actually (but without evidence at
> the time).
>
> Still, any thoughts will be appreciated.
>
> SJ
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/Consumers-to-ActiveMQ-shotdown-but-not-reconnecting-even-when-failover-is-used-tp5717552p5717782.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
You can try to add a keep alive to the TCP or SSL transport in ActiveMQ.
Most firewall problems I have had in various protocols like SSH have
been solved that way.
http://activemq.apache.org/tcp-transport-reference.html
--
Miguel Ruiz Velasco Sobrino
Re: Consumers to ActiveMQ shotdown but not reconnecting, even when
failover is used
Posted by Charles Moulliard <ch...@gmail.com>.
0.0.0.0 is not a problem but a convention which is used to tell the broker
that it can use one of the IP addresses available on the ethernet port
On Wed, Aug 22, 2012 at 3:51 PM, ShlomiJ <sh...@gmail.com> wrote:
> The broker's *activemq.xml* has the transportConnector set with
> /uri="tcp://*0.0.0.0*:61616?wireFormat.maxInactivityDuration=0"/.
>
> Is 0.0.0.0 a problem?
>
> Should it be the actual IP/machine name of the broker?
>
> We are not running any other brokers.
>
> This configuration never caused problems in the past.
>
> SJ
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Consumers-to-ActiveMQ-shotdown-but-not-reconnecting-even-when-failover-is-used-tp5717552p5717854.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Charles Moulliard
Apache Committer / Sr. Pr. Consultant at FuseSource.com
Twitter : @cmoulliard
Blog : http://cmoulliard.blogspot.com
Re: Consumers to ActiveMQ shotdown but not reconnecting, even when
failover is used
Posted by ShlomiJ <sh...@gmail.com>.
And what about the http://activemq.apache.org/tcp-transport-reference.html
TCP's keepAlive parameter?
When should I use it, if I get the keep alive from the inactivity monitor?
SJ
--
View this message in context: http://camel.465427.n5.nabble.com/Consumers-to-ActiveMQ-shotdown-but-not-reconnecting-even-when-failover-is-used-tp5717552p5717916.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Consumers to ActiveMQ shotdown but not reconnecting, even when
failover is used
Posted by Charles Moulliard <ch...@gmail.com>.
Why have you disabled the activity monitoring on ActiveMQ (more info here :
http://activemq.apache.org/activemq-inactivitymonitor.html) ?
On Wed, Aug 22, 2012 at 3:51 PM, ShlomiJ <sh...@gmail.com> wrote:
> The broker's *activemq.xml* has the transportConnector set with
> /uri="tcp://*0.0.0.0*:61616?wireFormat.maxInactivityDuration=0"/.
>
> Is 0.0.0.0 a problem?
>
> Should it be the actual IP/machine name of the broker?
>
> We are not running any other brokers.
>
> This configuration never caused problems in the past.
>
> SJ
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/Consumers-to-ActiveMQ-shotdown-but-not-reconnecting-even-when-failover-is-used-tp5717552p5717854.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
--
Charles Moulliard
Apache Committer / Sr. Pr. Consultant at FuseSource.com
Twitter : @cmoulliard
Blog : http://cmoulliard.blogspot.com
Re: Consumers to ActiveMQ shotdown but not reconnecting, even when
failover is used
Posted by ShlomiJ <sh...@gmail.com>.
The broker's *activemq.xml* has the transportConnector set with
/uri="tcp://*0.0.0.0*:61616?wireFormat.maxInactivityDuration=0"/.
Is 0.0.0.0 a problem?
Should it be the actual IP/machine name of the broker?
We are not running any other brokers.
This configuration never caused problems in the past.
SJ
--
View this message in context: http://camel.465427.n5.nabble.com/Consumers-to-ActiveMQ-shotdown-but-not-reconnecting-even-when-failover-is-used-tp5717552p5717854.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Consumers to ActiveMQ shotdown but not reconnecting, even when
failover is used
Posted by ShlomiJ <sh...@gmail.com>.
We noticed that *after an idle period* (around 40 minutes), during which no
new message is put in MyQueue, *when finally a message is sent* over the
queue, a *SocketException *is reported in the broker's log.
We were able to verify this by the second, from examining the producer log,
the broker log and the messages pending in the queue.
We suspect it now to be a firewall issue, actually (but without evidence at
the time).
Still, any thoughts will be appreciated.
SJ
--
View this message in context: http://camel.465427.n5.nabble.com/Consumers-to-ActiveMQ-shotdown-but-not-reconnecting-even-when-failover-is-used-tp5717552p5717782.html
Sent from the Camel - Users mailing list archive at Nabble.com.