You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by MikeCola <mc...@yahoo.com> on 2010/03/01 17:05:10 UTC
Master/Slave issue with Network of Brokers
I am running a master/slave broker on 2 servers in a shared-nothing topology
and a networked broker on a third with a duplex connection. When everything
is started up, I am able to produce messages to the master/slave and consume
messages on the networked broker with no problems. However, when I kill the
master , my consumer on the networked broker stops picking up messages from
the queue. Looking at the console messages, it appears that the networked
broker knows the master failed and connects to the slave, but I don’t know
why it won’t consume messages.
http://old.nabble.com/file/p27745457/master-activemq.xml master-activemq.xml
http://old.nabble.com/file/p27745457/slave-activemq.xml slave-activemq.xml
http://old.nabble.com/file/p27745457/remote-activemq.xml remote-activemq.xml
--
View this message in context: http://old.nabble.com/Master-Slave-issue-with-Network-of-Brokers-tp27745457p27745457.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Master/Slave issue with Network of Brokers
Posted by Roxana Grigore <ro...@yahoo.com>.
Hello Mike,
I think I found a way to make it work:
-no network connector on Master
-on the networked broker a static duplex connector to the Master
-on the Slave a static duplex connector to the networked connector
This way, when the Master is down, the Slave will start its own duplex
connector with the networked broker.
--
View this message in context: http://old.nabble.com/Master-Slave-issue-with-Network-of-Brokers-tp27745457p28094093.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Master/Slave issue with Network of Brokers
Posted by Roxana Grigore <ro...@yahoo.com>.
Hello Garry,
The problem is not with the clients using the failover transport, but on the
remote broker that is using static network connector. For brokers discovery
I cannot use failover but static and this network connector is not behaving
the same way as the failover transport which "connect to a primary first and
only connect to a secondary backup broker if the primary is unavailable".
How could I simulate this behavior with the network connector and tell the
remote broker: connect to the master, but if the master is down then connect
to slave?
--
View this message in context: http://old.nabble.com/Master-Slave-issue-with-Network-of-Brokers-tp27745457p28081882.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Master/Slave issue with Network of Brokers
Posted by pollotek <cl...@gmail.com>.
So nobody else is having a similar issue that could have some knowledge to
share? The topology I am trying is:
embedded broker -> share nothing master slave -> consumer
I also tried:
embedded broker -> share nothing master slave -> embedded broker -> consumer
and
embedded broker -> shared FS master slave -> consumer
None of them really work.
--
View this message in context: http://activemq.2283324.n4.nabble.com/Master-Slave-issue-with-Network-of-Brokers-tp2354370p4616180.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Master/Slave issue with Network of Brokers
Posted by pollotek <cl...@gmail.com>.
I'm having a very similar issue. I'm using 5.5 on Linux Ubuntu with Java 6. I
have an embedded broker inside Tomcat forwarding messages to a master slave
share nothing setup. Consumers are in a different machine running Tomcat
connected directly to the master. Everything seems to work just fine while
the master and the embedded broker are initially connected but as soon as
the master dies I get the following message:
12:07:39,755 INFO DemandForwardingBridgeSupport:538 - Stopping network
bridge on shutdown of remote broker
12:07:39,756 WARN DemandForwardingBridgeSupport:434 - Network connection
between vm://localhost#0 and tcp://127.0.0.1:61618 shutdown due to a remote
error: java.io.IOException: ShutdownInfo {commandId = 183, responseRequired
= false}
12:07:39,794 INFO DemandForwardingBridgeSupport:424 - localhost bridge to
mc01master stopped
12:07:39,796 WARN FailoverTransport:231 - Transport (/127.0.0.1:61618)
failed to tcp://127.0.0.1:61618 , attempting to automatically reconnect due
to: java.io.EOFException
12:07:39,797 INFO DiscoveryNetworkConnector:107 - Establishing network
connection from vm://localhost?async=false&network=true to
failover://(tcp://127.0.0.1:61618,tcp://127.0.0.1:61619)?randomize=false&maxReconnectAttempts=0
12:07:39,909 INFO FailoverTransport:893 - Successfully reconnected to
tcp://127.0.0.1:61619
12:07:39,977 INFO FailoverTransport:891 - Successfully connected to
tcp://127.0.0.1:61619
12:07:40,155 INFO FailoverTransport:893 - Successfully reconnected to
tcp://127.0.0.1:61619
12:07:40,163 WARN FailoverTransport:231 - Transport (/127.0.0.1:61619)
failed to tcp://127.0.0.1:61619 , attempting to automatically reconnect due
to: java.io.EOFException
12:07:40,168 INFO DemandForwardingBridgeSupport:176 - Outbound transport to
Unknown interrupted.
12:07:40,498 INFO TransportConnection:1188 - Active Slave Broker mc02slave
is attached
12:07:40,500 WARN DemandForwardingBridgeSupport:434 - Network connection
between vm://localhost#4 and tcp://127.0.0.1:61619 shutdown due to a remote
error: javax.jms.InvalidClientIDException: Broker: mc02slave - Client:
NC_localhost_outbound already connected from /10.20.64.134:58469
12:07:41,504 INFO DiscoveryNetworkConnector:107 - Establishing network
connection from vm://localhost?async=false&network=true to
failover://(tcp://127.0.0.1:61618,tcp://127.0.0.1:61619)?randomize=false&maxReconnectAttempts=0
12:07:41,693 INFO FailoverTransport:891 - Successfully connected to
tcp://127.0.0.1:61619
12:07:41,696 WARN FailoverTransport:231 - Transport (/127.0.0.1:61619)
failed to tcp://127.0.0.1:61619 , attempting to automatically reconnect due
to: java.io.EOFException
12:07:41,696 INFO DemandForwardingBridgeSupport:176 - Outbound transport to
Unknown interrupted.
12:07:41,947 INFO TransportConnection:1188 - Active Slave Broker mc02slave
is attached
The consumers don't get any more messages.
When I try to put in a new message I get:
12:09:28,014 WARN DemandForwardingBridgeSupport:807 - Unexpected local
command: SessionInfo {commandId = 2, responseRequired = false, sessionId =
ID:ubuntu-58340-1336158336548-3:2:1001}
12:09:28,015 WARN DemandForwardingBridgeSupport:807 - Unexpected local
command: ProducerInfo {commandId = 3, responseRequired = false, producerId =
ID:ubuntu-58340-1336158336548-3:2:1001:1, destination =
queue://queue.destination1, brokerPath = null, dispatchAsync = false,
windowSize = 0}
12:09:28,015 WARN DemandForwardingBridgeSupport:807 - Unexpected local
command: ActiveMQTextMessage {commandId = 4, responseRequired = false,
messageId = ID:ubuntu-58340-1336158336548-3:2:1001:1:1, originalDestination
= null, originalTransactionId = null, producerId =
ID:ubuntu-58340-1336158336548-3:2:1001:1, destination =
queue://queue.destination1, transactionId = null, expiration = 0, timestamp
= 1336158568014, arrival = 0, brokerInTime = 0, brokerOutTime = 0,
correlationId = null, replyTo = null, persistent = true, type = null,
priority = 4, groupID = null, groupSequence = 0, targetConsumerId = null,
compressed = false, userID = null, content = null, marshalledProperties =
null, dataStructure = null, redeliveryCounter = 0, size = 0, properties =
{counter=1001}, readOnlyProperties = true, readOnlyBody = true, droppable =
false, text = Test message number 1001 created at 13361585...************}
I can't really point the network connectors of the master and slave back at
the consumers because I have too many of them.
Any ideas?
--
View this message in context: http://activemq.2283324.n4.nabble.com/Master-Slave-issue-with-Network-of-Brokers-tp2354370p4609645.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Master/Slave issue with Network of Brokers
Posted by Gary Tully <ga...@gmail.com>.
Some steps to help diagnose:
1) enable debug logging for both the client and broker.
org.apache.activemq=DEBUG in the log4j.properties
You should see the client failover transport detect the connection
failure and start retrying
2) A thread dump of the client should also show threads waiting for
reconnect.
This will validate that the client is doing the expected thing.
On 30 March 2010 11:25, Roxana Grigore <ro...@yahoo.com> wrote:
>
> Hello Mike,
> I have the same problem. I tested and re-tested and it is not working. Have
> you found any solution?
>
> --
> View this message in context:
> http://old.nabble.com/Master-Slave-issue-with-Network-of-Brokers-tp27745457p28080554.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
--
http://blog.garytully.com
Open Source Integration
http://fusesource.com
Re: Master/Slave issue with Network of Brokers
Posted by Roxana Grigore <ro...@yahoo.com>.
Hello Mike,
I have the same problem. I tested and re-tested and it is not working. Have
you found any solution?
I'm currently running it in a Windows environment. The client seems to be
fine, it is the broker that's the issue.
Mike
--
View this message in context: http://old.nabble.com/Master-Slave-issue-with-Network-of-Brokers-tp27745457p28080554.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Master/Slave issue with Network of Brokers
Posted by Mike Colarusso <mc...@yahoo.com>.
I'm currently running it in a Windows environment. The client seems to be fine, it is the broker that's the issue.
Mike
________________________________
From: Josh Carlson <jc...@e-dialog.com>
To: "users@activemq.apache.org" <us...@activemq.apache.org>; MikeCola <mc...@yahoo.com>
Sent: Mon, March 1, 2010 11:22:47 AM
Subject: Re: Master/Slave issue with Network of Brokers
Is your client hung? Is your client running under Linux? I have a problem where if the actual host machine serving as the master goes down (as opposed to just the process) the client processes hang trying to read/write from the socket connection. I've yet to investigate the issue but was just curious if you were running into the same problem?
On 03/01/2010 11:05 AM, MikeCola wrote:
> I am running a master/slave broker on 2 servers in a shared-nothing topology
> and a networked broker on a third with a duplex connection. When everything
> is started up, I am able to produce messages to the master/slave and consume
> messages on the networked broker with no problems. However, when I kill the
> master , my consumer on the networked broker stops picking up messages from
> the queue. Looking at the console messages, it appears that the networked
> broker knows the master failed and connects to the slave, but I don’t know
> why it won’t consume messages.
> http://old.nabble.com/file/p27745457/master-activemq.xml master-activemq.xml
> http://old.nabble.com/file/p27745457/slave-activemq.xml slave-activemq.xml
> http://old.nabble.com/file/p27745457/remote-activemq.xml remote-activemq.xml
>
Re: Master/Slave issue with Network of Brokers
Posted by Josh Carlson <jc...@e-dialog.com>.
Is your client hung? Is your client running under Linux? I have a
problem where if the actual host machine serving as the master goes down
(as opposed to just the process) the client processes hang trying to
read/write from the socket connection. I've yet to investigate the issue
but was just curious if you were running into the same problem?
On 03/01/2010 11:05 AM, MikeCola wrote:
> I am running a master/slave broker on 2 servers in a shared-nothing topology
> and a networked broker on a third with a duplex connection. When everything
> is started up, I am able to produce messages to the master/slave and consume
> messages on the networked broker with no problems. However, when I kill the
> master , my consumer on the networked broker stops picking up messages from
> the queue. Looking at the console messages, it appears that the networked
> broker knows the master failed and connects to the slave, but I don’t know
> why it won’t consume messages.
> http://old.nabble.com/file/p27745457/master-activemq.xml master-activemq.xml
> http://old.nabble.com/file/p27745457/slave-activemq.xml slave-activemq.xml
> http://old.nabble.com/file/p27745457/remote-activemq.xml remote-activemq.xml
>