Master/Slave mode - slave never taking over

[RichStephens <rs...@sumglobal.com>]

We are currently using ActiveMQ 5.3.1 in a "pure Master/Slave" configuration. 
We have one broker, set up as "producer", that receives messages from java
and C++ clients.  This broker forwards all messages to another broker,
called "consumer", which is set up as the master in a pure "Master/Slave"
configuration.  We have java clients connected both the "consumer" and the
"slave" broker to pull those messages, so we don't have a situation of the
brokers getting "filled up", as it were.

What we are seeing is this:  The "producer" is forwarding messages to the
"consumer" broker, and the "slave" broker is properly replicating those
messages that its master, the "consumer" broker is receiving.  However, if
we shut down the "master", the "slave" should finish replicating all the
messages it got from the master "consumer" broker, and then take over to
"become" the master.  What we are seeing is that once the last messages from
the "consumer" broker have been replicated, the "slave" broker never
actually takes over and begins processing messages.  The flow of messages
from client->producer broker->slave broker just STOPS.

The configuration file on the slave broker has the following setting:

 <broker xmlns="http://activemq.apache.org/schema/core" brokerName="slave"
dataDirectory="${activemq.base}/data"
destroyApplicationContextOnStop="true">
        <services>
            <masterConnector remoteURI= "tcp://67.23.5.229:61616"
userName="system" password="manager"/>
       </services>

We have tried different configuration settings on the master "consumer"
broker, like these:

 <networkConnector name="producer"
uri="static:(failover:(tcp://67.23.5.229:61616,tcp://67.23.8.252:61616)?initialReconnectDelay=100)">

 <networkConnector name="producer"
uri="static:(failover:(ssl://67.23.5.229:61616,tcp://67.23.8.252:61616)?initialReconnectDelay=1000)"
userName="system" password="manager">

These are all 3 separate Centos Linux 5.4 instances, from a "cloud" server
provider (rackspace.com)

The system performs as expected until the master "consumer" broker actually
goes down and the "slave" takeover is supposed to happen.

We really need to get this working, and are so far stumped.  Any thoughts or
ideas would be welcome.

Thanks,
Rich Stephens


-- 
View this message in context: http://old.nabble.com/Master-Slave-mode---slave-never-taking-over-tp28437108p28437108.html
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.

Re: Master/Slave mode - slave never taking over

[Bruce Snyder <br...@gmail.com>]

On Mon, May 3, 2010 at 10:18 AM, RichStephens <rs...@sumglobal.com> wrote:
>
> We are currently using ActiveMQ 5.3.1 in a "pure Master/Slave" configuration.
> We have one broker, set up as "producer", that receives messages from java
> and C++ clients.  This broker forwards all messages to another broker,
> called "consumer", which is set up as the master in a pure "Master/Slave"
> configuration.  We have java clients connected both the "consumer" and the
> "slave" broker to pull those messages, so we don't have a situation of the
> brokers getting "filled up", as it were.
>
> What we are seeing is this:  The "producer" is forwarding messages to the
> "consumer" broker, and the "slave" broker is properly replicating those
> messages that its master, the "consumer" broker is receiving.  However, if
> we shut down the "master", the "slave" should finish replicating all the
> messages it got from the master "consumer" broker, and then take over to
> "become" the master.  What we are seeing is that once the last messages from
> the "consumer" broker have been replicated, the "slave" broker never
> actually takes over and begins processing messages.  The flow of messages
> from client->producer broker->slave broker just STOPS.
>
> The configuration file on the slave broker has the following setting:
>
>  <broker xmlns="http://activemq.apache.org/schema/core" brokerName="slave"
> dataDirectory="${activemq.base}/data"
> destroyApplicationContextOnStop="true">
>        <services>
>            <masterConnector remoteURI= "tcp://67.23.5.229:61616"
> userName="system" password="manager"/>
>       </services>
>
> We have tried different configuration settings on the master "consumer"
> broker, like these:
>
>  <networkConnector name="producer"
> uri="static:(failover:(tcp://67.23.5.229:61616,tcp://67.23.8.252:61616)?initialReconnectDelay=100)">
>
>  <networkConnector name="producer"
> uri="static:(failover:(ssl://67.23.5.229:61616,tcp://67.23.8.252:61616)?initialReconnectDelay=1000)"
> userName="system" password="manager">
>
> These are all 3 separate Centos Linux 5.4 instances, from a "cloud" server
> provider (rackspace.com)
>
> The system performs as expected until the master "consumer" broker actually
> goes down and the "slave" takeover is supposed to happen.
>
> We really need to get this working, and are so far stumped.  Any thoughts or
> ideas would be welcome.

In testing out this type of setup, I have determined that the pure
master/slave functionality in 5.3.1 is broken. Drop back to 5.3.0 and
it works just fine. I will create a JIRA issue to get this fixed in a
future release.

Bruce
-- 
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'

ActiveMQ in Action: http://bit.ly/2je6cQ
Blog: http://bruceblog.org/
Twitter: http://twitter.com/brucesnyder