You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by wwang2016 <wa...@impactmobile.com> on 2016/08/11 16:06:33 UTC
How to configure colocated backup servers for data replication
approach
Hi,
The colocated backup servers for shared store approach is to set up a backup
server which basically read the messages that are written by another
instance of live server.
instance #1(master.1 + slave.1)
instance #2 (master.2 + slave.2)
Where the directory of slave.1 is the same as that of master.2. This will
allow instance#1 to get unprocessed messages from instance#2 when instance#2
has shut down, and vise versa.
Normally when you set up data replication, you would set one instance as
master, and another instance as slave.
I am working on a project that needs me to set up HA for messaging with data
replication, and it looks like I am running into issues with the normal
setup.
What is the deal of setting up colocated backup servers for data replication
approach?
What are the use cases which we need to set up collocated backup?
How to set it up?
Thanks
Wayne
--
View this message in context: http://activemq.2283324.n4.nabble.com/How-to-configure-colocated-backup-servers-for-data-replication-approach-tp4715415.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: How to configure colocated backup servers for data replication
approach
Posted by wwang2016 <wa...@impactmobile.com>.
The following is the use case:
The application is currently running as one standalone server (no cluster)
in production. We do not currently have a need to make it to run wildfly in
cluster mode, but we do have concern over a sudden shutdown of the wildfly
instance and no backup. In addition, we need to do routine maintenance work
and need to take down the standalone server. It would be idea to have a
standby server to switch on during our maintenance work to reduce down time.
So the work is to set up a HA singleton scenario so that we will have one
wildfly instance running while another wildfly instance is in standby mode.
I set up the colocated shared store approach with the following
configuration:
Wildfly #1 (activemq master #1 + activemq slave #1)
Wildfly #2 (activemq master #2 + activemq slave #2)
The directory of activemq master #1 is referenced by activemq slave #2 and
activemq master #2 is referenced by activemq slave #1
Note: I set up NFS (for demo purpose) to allow both instances to access the
shared drive paths.
With the above configuration, I was able to test a scenario:
start wildfly #1, then wildfly #2. wildfly #1 becomes HA singleton provider.
Now send many messages to wildfly #1, and shut it down while it is in the
middle of processing messages. Then wildfly #2 was elected as HA singleton
provider, and started to process remainder messages. This is what we need:
we want to make sure messages will be processed by the newly elected HA
singleton provider.
I need to find out if we need to do the same thing with data replication
approach since we may not use the colocated shared-store in production.
I have not been able to make the newly elected HA singleton provider to
process the remainder messages until the previous HA singleton provider is
restarted (in standby mode). This pretty much functions as if I had not set
up the configuration at all.
Based on the observation, I think somehow the messages may not get
replicated ???
--
View this message in context: http://activemq.2283324.n4.nabble.com/How-to-configure-colocated-backup-servers-for-data-replication-approach-tp4715415p4715449.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: How to configure colocated backup servers for data replication
approach
Posted by Justin Bertram <jb...@apache.com>.
> I was trying to do similar things with data replication provider, but found
> the remaining messages were not processed by the newly elected HA singleton
> provider, and seemed to be lost. I am not sure if this is doable.
You'll have to go into more detail here for me to help further. A full description of the use-case, configuration, expected outcome, and actual outcome would be a good place to start.
For the most part the semantics of shared-store and replication are the same, but there are places where this is not true because (surprise!) the implementations are different. I'm not sure you've run into one of those places because I'm not totally clear on what exactly you're doing.
I mentioned this on the Wildfly forum thread [1] you opened about this issue, but I suppose it bears repeating. You're mixing the Wildfly application server's implementation of HA singleton with Artemis' implementation of HA, and Artemis doesn't have a notion of HA singleton. You might be better of splitting Artemis from the application server so each can be configured as necessary.
Justin
[1] https://developer.jboss.org/thread/271845
----- Original Message -----
From: "wwang2016" <wa...@impactmobile.com>
To: users@activemq.apache.org
Sent: Friday, August 12, 2016 7:45:29 AM
Subject: Re: How to configure colocated backup servers for data replication approach
Hi Justin,
I was able to test the scenario with collocated shared-store approach with
two wildfly instances which are set up as HA singleton (only one wildfly
will be active at anytime). The following is the configuration
wildfly #1 (master1 activemq + slave1 activemq)
wildfly #2 (master2 activemq + slave2 activemq)
The directories for master1 activemq are referenced by slave2 activemq and
the directories for master2 activemq are referenced by slave1 activemq.
When I start up these two wildfly instances in order, the one that was
started up earlier will be HA singleton provider, and the other will be
standby. I was able to test HA for messaging by shutting down the current HA
singleton provider while it is in the process of processing messages (the
remaining messages will be processed by the newly elected HA singleton
provider).
I was trying to do similar things with data replication provider, but found
the remaining messages were not processed by the newly elected HA singleton
provider, and seemed to be lost. I am not sure if this is doable.
Thanks,
Wayne
--
View this message in context: http://activemq.2283324.n4.nabble.com/How-to-configure-colocated-backup-servers-for-data-replication-approach-tp4715415p4715445.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: How to configure colocated backup servers for data replication
approach
Posted by wwang2016 <wa...@impactmobile.com>.
Hi Justin,
I was able to test the scenario with collocated shared-store approach with
two wildfly instances which are set up as HA singleton (only one wildfly
will be active at anytime). The following is the configuration
wildfly #1 (master1 activemq + slave1 activemq)
wildfly #2 (master2 activemq + slave2 activemq)
The directories for master1 activemq are referenced by slave2 activemq and
the directories for master2 activemq are referenced by slave1 activemq.
When I start up these two wildfly instances in order, the one that was
started up earlier will be HA singleton provider, and the other will be
standby. I was able to test HA for messaging by shutting down the current HA
singleton provider while it is in the process of processing messages (the
remaining messages will be processed by the newly elected HA singleton
provider).
I was trying to do similar things with data replication provider, but found
the remaining messages were not processed by the newly elected HA singleton
provider, and seemed to be lost. I am not sure if this is doable.
Thanks,
Wayne
--
View this message in context: http://activemq.2283324.n4.nabble.com/How-to-configure-colocated-backup-servers-for-data-replication-approach-tp4715415p4715445.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: How to configure colocated backup servers for data replication
approach
Posted by Justin Bertram <jb...@apache.com>.
I think you've misunderstood something important here.
> I was not clear of the benefits of setting up a colocated backup servers
> with data replication approach.
The benefits of colocation are the same whether you use shared-store or replication. The difference is simply how the messages get from the master to the slave.
> If I set up master activemq server and slave activemq server in the same
> wildfly instance, then the messages will be copied from master to slave?
You would never colocate a slave in the same JVM as its master as it would be useless for HA purposes. I'm not sure where you got that idea.
> But then the messages in slave activemq.2 will still not
> be accessible since it is master activemq.2 that is active.
When master activemq.1 dies then slave activemq.2 will activate so the messages are available. That's the whole point of fail-over.
Justin
----- Original Message -----
From: "wwang2016" <wa...@impactmobile.com>
To: users@activemq.apache.org
Sent: Thursday, August 11, 2016 1:07:08 PM
Subject: Re: How to configure colocated backup servers for data replication approach
Hi Justin,
I was not clear of the benefits of setting up a colocated backup servers
with data replication approach.
If I set up master activemq server and slave activemq server in the same
wildfly instance, then the messages will be copied from master to slave?
That does not seem to be a useful feature since you would like the messages
to be copied between wildfly instances to support HA for messaging.
The only scenario I can imagine is that for data replication, we set up the
following:
Wildfly #1 (master activemq.1 + slave activemq.1)
Wildfly #2 (master activemq.2 + slave activemq.2)
Where master activemq.1 pairs with slave activemq.2 and master activemq.2
pairs with slave activemq.1.
This way, if wildfly #1 is down, the slave activemq.2 will get a copy of
master activemq.1. But then the messages in slave activemq.2 will still not
be accessible since it is master activemq.2 that is active.
I can see the benefit of setting up colocated backup servers with
shared-store, but not with data replication.
Thanks,
Wayne
--
View this message in context: http://activemq.2283324.n4.nabble.com/How-to-configure-colocated-backup-servers-for-data-replication-approach-tp4715415p4715417.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: How to configure colocated backup servers for data replication
approach
Posted by wwang2016 <wa...@impactmobile.com>.
Hi Justin,
I was not clear of the benefits of setting up a colocated backup servers
with data replication approach.
If I set up master activemq server and slave activemq server in the same
wildfly instance, then the messages will be copied from master to slave?
That does not seem to be a useful feature since you would like the messages
to be copied between wildfly instances to support HA for messaging.
The only scenario I can imagine is that for data replication, we set up the
following:
Wildfly #1 (master activemq.1 + slave activemq.1)
Wildfly #2 (master activemq.2 + slave activemq.2)
Where master activemq.1 pairs with slave activemq.2 and master activemq.2
pairs with slave activemq.1.
This way, if wildfly #1 is down, the slave activemq.2 will get a copy of
master activemq.1. But then the messages in slave activemq.2 will still not
be accessible since it is master activemq.2 that is active.
I can see the benefit of setting up colocated backup servers with
shared-store, but not with data replication.
Thanks,
Wayne
--
View this message in context: http://activemq.2283324.n4.nabble.com/How-to-configure-colocated-backup-servers-for-data-replication-approach-tp4715415p4715417.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: How to configure colocated backup servers for data replication
approach
Posted by Justin Bertram <jb...@apache.com>.
> What is the deal of setting up colocated backup servers for data replication approach?
I'm not clear on what you're asking here. Can you clarify your question?
> What are the use cases which we need to set up collocated backup? How to set it up?
Colocated backups are useful when you want the performance benefits of load-balanced clustering as well as the data integrity enhancement of high-availability all using the same hardware. For example, if you had 2 physical servers and you wanted a load-balanced 2-node cluster then you would configure a clustered broker on each physical server. However, there is no high-availability or fail-over with this approach. Typically if you wanted to configure HA you'd add another 2 nodes to serve as backups for the current ones. However, using colocation the backup brokers can run in the same JVM as the live brokers..
For configuration information I suggest you consult the documentation [1] as well as the examples we ship with Artemis (specifically the "ha" examples).
Justin
[1] http://activemq.apache.org/artemis/docs/1.3.0/ha.html
----- Original Message -----
From: "wwang2016" <wa...@impactmobile.com>
To: users@activemq.apache.org
Sent: Thursday, August 11, 2016 11:06:33 AM
Subject: How to configure colocated backup servers for data replication approach
Hi,
The colocated backup servers for shared store approach is to set up a backup
server which basically read the messages that are written by another
instance of live server.
instance #1(master.1 + slave.1)
instance #2 (master.2 + slave.2)
Where the directory of slave.1 is the same as that of master.2. This will
allow instance#1 to get unprocessed messages from instance#2 when instance#2
has shut down, and vise versa.
Normally when you set up data replication, you would set one instance as
master, and another instance as slave.
I am working on a project that needs me to set up HA for messaging with data
replication, and it looks like I am running into issues with the normal
setup.
What is the deal of setting up colocated backup servers for data replication
approach?
What are the use cases which we need to set up collocated backup?
How to set it up?
Thanks
Wayne
--
View this message in context: http://activemq.2283324.n4.nabble.com/How-to-configure-colocated-backup-servers-for-data-replication-approach-tp4715415.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.