You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Thomas Swindells <ts...@ndsuk.com> on 2006/06/07 15:43:38 UTC
HA system design
Hi, I'm new to JMS and ActiveMQ and need to develope a high available system.
I need a system without a single point of failure so I think I need multiple
brokers. The system consists of multiple clients primarily subscribing to
topics. If one of the broker goes down the subscribers need to be able to
pick up from the other broker without loosing messages. Once the first
broker has come back again it needs to keep functioning, again without
loosing messages, if the second broker subsequentely dies.
>From what I have been reading the Master-Slave connection only supports a
single failure occouring and then requires both brokers to be restarted, is
this correct?
If this is the case please could someone suggest to me what then is the best
configuration in order to create a rebust high availability system?
Are there any advantages/disadvantages to having embeded brokers within each
client which then connect to the main broker system or should the client
just connect directly to the main broker system?
Thanks
Thomas
--
View this message in context: http://www.nabble.com/HA-system-design-t1747970.html#a4751497
Sent from the ActiveMQ - User forum at Nabble.com.
Re: HA system design
Posted by James Strachan <ja...@gmail.com>.
On 6/8/06, Thomas Swindells <ts...@ndsuk.com> wrote:
>
> >Re: HA system design James.Strachan 2006-06-08 12:26
> >On 6/7/06, Thomas Swindells <ts...@...> wrote:
> >>
> >> Hi, I'm new to JMS and ActiveMQ
>
> >Welcome!
> Thank you
> >> and need to develope a high available system.
> ...
> >> From what I have been reading the Master-Slave connection only supports a
> >> single failure occouring and then requires both brokers to be restarted,
> >> is
> >> this correct?
> >Yes
>
> >> If this is the case please could someone suggest to me what then is the
> >> best
> >> configuration in order to create a rebust high availability system?
> >>
> >> Are there any advantages/disadvantages to having embeded brokers within
> >> each
> >> client which then connect to the main broker system or should the client
> >> just connect directly to the main broker system?
>
> >So i'd recommend 2 approaches; it kinda depends on your requirements
> >really which is best. The easiest option to go with is Master/Slave.
> >The issue is if you loose a broker its currently a manual process to
> >bring it back online again since we don't yet have an automatic
> >old-master <-> new-master synchronization protocol.
> Unfortunately this isn't an option, the system needs to be HA meaning a full
> failover/failback solution.
> Is any work being done on creating the necessary synchronization?
Not yet - hopefully in the future someone will build this.
> >The other option; if you're using topics and need really high
> >performance you could consider using subscription recovery...
> >http://incubator.apache.org/activemq/subscription-recovery-policy.html
> >which basically means if a broker dies, you connect to another broker
> >(in a store/forward network so messages on a topic get replicated to
> >all available consumers). However by default you'd miss some messages
> >during the time between reconnecting & resubscribing, so subscription
> >recovery policy allows you to configure a buffer in each broker (say 1
> >minutes worth) of messages kept around in RAM so that they can be
> >replayed to any new retroactive consumers that reconnect.
>
> >The latter option is crazy fast and does not require any persistence,
> >the former option is the solution you should go with if you need
> >persistence or use queues.
> The later option sounds closer to the solution I need however I do also need
> to use some queues in the system. Are there any other solutions which allow
> full failover/failback support which do also support queues or is it going
> to be a comprimise between one or the other?
For queues the only real option is master/slave as we have to
replicate things to 1-N brokers.
So it sounds like you need a broker-broker reconciliation capability
so you can automatically bring back online any failed masters.
--
James
-------
http://radio.weblogs.com/0112098/
Re: HA system design
Posted by Thomas Swindells <ts...@ndsuk.com>.
>Re: HA system design James.Strachan 2006-06-08 12:26
>On 6/7/06, Thomas Swindells <ts...@...> wrote:
>>
>> Hi, I'm new to JMS and ActiveMQ
>Welcome!
Thank you
>> and need to develope a high available system.
...
>> From what I have been reading the Master-Slave connection only supports a
>> single failure occouring and then requires both brokers to be restarted,
>> is
>> this correct?
>Yes
>> If this is the case please could someone suggest to me what then is the
>> best
>> configuration in order to create a rebust high availability system?
>>
>> Are there any advantages/disadvantages to having embeded brokers within
>> each
>> client which then connect to the main broker system or should the client
>> just connect directly to the main broker system?
>So i'd recommend 2 approaches; it kinda depends on your requirements
>really which is best. The easiest option to go with is Master/Slave.
>The issue is if you loose a broker its currently a manual process to
>bring it back online again since we don't yet have an automatic
>old-master <-> new-master synchronization protocol.
Unfortunately this isn't an option, the system needs to be HA meaning a full
failover/failback solution.
Is any work being done on creating the necessary synchronization?
>The other option; if you're using topics and need really high
>performance you could consider using subscription recovery...
>http://incubator.apache.org/activemq/subscription-recovery-policy.html
>which basically means if a broker dies, you connect to another broker
>(in a store/forward network so messages on a topic get replicated to
>all available consumers). However by default you'd miss some messages
>during the time between reconnecting & resubscribing, so subscription
>recovery policy allows you to configure a buffer in each broker (say 1
>minutes worth) of messages kept around in RAM so that they can be
>replayed to any new retroactive consumers that reconnect.
>The latter option is crazy fast and does not require any persistence,
>the former option is the solution you should go with if you need
>persistence or use queues.
The later option sounds closer to the solution I need however I do also need
to use some queues in the system. Are there any other solutions which allow
full failover/failback support which do also support queues or is it going
to be a comprimise between one or the other?
Thanks,
Thomas
--
View this message in context: http://www.nabble.com/HA-system-design-t1747970.html#a4773646
Sent from the ActiveMQ - User forum at Nabble.com.
Re: HA system design
Posted by James Strachan <ja...@gmail.com>.
On 6/7/06, Thomas Swindells <ts...@ndsuk.com> wrote:
>
> Hi, I'm new to JMS and ActiveMQ
Welcome!
> and need to develope a high available system.
Cool. BTW before we start there are quite a few different ways of
doing things & we've tried to make ActiveMQ flexible enough with
dealing with most things...
> I need a system without a single point of failure so I think I need multiple
> brokers. The system consists of multiple clients primarily subscribing to
> topics. If one of the broker goes down the subscribers need to be able to
> pick up from the other broker without loosing messages. Once the first
> broker has come back again it needs to keep functioning, again without
> loosing messages, if the second broker subsequentely dies.
>
> From what I have been reading the Master-Slave connection only supports a
> single failure occouring and then requires both brokers to be restarted, is
> this correct?
Yes
> If this is the case please could someone suggest to me what then is the best
> configuration in order to create a rebust high availability system?
>
> Are there any advantages/disadvantages to having embeded brokers within each
> client which then connect to the main broker system or should the client
> just connect directly to the main broker system?
So i'd recommend 2 approaches; it kinda depends on your requirements
really which is best. The easiest option to go with is Master/Slave.
The issue is if you loose a broker its currently a manual process to
bring it back online again since we don't yet have an automatic
old-master <-> new-master synchronization protocol.
The other option; if you're using topics and need really high
performance you could consider using subscription recovery...
http://incubator.apache.org/activemq/subscription-recovery-policy.html
which basically means if a broker dies, you connect to another broker
(in a store/forward network so messages on a topic get replicated to
all available consumers). However by default you'd miss some messages
during the time between reconnecting & resubscribing, so subscription
recovery policy allows you to configure a buffer in each broker (say 1
minutes worth) of messages kept around in RAM so that they can be
replayed to any new retroactive consumers that reconnect.
The latter option is crazy fast and does not require any persistence,
the former option is the solution you should go with if you need
persistence or use queues.
--
James
-------
http://radio.weblogs.com/0112098/