Best approach for HA message broker + grid

[Manuel Teira <mt...@tid.es>]

Hello.

My aim is to have a set of brokers, running in different machines, to 
consume messages coming from different clients, to be able to scale the 
system for future needs. Furthermore, HA features are also required, to 
protect the system on hardware malfunction.

Looking at the alternatives that ActiveMQ provides to achieve these 
goals, it seems to me that I could get the grid behaviour using the 
"Network of Brokers" feature, where all the brokers are "alive".

I understand that  when more than one broker have consumers for a given 
queue, the messages are shared among them. But in my first tests, I've 
found that most of times, all the messages get consumed by one of the 
brokers. Could this be caused by a (default) high prefetch value?

Talking about the HA needs, I've read that a master-slave aproximation 
could be used, but here I won't have the grid behaviour benefits of the 
"Network of brokers" approach (as only one of the brokers is actually 
active). I would like to have a combined behaviour, where messages 
persist a hardware failure (better if restarting the failing node was 
not needed to restore them), and where messages were shared among 
consumers from different brokers.

I was thinking about using a "Network of Brokers" with a low prefetch 
value for the shared queues and disable journal files, to minimize the 
amount of messages "owned" by a given broker. As the database is located 
in a different machine, I could always restart  a new broker using the 
schema of the failing one. Is this a right approach?

Any other idea to implement this scenario?

Best regards.

Re: Best approach for HA message broker + grid

[Manuel Teira <mt...@tid.es>]

Thanks for your fast answer, James. Some doubts between the lines:


James Strachan escribió:
> On 4/3/07, Manuel Teira <mt...@tid.es> wrote:
>> Hello.
>>
>> My aim is to have a set of brokers, running in different machines, to
>> consume messages coming from different clients, to be able to scale the
>> system for future needs. Furthermore, HA features are also required, to
>> protect the system on hardware malfunction.
>>
>> Looking at the alternatives that ActiveMQ provides to achieve these
>> goals, it seems to me that I could get the grid behaviour using the
>> "Network of Brokers" feature, where all the brokers are "alive".
>
> Not quite. HA is done my Master/Slave. The "Network of Brokers" is
> purely for store and forward.
> http://activemq.apache.org/masterslave.html
>
> Multiple Master/Slave clusters would do the trick; you may want to
> connect the broker clusters together in a store and forward network
> but its not essential.
>
So, would I need four servers for the minimal topology? Two master/slave 
pairs? How would I get grid behaviour  in this environment? Using 
"Network of brokers" among the or using failover:// protocol in clients?

>
>> I understand that  when more than one broker have consumers for a given
>> queue, the messages are shared among them. But in my first tests, I've
>> found that most of times, all the messages get consumed by one of the
>> brokers. Could this be caused by a (default) high prefetch value?
>
> When using store and forward, messages only move to another broker if
> there are no local consumers.
> http://activemq.apache.org/consumer-priority.html

So, what can I do to get the message load shared among consumers in 
different brokers? Shouldn't messages be forwarded when there's not an 
idle local consumer?




>
>
>> Talking about the HA needs, I've read that a master-slave aproximation
>> could be used, but here I won't have the grid behaviour benefits of the
>> "Network of brokers" approach (as only one of the brokers is actually
>> active). I would like to have a combined behaviour, where messages
>> persist a hardware failure (better if restarting the failing node was
>> not needed to restore them), and where messages were shared among
>> consumers from different brokers.
>>
>> I was thinking about using a "Network of Brokers" with a low prefetch
>> value for the shared queues and disable journal files, to minimize the
>> amount of messages "owned" by a given broker. As the database is located
>> in a different machine, I could always restart  a new broker using the
>> schema of the failing one. Is this a right approach?
>
> With Network Of Brokers, exactly one broker owns a message at any
> point in time so you are exposed to hardware failure & there is no HA.
> If a broker goes down, its messages remain lost until it is rebooted;
> whereas with Master/Slave you get HA.
>
So, to summarize. The only way to get combined HA and Grid features is 
to have master/slave groups using 'network of brokers', or clients using 
failover:// to get messages shared through the consumers in the 
different active brokers.

Re: Best approach for HA message broker + grid

[James Strachan <ja...@gmail.com>]

On 4/3/07, Manuel Teira <mt...@tid.es> wrote:
> Hello.
>
> My aim is to have a set of brokers, running in different machines, to
> consume messages coming from different clients, to be able to scale the
> system for future needs. Furthermore, HA features are also required, to
> protect the system on hardware malfunction.
>
> Looking at the alternatives that ActiveMQ provides to achieve these
> goals, it seems to me that I could get the grid behaviour using the
> "Network of Brokers" feature, where all the brokers are "alive".

Not quite. HA is done my Master/Slave. The "Network of Brokers" is
purely for store and forward.
http://activemq.apache.org/masterslave.html

Multiple Master/Slave clusters would do the trick; you may want to
connect the broker clusters together in a store and forward network
but its not essential.


> I understand that  when more than one broker have consumers for a given
> queue, the messages are shared among them. But in my first tests, I've
> found that most of times, all the messages get consumed by one of the
> brokers. Could this be caused by a (default) high prefetch value?

When using store and forward, messages only move to another broker if
there are no local consumers.
http://activemq.apache.org/consumer-priority.html


> Talking about the HA needs, I've read that a master-slave aproximation
> could be used, but here I won't have the grid behaviour benefits of the
> "Network of brokers" approach (as only one of the brokers is actually
> active). I would like to have a combined behaviour, where messages
> persist a hardware failure (better if restarting the failing node was
> not needed to restore them), and where messages were shared among
> consumers from different brokers.
>
> I was thinking about using a "Network of Brokers" with a low prefetch
> value for the shared queues and disable journal files, to minimize the
> amount of messages "owned" by a given broker. As the database is located
> in a different machine, I could always restart  a new broker using the
> schema of the failing one. Is this a right approach?

With Network Of Brokers, exactly one broker owns a message at any
point in time so you are exposed to hardware failure & there is no HA.
If a broker goes down, its messages remain lost until it is rebooted;
whereas with Master/Slave you get HA.

-- 

James
-------
http://radio.weblogs.com/0112098/