clustering options for HA and horizontal scalability

[boday <bo...@vektrel.com>]

I'm trying to determine my options for clustering my AMQ 5.3 application that
uses ServiceMix 3.3.1 and Camel 2.1.  I'm performing high volume message
processing and I need to cluster for high availability and horizontal
scalability.

Here is basically what my application does (using Apache
Camel)...HTTP->QUEUE->PROCESS->DATABASE->TOPIC

So, I've read all the ActiveMQ and ServiceMix clustering pages, but am still
not sure what to do.  

I know I can use a Master/Slave setup for HA, but that doesn't help with
scalability.

I've read about network of brokers, but am not sure how this applies to my
application.  For example, if I deploy identical Camel components on
multiple nodes in a cluster, how will the nodes "interact" exactly?  If I
point my HTTP producer at one node (NodeA), which messages will get sent to
NodeB?  Will the queues/topics be shared between Node A/B...if so how, are
messages split or duplicated?  Also, how would an external client subscribe
to my "outboundTopic" exactly (and get all messages, etc)?

Alternatively, I've been thinking that I should just share a single AMQ
broker between multiple ServiceMix instances.  That would be cleaner in that
there would only be one set of queues/topics to manage and I could scale by
adding more instances.  But, now I'm limited to the vertical scalability of
a single broker and I'm back to a single point of failure...

If anyone can clarify the trade-offs for me...I'd appreciate it. 

Also, I posted this on the Camel forum as well...not sure where it best fits

http://old.nabble.com/clustering-Camel-with-AMQ-ServiceMix-td27462441s22882.html

-----
Ben - Senior Consultant
using SMX 3.3.1/Camel 2.1
-- 
View this message in context: http://old.nabble.com/clustering-options-for-HA-and-horizontal-scalability-tp27473571p27473571.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: clustering options for HA and horizontal scalability

[Bruce Snyder <br...@gmail.com>]

On Fri, Feb 5, 2010 at 1:35 PM, boday <bo...@vektrel.com> wrote:
>
> I'm trying to determine my options for clustering my AMQ 5.3 application that
> uses ServiceMix 3.3.1 and Camel 2.1.  I'm performing high volume message
> processing and I need to cluster for high availability and horizontal
> scalability.
>
> Here is basically what my application does (using Apache
> Camel)...HTTP->QUEUE->PROCESS->DATABASE->TOPIC
>
> So, I've read all the ActiveMQ and ServiceMix clustering pages, but am still
> not sure what to do.
>
> I know I can use a Master/Slave setup for HA, but that doesn't help with
> scalability.
>
> I've read about network of brokers, but am not sure how this applies to my
> application.  For example, if I deploy identical Camel components on
> multiple nodes in a cluster, how will the nodes "interact" exactly?  If I
> point my HTTP producer at one node (NodeA), which messages will get sent to
> NodeB?  Will the queues/topics be shared between Node A/B...if so how, are
> messages split or duplicated?  Also, how would an external client subscribe
> to my "outboundTopic" exactly (and get all messages, etc)?
>
> Alternatively, I've been thinking that I should just share a single AMQ
> broker between multiple ServiceMix instances.  That would be cleaner in that
> there would only be one set of queues/topics to manage and I could scale by
> adding more instances.  But, now I'm limited to the vertical scalability of
> a single broker and I'm back to a single point of failure...
>
> If anyone can clarify the trade-offs for me...I'd appreciate it.
>
> Also, I posted this on the Camel forum as well...not sure where it best fits

Please see my answer to your question over on StackOverflow:

http://stackoverflow.com/questions/2225761/apache-camel-with-activemq-clustering

Bruce
-- 
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'

ActiveMQ in Action: http://bit.ly/2je6cQ
Blog: http://bruceblog.org/
Twitter: http://twitter.com/brucesnyder