Dynamic/Programmatic message bridge/routes?

[ColinStefani <co...@tideworks.com>]

First, I am not the dev here so grain of salt if I'm overlooking some obvious
feature of AMQ, but am looking for guidance/info on the concept of
routing/bridging messages around between topics and queues within the broker
(not between brokers). I am very familiar with Tibco EMS and they have a
"bridge" concept where one can (on the fly, via config file or
programmatically) create a "bridge" between a topic or queue. For example I
can go in to their VERY easy to use console admin tool and do a "create
bridge source=... target=... selector=..." and voila I now have an easy
bridge built to move messages around between the source and target. AMQ
seems to have nothing of this sort that can be done outside of manually
editing the configuration file and restarting the broker (not very cool when
you're trying do this in the background while the system is in production).

The question I am interested in answering, is in ActiveMQ is there a way to
do this type of things outside of the configuration file? I am familiar with
the Camel routing and some of the methods of enterprise patterns in AMQ,
however I cannot find any reference to how one can, while the broker is
running, create a bridge/route between topics/queues or vice versa without
having to edit the configuration file and restart the broker.

Basically, what it boils down to for me is that I use these bridges often
for both production use (they are static in my Tibco configuration in this
case) and for testing/monitoring use (where they are created on demand while
the system is running). The latter is where I might create a bridge on the
fly to "watch" messages from some application or going to some
service/application.

Side note (suggestion?) to this, often I want to watch the inbound messages
to a queue, but so far in AMQ I have not been able to figure out how to
bridge messages from a queue to a topic in a way that doesn't actually
consume the messages (in Tibco they are "copied" without impact to the
consumers on the queue).

I hope this makes sense. There maybe ways to do this I have not found yet
(i.e. via coding so that I could build a web app to bridge messages around
or something) but I haven't been able to find any reference to this so far
in AMQ docs or forums. We leverage this heavily given the complex
environment I'm using, and my goal is to replace Tibco EMS with AMQ
eventually, but was hoping for some equivalency in this area since it's
common in other brokers as well and I might just be missing it.

Thanks and Regards,

Colin
-- 
View this message in context: http://www.nabble.com/Dynamic-Programmatic-message-bridge-routes--tp19148007p19148007.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Dynamic/Programmatic message bridge/routes?

[ColinStefani <co...@tideworks.com>]

James,

Excellent, this gives us a good starting point and we'll certainly check out
the ServiceMix scenario.

Appreciate the feedback.

Colin





James.Strachan wrote:
> 
> Hi Colin
> 
> 2008/8/25 ColinStefani <co...@tideworks.com>:
>>
>> First, I am not the dev here so grain of salt if I'm overlooking some
>> obvious
>> feature of AMQ, but am looking for guidance/info on the concept of
>> routing/bridging messages around between topics and queues within the
>> broker
>> (not between brokers). I am very familiar with Tibco EMS and they have a
>> "bridge" concept where one can (on the fly, via config file or
>> programmatically) create a "bridge" between a topic or queue. For example
>> I
>> can go in to their VERY easy to use console admin tool and do a "create
>> bridge source=... target=... selector=..." and voila I now have an easy
>> bridge built to move messages around between the source and target. AMQ
>> seems to have nothing of this sort that can be done outside of manually
>> editing the configuration file and restarting the broker (not very cool
>> when
>> you're trying do this in the background while the system is in
>> production).
>>
>> The question I am interested in answering, is in ActiveMQ is there a way
>> to
>> do this type of things outside of the configuration file? I am familiar
>> with
>> the Camel routing and some of the methods of enterprise patterns in AMQ,
>> however I cannot find any reference to how one can, while the broker is
>> running, create a bridge/route between topics/queues or vice versa
>> without
>> having to edit the configuration file and restart the broker.
> 
> So Camel is the preferred way to do things like you suggest - such as
> a content based router taking messages from a queue/topic and routing
> them to any other queue/topic based on some predicate etc.
> 
> However adding/removing routing rules doesn't necessarily mean you
> have to restart the broker; you can run the Camel routing rules
> anywhere - in any JVM. So you can boot up a camel routing rule in
> production whenever you like - then stop it whenever you like.
> 
> FWIW moving forward, the ActiveMQ broker is gonna be using the
> ServiceMix 4 kernel...
> http://servicemix.apache.org/SMX4/index.html
> 
> this means that within the broker JVM we'll be able to easily
> add/remove/update camel routing rules without having to restart the
> broker. For example today you can run the ActiveMQ broker inside the
> SMX4 kernel and also deploy an OSGi bundle containing Camel routes in
> expanded form (rather like WAR expanded form) - then at any time
> update your Java or Spring XML code and it'll auto-reload and restart
> the routing rules dynamically without restarting the broker.
> 
> In the meantime, you can just run your routing rules separately from the
> broker.
> 
> Also we're working on a web based console to allow you to add/remove
> Camel routing rules dynamically...
> http://open.iona.com/products/enterprise-activemq/
> 
> which should be available in a few months time
> 
> 
>> Basically, what it boils down to for me is that I use these bridges often
>> for both production use (they are static in my Tibco configuration in
>> this
>> case) and for testing/monitoring use (where they are created on demand
>> while
>> the system is running). The latter is where I might create a bridge on
>> the
>> fly to "watch" messages from some application or going to some
>> service/application.
>>
>> Side note (suggestion?) to this, often I want to watch the inbound
>> messages
>> to a queue, but so far in AMQ I have not been able to figure out how to
>> bridge messages from a queue to a topic in a way that doesn't actually
>> consume the messages (in Tibco they are "copied" without impact to the
>> consumers on the queue).
> 
> FWIW if you just want to watch messages arriving at a queue, you could
> use Mirrored Queues
> http://activemq.apache.org/mirrored-queues.html
> 
> but really Camel is the better, more powerful solution long term.
> 
> 
>> I hope this makes sense. There maybe ways to do this I have not found yet
>> (i.e. via coding so that I could build a web app to bridge messages
>> around
>> or something) but I haven't been able to find any reference to this so
>> far
>> in AMQ docs or forums.
> 
> You could host Camel routing rules in a web app too if you like
> 
> 
>> We leverage this heavily given the complex
>> environment I'm using, and my goal is to replace Tibco EMS with AMQ
>> eventually, but was hoping for some equivalency in this area since it's
>> common in other brokers as well and I might just be missing it.
> 
> Good luck with your replacement of EMS :) Let us know if there's
> anything else you need
> 
> -- 
> James
> -------
> http://macstrac.blogspot.com/
> 
> Open Source Integration
> http://open.iona.com
> 
> 

-- 
View this message in context: http://www.nabble.com/Dynamic-Programmatic-message-bridge-routes--tp19148007p19164847.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Dynamic/Programmatic message bridge/routes?

[James Strachan <ja...@gmail.com>]

Hi Colin

2008/8/25 ColinStefani <co...@tideworks.com>:
>
> First, I am not the dev here so grain of salt if I'm overlooking some obvious
> feature of AMQ, but am looking for guidance/info on the concept of
> routing/bridging messages around between topics and queues within the broker
> (not between brokers). I am very familiar with Tibco EMS and they have a
> "bridge" concept where one can (on the fly, via config file or
> programmatically) create a "bridge" between a topic or queue. For example I
> can go in to their VERY easy to use console admin tool and do a "create
> bridge source=... target=... selector=..." and voila I now have an easy
> bridge built to move messages around between the source and target. AMQ
> seems to have nothing of this sort that can be done outside of manually
> editing the configuration file and restarting the broker (not very cool when
> you're trying do this in the background while the system is in production).
>
> The question I am interested in answering, is in ActiveMQ is there a way to
> do this type of things outside of the configuration file? I am familiar with
> the Camel routing and some of the methods of enterprise patterns in AMQ,
> however I cannot find any reference to how one can, while the broker is
> running, create a bridge/route between topics/queues or vice versa without
> having to edit the configuration file and restart the broker.

So Camel is the preferred way to do things like you suggest - such as
a content based router taking messages from a queue/topic and routing
them to any other queue/topic based on some predicate etc.

However adding/removing routing rules doesn't necessarily mean you
have to restart the broker; you can run the Camel routing rules
anywhere - in any JVM. So you can boot up a camel routing rule in
production whenever you like - then stop it whenever you like.

FWIW moving forward, the ActiveMQ broker is gonna be using the
ServiceMix 4 kernel...
http://servicemix.apache.org/SMX4/index.html

this means that within the broker JVM we'll be able to easily
add/remove/update camel routing rules without having to restart the
broker. For example today you can run the ActiveMQ broker inside the
SMX4 kernel and also deploy an OSGi bundle containing Camel routes in
expanded form (rather like WAR expanded form) - then at any time
update your Java or Spring XML code and it'll auto-reload and restart
the routing rules dynamically without restarting the broker.

In the meantime, you can just run your routing rules separately from the broker.

Also we're working on a web based console to allow you to add/remove
Camel routing rules dynamically...
http://open.iona.com/products/enterprise-activemq/

which should be available in a few months time


> Basically, what it boils down to for me is that I use these bridges often
> for both production use (they are static in my Tibco configuration in this
> case) and for testing/monitoring use (where they are created on demand while
> the system is running). The latter is where I might create a bridge on the
> fly to "watch" messages from some application or going to some
> service/application.
>
> Side note (suggestion?) to this, often I want to watch the inbound messages
> to a queue, but so far in AMQ I have not been able to figure out how to
> bridge messages from a queue to a topic in a way that doesn't actually
> consume the messages (in Tibco they are "copied" without impact to the
> consumers on the queue).

FWIW if you just want to watch messages arriving at a queue, you could
use Mirrored Queues
http://activemq.apache.org/mirrored-queues.html

but really Camel is the better, more powerful solution long term.


> I hope this makes sense. There maybe ways to do this I have not found yet
> (i.e. via coding so that I could build a web app to bridge messages around
> or something) but I haven't been able to find any reference to this so far
> in AMQ docs or forums.

You could host Camel routing rules in a web app too if you like


> We leverage this heavily given the complex
> environment I'm using, and my goal is to replace Tibco EMS with AMQ
> eventually, but was hoping for some equivalency in this area since it's
> common in other brokers as well and I might just be missing it.

Good luck with your replacement of EMS :) Let us know if there's
anything else you need

-- 
James
-------
http://macstrac.blogspot.com/

Open Source Integration
http://open.iona.com