Broker Pipeline

[Banana Man <al...@btinternet.com>]

Hi All,

I am looking for some advice on using the broker for a pipeline of
components (http://activemq.apache.org/camel/pipes-and-filters.html). 

A couple of pre-requisites of the pipeline are:
1) the processing components should not have any knowledge of their position
in the pipeline so that additional processing components could be easily
added and existing ones removed
2) all of the processing components in the pipeline should be optional
depending on message type and content
3) all messages into the pipeline and from processing components within the
pipeline should be sent to a single queue. The Broker should then direct the
message to the correct single physical destination based on message content
(and from there picked up by a processing component). 

I have got as far as thinking that Filtered Composite Destinations look like
a good bet. I read that it is not possible to forward to a virtual
destination from a virtual destination so I am a bit concerned about how
many rules might fire against the filter destination selectors. Is it
possible to force a best match on only one filter destination because I want
a single 'message in' to trigger only a single 'message out'.

Might this work be more appropriately put in a dedicated rules
engine/broker?

Thanks again.


-- 
View this message in context: http://www.nabble.com/Broker-Pipeline-tf3706441s2354.html#a10366013
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Broker Pipeline

[James Strachan <ja...@gmail.com>]

On 5/7/07, Banana Man <al...@btinternet.com> wrote:
>
> Hi All,

Hi Banana Man :)


> I am looking for some advice on using the broker for a pipeline of
> components (http://activemq.apache.org/camel/pipes-and-filters.html).
>
> A couple of pre-requisites of the pipeline are:
> 1) the processing components should not have any knowledge of their position
> in the pipeline so that additional processing components could be easily
> added and existing ones removed
> 2) all of the processing components in the pipeline should be optional
> depending on message type and content

That all sounds good so far


> 3) all messages into the pipeline and from processing components within the
> pipeline should be sent to a single queue.

FWIW its often useful to use a separate queue for each step in a
pipeline; if nothing else it just makes it easier to monitor and
manage the system. e.g. if one particular step of a pipeline is not
running, or is blocked, you can then easily see by watching the
relative queue depths


> The Broker should then direct the
> message to the correct single physical destination based on message content
> (and from there picked up by a processing component).

Sounds good. A single content based route would do the trick....
http://activemq.apache.org/camel/content-based-router.html


> I have got as far as thinking that Filtered Composite Destinations look like
> a good bet. I read that it is not possible to forward to a virtual
> destination from a virtual destination so I am a bit concerned about how
> many rules might fire against the filter destination selectors. Is it
> possible to force a best match on only one filter destination because I want
> a single 'message in' to trigger only a single 'message out'.
>
> Might this work be more appropriately put in a dedicated rules
> engine/broker?

Yeah, Camel sounds like just the thing for the routing rules engine.
ActiveMQ is great at dealing with individual destinations (whether
physical or virtual) but as soon as complex routing requirements are
needed, like smart content based routing or whatever, using Camel is a
more flexible & powerful option.



-- 
James
-------
http://macstrac.blogspot.com/