You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@servicemix.apache.org by Stefan Klinger <kl...@cs.york.ac.uk> on 2006/05/09 12:04:09 UTC
Re: Distributed ESB
Hello,
After scanning through the ActiveMQ documentation, I noticed that it
offers a number of network topologies. I am really interested in the
peer-to-peer JMS network. Could someone show me an example of how I
could configure two or more instances of ServiceMix running
in-memory-flows (seda or st)? However, I don't want to multicast the
messages to each instance, rather send it to one which I specify, this
instance then acts as a broker (or master) and forwards the message to
the other instances (slaves). The returned messages are aggregated in my
master and returned to the client.
Thanks for any suggestions.
Stefan
Bruce Snyder wrote:
> On 4/27/06, Stefan Klinger <kl...@cs.york.ac.uk> wrote:
>> Hello,
>>
>> I have been wondering for a while about the notion of a distributed ESB,
>> i.e. several separate instances of SM distributed across the world and
>> each instance is only aware of local services. However, each instance of
>> the SM is aware of the other one, and if it cannot fulfill a routing
>> request locally, it contacts the other SM instances it is aware of. The
>> rationale behind it is that we would like to run intensive processing
>> close to the data. One could extend this notion to a hierarchical master
>> and slave model, where the master is in charge of a global workflow, and
>> each slave has to run a local workflow on behalf of the master.
>>
>> Would this be possible with SM? If not, does anybody think it is a good/
>> bad idea to have an architecture like this?
>
> Stefan,
>
> ServiceMix provides distributed capabilities with regard to failover,
> load-balancing, message durability and routing between instances.
> ServiceMix is built on top of ActiveMQ and this is how it achieves all
> of these features. Another feature that is part of this scenario is
> the ability to name multiple flows so that an in-memory flow can be
> used for local communication and a distributed flow for distributed
> communication. I'm sure Guillaume can comment more on multiple flows.
>
> Let us know if you have additional questions.
>
> Bruce
> --
> perl -e 'print
> unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
> );'
>
> Apache Geronimo - http://geronimo.apache.org/
> Apache ActiveMQ - http://incubator.apache.org/activemq/
> Apache ServiceMix - http://incubator.apache.org/servicemix/
> Castor - http://castor.org/
Re: Distributed ESB
Posted by Guillaume Nodet <gn...@gmail.com>.
The way ServiceMix handle clustering is transparent, so that you
should not bother wether your different services are on the same node
or on different nodes.
If you have a known (static) service list, you could take a look at
http://sevicemix.org/servicemix-eip, where the StaticRecipientList
could solve this problem, else you may want to use WS-Notification
(servicemix-wsn2005 component) so that services can be added / removed
dynamically without disturbing the flow.
You will also have to write an aggregator, which is a feature planned
for addition in servicemix-eip.
Cheers,
Guillaume Nodet
On 5/9/06, Stefan Klinger <kl...@cs.york.ac.uk> wrote:
> Thanks Guillaume,
>
> My idea is to have several ESB instances running, each one being a copy
> of the other, so they have equivalent consumers. The consumers have the
> same interfaces, so the messages are identical, however, they work on
> different data, so the results returned will be different. Therefore, I
> would like to send the same messages to similar consumers on all ESB
> instances, and aggregate the results in one instance. Most of the
> processing would be done local in regard to the data.
>
> If I used the jms flow, it could only send it to one consumer on one of
> the ESB instances, is that right? I could create a broker myself on each
> instance which sends copies of the message to the other ESB instances
> discovered. Or are there better ways in achieving this architecture?
>
> Stefan
>
> Guillaume Nodet wrote:
> > If you use a seda or st flow, you won't have clustered ServiceMix
> > instances. You have to use the jms or jca flow (note that can
> > configure more than one flow in ServiceMix).
> > By default, the jms flow uses a peer protocol. This does not mean
> > that jms messages are broadcast to all nodes: the peer protocol uses
> > multicast for discovery and tcp for sending jms messages. Messages
> > are only sent to the jms broker that can / will process it (it has a
> > consumer on the destination).
> > What is the prupose of your the master / slave ? HA ?
> >
> > Cheers,
> > Guillaume Nodet
> >
> > On 5/9/06, Stefan Klinger <kl...@cs.york.ac.uk> wrote:
> >> Hello,
> >>
> >> After scanning through the ActiveMQ documentation, I noticed that it
> >> offers a number of network topologies. I am really interested in the
> >> peer-to-peer JMS network. Could someone show me an example of how I
> >> could configure two or more instances of ServiceMix running
> >> in-memory-flows (seda or st)? However, I don't want to multicast the
> >> messages to each instance, rather send it to one which I specify, this
> >> instance then acts as a broker (or master) and forwards the message to
> >> the other instances (slaves). The returned messages are aggregated in my
> >> master and returned to the client.
> >>
> >> Thanks for any suggestions.
> >>
> >> Stefan
> >>
> >> Bruce Snyder wrote:
> >> > On 4/27/06, Stefan Klinger <kl...@cs.york.ac.uk> wrote:
> >> >> Hello,
> >> >>
> >> >> I have been wondering for a while about the notion of a
> >> distributed ESB,
> >> >> i.e. several separate instances of SM distributed across the world
> >> and
> >> >> each instance is only aware of local services. However, each
> >> instance of
> >> >> the SM is aware of the other one, and if it cannot fulfill a routing
> >> >> request locally, it contacts the other SM instances it is aware
> >> of. The
> >> >> rationale behind it is that we would like to run intensive processing
> >> >> close to the data. One could extend this notion to a hierarchical
> >> master
> >> >> and slave model, where the master is in charge of a global
> >> workflow, and
> >> >> each slave has to run a local workflow on behalf of the master.
> >> >>
> >> >> Would this be possible with SM? If not, does anybody think it is a
> >> good/
> >> >> bad idea to have an architecture like this?
> >> >
> >> > Stefan,
> >> >
> >> > ServiceMix provides distributed capabilities with regard to failover,
> >> > load-balancing, message durability and routing between instances.
> >> > ServiceMix is built on top of ActiveMQ and this is how it achieves all
> >> > of these features. Another feature that is part of this scenario is
> >> > the ability to name multiple flows so that an in-memory flow can be
> >> > used for local communication and a distributed flow for distributed
> >> > communication. I'm sure Guillaume can comment more on multiple flows.
> >> >
> >> > Let us know if you have additional questions.
> >> >
> >> > Bruce
> >> > --
> >> > perl -e 'print
> >> > unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
> >> > );'
> >> >
> >> > Apache Geronimo - http://geronimo.apache.org/
> >> > Apache ActiveMQ - http://incubator.apache.org/activemq/
> >> > Apache ServiceMix - http://incubator.apache.org/servicemix/
> >> > Castor - http://castor.org/
> >>
> >>
>
>
Re: Distributed ESB
Posted by Stefan Klinger <kl...@cs.york.ac.uk>.
Thanks Guillaume,
My idea is to have several ESB instances running, each one being a copy
of the other, so they have equivalent consumers. The consumers have the
same interfaces, so the messages are identical, however, they work on
different data, so the results returned will be different. Therefore, I
would like to send the same messages to similar consumers on all ESB
instances, and aggregate the results in one instance. Most of the
processing would be done local in regard to the data.
If I used the jms flow, it could only send it to one consumer on one of
the ESB instances, is that right? I could create a broker myself on each
instance which sends copies of the message to the other ESB instances
discovered. Or are there better ways in achieving this architecture?
Stefan
Guillaume Nodet wrote:
> If you use a seda or st flow, you won't have clustered ServiceMix
> instances. You have to use the jms or jca flow (note that can
> configure more than one flow in ServiceMix).
> By default, the jms flow uses a peer protocol. This does not mean
> that jms messages are broadcast to all nodes: the peer protocol uses
> multicast for discovery and tcp for sending jms messages. Messages
> are only sent to the jms broker that can / will process it (it has a
> consumer on the destination).
> What is the prupose of your the master / slave ? HA ?
>
> Cheers,
> Guillaume Nodet
>
> On 5/9/06, Stefan Klinger <kl...@cs.york.ac.uk> wrote:
>> Hello,
>>
>> After scanning through the ActiveMQ documentation, I noticed that it
>> offers a number of network topologies. I am really interested in the
>> peer-to-peer JMS network. Could someone show me an example of how I
>> could configure two or more instances of ServiceMix running
>> in-memory-flows (seda or st)? However, I don't want to multicast the
>> messages to each instance, rather send it to one which I specify, this
>> instance then acts as a broker (or master) and forwards the message to
>> the other instances (slaves). The returned messages are aggregated in my
>> master and returned to the client.
>>
>> Thanks for any suggestions.
>>
>> Stefan
>>
>> Bruce Snyder wrote:
>> > On 4/27/06, Stefan Klinger <kl...@cs.york.ac.uk> wrote:
>> >> Hello,
>> >>
>> >> I have been wondering for a while about the notion of a
>> distributed ESB,
>> >> i.e. several separate instances of SM distributed across the world
>> and
>> >> each instance is only aware of local services. However, each
>> instance of
>> >> the SM is aware of the other one, and if it cannot fulfill a routing
>> >> request locally, it contacts the other SM instances it is aware
>> of. The
>> >> rationale behind it is that we would like to run intensive processing
>> >> close to the data. One could extend this notion to a hierarchical
>> master
>> >> and slave model, where the master is in charge of a global
>> workflow, and
>> >> each slave has to run a local workflow on behalf of the master.
>> >>
>> >> Would this be possible with SM? If not, does anybody think it is a
>> good/
>> >> bad idea to have an architecture like this?
>> >
>> > Stefan,
>> >
>> > ServiceMix provides distributed capabilities with regard to failover,
>> > load-balancing, message durability and routing between instances.
>> > ServiceMix is built on top of ActiveMQ and this is how it achieves all
>> > of these features. Another feature that is part of this scenario is
>> > the ability to name multiple flows so that an in-memory flow can be
>> > used for local communication and a distributed flow for distributed
>> > communication. I'm sure Guillaume can comment more on multiple flows.
>> >
>> > Let us know if you have additional questions.
>> >
>> > Bruce
>> > --
>> > perl -e 'print
>> > unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
>> > );'
>> >
>> > Apache Geronimo - http://geronimo.apache.org/
>> > Apache ActiveMQ - http://incubator.apache.org/activemq/
>> > Apache ServiceMix - http://incubator.apache.org/servicemix/
>> > Castor - http://castor.org/
>>
>>
Re: Distributed ESB
Posted by Guillaume Nodet <gn...@gmail.com>.
If you use a seda or st flow, you won't have clustered ServiceMix
instances. You have to use the jms or jca flow (note that can
configure more than one flow in ServiceMix).
By default, the jms flow uses a peer protocol. This does not mean
that jms messages are broadcast to all nodes: the peer protocol uses
multicast for discovery and tcp for sending jms messages. Messages
are only sent to the jms broker that can / will process it (it has a
consumer on the destination).
What is the prupose of your the master / slave ? HA ?
Cheers,
Guillaume Nodet
On 5/9/06, Stefan Klinger <kl...@cs.york.ac.uk> wrote:
> Hello,
>
> After scanning through the ActiveMQ documentation, I noticed that it
> offers a number of network topologies. I am really interested in the
> peer-to-peer JMS network. Could someone show me an example of how I
> could configure two or more instances of ServiceMix running
> in-memory-flows (seda or st)? However, I don't want to multicast the
> messages to each instance, rather send it to one which I specify, this
> instance then acts as a broker (or master) and forwards the message to
> the other instances (slaves). The returned messages are aggregated in my
> master and returned to the client.
>
> Thanks for any suggestions.
>
> Stefan
>
> Bruce Snyder wrote:
> > On 4/27/06, Stefan Klinger <kl...@cs.york.ac.uk> wrote:
> >> Hello,
> >>
> >> I have been wondering for a while about the notion of a distributed ESB,
> >> i.e. several separate instances of SM distributed across the world and
> >> each instance is only aware of local services. However, each instance of
> >> the SM is aware of the other one, and if it cannot fulfill a routing
> >> request locally, it contacts the other SM instances it is aware of. The
> >> rationale behind it is that we would like to run intensive processing
> >> close to the data. One could extend this notion to a hierarchical master
> >> and slave model, where the master is in charge of a global workflow, and
> >> each slave has to run a local workflow on behalf of the master.
> >>
> >> Would this be possible with SM? If not, does anybody think it is a good/
> >> bad idea to have an architecture like this?
> >
> > Stefan,
> >
> > ServiceMix provides distributed capabilities with regard to failover,
> > load-balancing, message durability and routing between instances.
> > ServiceMix is built on top of ActiveMQ and this is how it achieves all
> > of these features. Another feature that is part of this scenario is
> > the ability to name multiple flows so that an in-memory flow can be
> > used for local communication and a distributed flow for distributed
> > communication. I'm sure Guillaume can comment more on multiple flows.
> >
> > Let us know if you have additional questions.
> >
> > Bruce
> > --
> > perl -e 'print
> > unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
> > );'
> >
> > Apache Geronimo - http://geronimo.apache.org/
> > Apache ActiveMQ - http://incubator.apache.org/activemq/
> > Apache ServiceMix - http://incubator.apache.org/servicemix/
> > Castor - http://castor.org/
>
>