You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by yg_cvg <yu...@ugcs.caltech.edu> on 2007/12/08 01:19:13 UTC
question about the store/forward algorithm
This may seem to be more of a developer question, but I think it applies
here. We are considering using ActiveMQ in a large distributed network.
The network is spread all over the world and consists of many small
"regions," each of which contains a few servers. At any time, a piece of
data can come in to one of these servers and should then be sent as quickly
as possible to not only every server in the region, but to every server in
ALL regions. So, basically, there is one big JMS Topic to which everyone is
subscribed.
I figure we'd organize each region into some kind of a hub/spoke thing, or
whatever, to ensure that the message gets sent to everyone in the region.
Then, we'd hook up the "leader" (hub) ActiveMQ servers from each region
together. The question is how to organize this network of leaders in the
best way possible. I mean, it depends on the physical distance between any
given pair of regions, etc.
My question isn't this, though, but something that might help figure out a
good way to do this. It seems I need to understand the ActiveMQ forwarding
algorithm if I want to figure out how to arrange the broker network for
maximum effect. What is the exact algorithm used for broker forwarding of a
given pub/sub message? I have read
http://activemq.apache.org/how-do-distributed-queues-work.html
but don't get exactly how it works, at least for pub/sub, as opposed to a
queue. For a given forwarding broker, does the Message get sent to each
connected broker except the one from which it came, unless the Message has
already been to this forwarding broker (i.e., there is a cycle)? Or is the
Message somehow kept on only one broker at a given time? Does it to BFS or
something cool like that? Any details are welcome.
Any other advice on the general problem at hand is also welcome. I am new
to JMS and ActiveMQ.
--
View this message in context: http://www.nabble.com/question-about-the-store-forward-algorithm-tf4965292s2354.html#a14223478
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: question about the store/forward algorithm
Posted by yg_cvg <yu...@ugcs.caltech.edu>.
To add to that, is there a way to track a message (through some kind of log)
as it travels around the network of brokers?
--
View this message in context: http://www.nabble.com/question-about-the-store-forward-algorithm-tf4965292s2354.html#a14223572
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: question about the store/forward algorithm
Posted by yg_cvg <yu...@ugcs.caltech.edu>.
Thank you for your response. So, the messages spread out to clients in
roughly a BFS pattern.
rajdavies wrote:
>
> Hub and spoke architecture - as you've described - is certainly a
> valid one.
> By default, messages are forwarded to brokers that express an
> interest in the message by themselves having a consumer interested in
> the remote message. Subscription information is propagated from one
> broker to the next, and if configured to do so - that subscription
> information will be propagated further around the network - in BFS
> manner.
>
> However - by default the number of hops messages/subscriptions are
> allowed to pass through is 1 - see http://activemq.apache.org/
> networks-of-brokers.html
> So providing you configure the networkTTL to be equivalent to the
> maximum number of hops - your messages should be able to propagate
> across multiple broker-to-broker steps. Messages/subscriptions carry
> around information about who they've visited - so we can prevent
> cycles happening.
>
> Hope that helps
>
>
>
>
>
> cheers,
>
> Rob
>
> http://open.iona.com/ -Enterprise Open Integration
> http://rajdavies.blogspot.com/
>
>
>
> On Dec 8, 2007, at 12:19 AM, yg_cvg wrote:
>
>>
>> This may seem to be more of a developer question, but I think it
>> applies
>> here. We are considering using ActiveMQ in a large distributed
>> network.
>> The network is spread all over the world and consists of many small
>> "regions," each of which contains a few servers. At any time, a
>> piece of
>> data can come in to one of these servers and should then be sent as
>> quickly
>> as possible to not only every server in the region, but to every
>> server in
>> ALL regions. So, basically, there is one big JMS Topic to which
>> everyone is
>> subscribed.
>>
>> I figure we'd organize each region into some kind of a hub/spoke
>> thing, or
>> whatever, to ensure that the message gets sent to everyone in the
>> region.
>> Then, we'd hook up the "leader" (hub) ActiveMQ servers from each
>> region
>> together. The question is how to organize this network of leaders
>> in the
>> best way possible. I mean, it depends on the physical distance
>> between any
>> given pair of regions, etc.
>>
>> My question isn't this, though, but something that might help
>> figure out a
>> good way to do this. It seems I need to understand the ActiveMQ
>> forwarding
>> algorithm if I want to figure out how to arrange the broker network
>> for
>> maximum effect. What is the exact algorithm used for broker
>> forwarding of a
>> given pub/sub message? I have read
>>
>> http://activemq.apache.org/how-do-distributed-queues-work.html
>>
>> but don't get exactly how it works, at least for pub/sub, as
>> opposed to a
>> queue. For a given forwarding broker, does the Message get sent to
>> each
>> connected broker except the one from which it came, unless the
>> Message has
>> already been to this forwarding broker (i.e., there is a cycle)?
>> Or is the
>> Message somehow kept on only one broker at a given time? Does it to
>> BFS or
>> something cool like that? Any details are welcome.
>>
>> Any other advice on the general problem at hand is also welcome. I
>> am new
>> to JMS and ActiveMQ.
>> --
>> View this message in context: http://www.nabble.com/question-about-
>> the-store-forward-algorithm-tf4965292s2354.html#a14223478
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>
>
>
--
View this message in context: http://www.nabble.com/question-about-the-store-forward-algorithm-tp14223478s2354p14265183.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: question about the store/forward algorithm
Posted by Rob Davies <ra...@gmail.com>.
Hub and spoke architecture - as you've described - is certainly a
valid one.
By default, messages are forwarded to brokers that express an
interest in the message by themselves having a consumer interested in
the remote message. Subscription information is propagated from one
broker to the next, and if configured to do so - that subscription
information will be propagated further around the network - in BFS
manner.
However - by default the number of hops messages/subscriptions are
allowed to pass through is 1 - see http://activemq.apache.org/
networks-of-brokers.html
So providing you configure the networkTTL to be equivalent to the
maximum number of hops - your messages should be able to propagate
across multiple broker-to-broker steps. Messages/subscriptions carry
around information about who they've visited - so we can prevent
cycles happening.
Hope that helps
cheers,
Rob
http://open.iona.com/ -Enterprise Open Integration
http://rajdavies.blogspot.com/
On Dec 8, 2007, at 12:19 AM, yg_cvg wrote:
>
> This may seem to be more of a developer question, but I think it
> applies
> here. We are considering using ActiveMQ in a large distributed
> network.
> The network is spread all over the world and consists of many small
> "regions," each of which contains a few servers. At any time, a
> piece of
> data can come in to one of these servers and should then be sent as
> quickly
> as possible to not only every server in the region, but to every
> server in
> ALL regions. So, basically, there is one big JMS Topic to which
> everyone is
> subscribed.
>
> I figure we'd organize each region into some kind of a hub/spoke
> thing, or
> whatever, to ensure that the message gets sent to everyone in the
> region.
> Then, we'd hook up the "leader" (hub) ActiveMQ servers from each
> region
> together. The question is how to organize this network of leaders
> in the
> best way possible. I mean, it depends on the physical distance
> between any
> given pair of regions, etc.
>
> My question isn't this, though, but something that might help
> figure out a
> good way to do this. It seems I need to understand the ActiveMQ
> forwarding
> algorithm if I want to figure out how to arrange the broker network
> for
> maximum effect. What is the exact algorithm used for broker
> forwarding of a
> given pub/sub message? I have read
>
> http://activemq.apache.org/how-do-distributed-queues-work.html
>
> but don't get exactly how it works, at least for pub/sub, as
> opposed to a
> queue. For a given forwarding broker, does the Message get sent to
> each
> connected broker except the one from which it came, unless the
> Message has
> already been to this forwarding broker (i.e., there is a cycle)?
> Or is the
> Message somehow kept on only one broker at a given time? Does it to
> BFS or
> something cool like that? Any details are welcome.
>
> Any other advice on the general problem at hand is also welcome. I
> am new
> to JMS and ActiveMQ.
> --
> View this message in context: http://www.nabble.com/question-about-
> the-store-forward-algorithm-tf4965292s2354.html#a14223478
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>