You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by John Skopis <jo...@gmail.com> on 2010/05/12 19:10:30 UTC
equal cost routes
Hello,
I am just getting started with AMQP and qpid so please let me know if
I am making any newbie mistakes.
I have three locations feeding a fourth (centralized) location with
messages using a pub/sub model. Each feeder site has 8 publishers
connected to a local broker that federates the messages to the final
centralized destination.
The architecture I am currently evaluating is comprised of the following:
The part that works:
each of the 8 publishers maintains an exchange fed.topic with a queue,
fed.local, bound to the key '#'
there is a queue push route(s) from localhost(queue=fed.local) => [
gw01.siteN, gw02.siteN ] (ex=fed.topic)
The part I have yet to test:
the 'gw' brokers maintain fed.topic bound to (or multiple) a queue
fed.aggr on the key "#"
The centralized broker has a pull queue route(s) from
central(ex=fed.topic) <= [ gw01.site1, gw02.site1, gw01.site2 ...]
(queue=fed.aggr)
I actually haven't gotten as far as the centralized broker as I am
concerned with reliability from local => aggr. To test I am using a
slightly modified version of the topic_publisher and topic_subscriber
python example scripts.
My first reaction was "cool" the messages are equally routed across both links.
I then stopped qpid on one of the gateways and noticed I lost a few
test messages. I set --ack 1 and would only lose 1 message. I then set
the delivery_mode prop to persistent and did not lose any messages!!
(thanks jrobie)
Now the question:
So far it's working great, but I noticed with a persistent message vs
a non-persistent message the messages are not balanced across both
gw01, gw02, instead they operate in an active/passive fashion. I am
wondering if this is intentional or merely a coincidence?
My experience with qpid so far is that it "just works" (my favorite
kind of software). Keep up the good work.!
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: equal cost routes
Posted by Carl Trieloff <cc...@redhat.com>.
On 05/19/2010 12:24 PM, Gordon Sim wrote:
> On 05/19/2010 04:42 PM, John Skopis wrote:
>> My conclusion is that persistent messages are load balanced via
>> multiple routes, but only when there is more than one session active
>> on the source broker.
>> Non-persistent messages are load balanced via multiple routes
>> regardless of how many active sessions exist on the source broker.
>
> There is certainly no logic to do this, and I don't see the same
> behaviour myself. Messages on a queue are not shared out between
> subscribers in strict round-robin, subscribers compete to pull them
> off the queue as fast as they can. Over a large set of messages this
> usually results in a relatively 'fair' allocation. It is possible that
> the durability can alter the timings in some cases such that the
> distribution appears to be different
Are you perhaps running with worker-threads == 1 or on a single core
machine. It may be an artefact of threading if that is the case.
Carl.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: equal cost routes
Posted by Gordon Sim <gs...@redhat.com>.
On 05/19/2010 04:42 PM, John Skopis wrote:
> My conclusion is that persistent messages are load balanced via
> multiple routes, but only when there is more than one session active
> on the source broker.
> Non-persistent messages are load balanced via multiple routes
> regardless of how many active sessions exist on the source broker.
There is certainly no logic to do this, and I don't see the same
behaviour myself. Messages on a queue are not shared out between
subscribers in strict round-robin, subscribers compete to pull them off
the queue as fast as they can. Over a large set of messages this usually
results in a relatively 'fair' allocation. It is possible that the
durability can alter the timings in some cases such that the
distribution appears to be different
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: equal cost routes
Posted by John Skopis <jo...@skopis.com>.
On Wed, May 19, 2010 at 7:59 AM, Alan Conway <ac...@redhat.com> wrote:
> On 05/18/2010 06:35 PM, John Skopis wrote:
>>>
>>> This means that any matching message sent to fed.topic on pub1 will be
>>> enqueued on fed.local and delivered to *either* fed.topic on sub1 *or* to
>>> the exchange of the same name on sub2.
>>>
>>
>> Running dpublisher.py I see msg 1,2,3,4,5,6 ... on sub1
>>
>> Running publisher.py I see msg 1,3,5... on sub1
>> and
>> msg 2,4,6... on sub2
>>
>> I was wondering if this is a peculiarity with my environment or if it is
>> normal.
>>
>>
>
> That is unusual. Do you see this behaviour for large numbers (1000s) of
> messages? If so it deserves some investigation.
>
Hello,
I repeated my test using the multiprocessing module to create more
than 1 session.
With 1 session I received the same results that I described already.
With 2 or more sessions I received results similar to using a single
session and non-persistent message.
My conclusion is that persistent messages are load balanced via
multiple routes, but only when there is more than one session active
on the source broker.
Non-persistent messages are load balanced via multiple routes
regardless of how many active sessions exist on the source broker.
Thanks again,
John
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: equal cost routes
Posted by Alan Conway <ac...@redhat.com>.
On 05/18/2010 06:35 PM, John Skopis wrote:
>> This means that any matching message sent to fed.topic on pub1 will be
>> enqueued on fed.local and delivered to *either* fed.topic on sub1 *or* to
>> the exchange of the same name on sub2.
>>
>
> Running dpublisher.py I see msg 1,2,3,4,5,6 ... on sub1
>
> Running publisher.py I see msg 1,3,5... on sub1
> and
> msg 2,4,6... on sub2
>
> I was wondering if this is a peculiarity with my environment or if it is normal.
>
>
That is unusual. Do you see this behaviour for large numbers (1000s) of
messages? If so it deserves some investigation.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: equal cost routes
Posted by John Skopis <jo...@gmail.com>.
> This means that any matching message sent to fed.topic on pub1 will be
> enqueued on fed.local and delivered to *either* fed.topic on sub1 *or* to
> the exchange of the same name on sub2.
>
Running dpublisher.py I see msg 1,2,3,4,5,6 ... on sub1
Running publisher.py I see msg 1,3,5... on sub1
and
msg 2,4,6... on sub2
I was wondering if this is a peculiarity with my environment or if it is normal.
Thanks,
john
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: equal cost routes
Posted by Gordon Sim <gs...@redhat.com>.
On 05/14/2010 07:12 PM, John Skopis wrote:
> On Thu, May 13, 2010 at 12:38 PM, Gordon Sim<gs...@redhat.com> wrote:
>> On 05/12/2010 06:10 PM, John Skopis wrote:
> [snip]
>>
>> The persistence of the message should not affect this. However subscribers
>> to a queue will share the messages, so if you have more than one route
>> through the same queue then the messages will be shared between them. Does
>> that sound like your case?
>>
>
> What I observed was that if the message was persistent the routes are
> active/passive; if the message is non-persistent the routes are
> active/active
Maybe I'm misunderstanding what you mean by active/active v.
active/passive here, but the durability of the message should not affect
the routing at all.
> I put together this example:
> http://springer.networkgimps.com/~skopii/example.tgz
>
> If you have three systems to test pub1, sub1, sub2 run the
> pub_setup.sh and sub_setup.sh scripts to create the queues.
In pub_setup.sh you are creating a queue (fed.local) from which messages
will be routed to the subscriber brokers (sub1 and sub2).
This means that any matching message sent to fed.topic on pub1 will be
enqueued on fed.local and delivered to *either* fed.topic on sub1 *or*
to the exchange of the same name on sub2.
> Then run the example subscriber.py:
> while true; do python subscriber.py; done
>
> and then run publisher.py and dpublisher.py on pub1.
>
> Is it the same for you?
>
> Sorry for the mess. I think the examples demonstrate what I observed
> better than I could ever explain.
>
> Thanks,
> John
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: equal cost routes
Posted by John Skopis <jo...@gmail.com>.
On Thu, May 13, 2010 at 12:38 PM, Gordon Sim <gs...@redhat.com> wrote:
> On 05/12/2010 06:10 PM, John Skopis wrote:
[snip]
>
> The persistence of the message should not affect this. However subscribers
> to a queue will share the messages, so if you have more than one route
> through the same queue then the messages will be shared between them. Does
> that sound like your case?
>
What I observed was that if the message was persistent the routes are
active/passive; if the message is non-persistent the routes are
active/active
I put together this example:
http://springer.networkgimps.com/~skopii/example.tgz
If you have three systems to test pub1, sub1, sub2 run the
pub_setup.sh and sub_setup.sh scripts to create the queues.
Then run the example subscriber.py:
while true; do python subscriber.py; done
and then run publisher.py and dpublisher.py on pub1.
Is it the same for you?
Sorry for the mess. I think the examples demonstrate what I observed
better than I could ever explain.
Thanks,
John
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: equal cost routes
Posted by Gordon Sim <gs...@redhat.com>.
On 05/12/2010 06:10 PM, John Skopis wrote:
> Hello,
>
> I am just getting started with AMQP and qpid so please let me know if
> I am making any newbie mistakes.
>
> I have three locations feeding a fourth (centralized) location with
> messages using a pub/sub model. Each feeder site has 8 publishers
> connected to a local broker that federates the messages to the final
> centralized destination.
>
> The architecture I am currently evaluating is comprised of the following:
>
> The part that works:
> each of the 8 publishers maintains an exchange fed.topic with a queue,
> fed.local, bound to the key '#'
> there is a queue push route(s) from localhost(queue=fed.local) => [
> gw01.siteN, gw02.siteN ] (ex=fed.topic)
>
> The part I have yet to test:
> the 'gw' brokers maintain fed.topic bound to (or multiple) a queue
> fed.aggr on the key "#"
> The centralized broker has a pull queue route(s) from
> central(ex=fed.topic)<= [ gw01.site1, gw02.site1, gw01.site2 ...]
> (queue=fed.aggr)
>
> I actually haven't gotten as far as the centralized broker as I am
> concerned with reliability from local => aggr. To test I am using a
> slightly modified version of the topic_publisher and topic_subscriber
> python example scripts.
>
> My first reaction was "cool" the messages are equally routed across both links.
>
> I then stopped qpid on one of the gateways and noticed I lost a few
> test messages. I set --ack 1 and would only lose 1 message. I then set
> the delivery_mode prop to persistent and did not lose any messages!!
> (thanks jrobie)
>
>
> Now the question:
> So far it's working great, but I noticed with a persistent message vs
> a non-persistent message the messages are not balanced across both
> gw01, gw02, instead they operate in an active/passive fashion. I am
> wondering if this is intentional or merely a coincidence?
The persistence of the message should not affect this. However
subscribers to a queue will share the messages, so if you have more than
one route through the same queue then the messages will be shared
between them. Does that sound like your case?
>
> My experience with qpid so far is that it "just works" (my favorite
> kind of software). Keep up the good work.!
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org