You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by ac...@terra.com.br on 2009/09/02 16:33:09 UTC
How to properly implement a 1-to-N pattern
*
*
*
Hi,
By what I’ve read on the AMQP specification,
should I have N consumers to a queue, for each message posted to it,
one of the consumers is going to be activated in round-robin. The
spec call this a store-and-forward-queue (section 2.1.1). What If I
want to have all the consumers alerted about each message posted?
I figured that I could manage to have each of them
creating a queue and binding it to a fanout Exchange. Then, for each
message posted to the Exchange, all the queues would get their copies
and thus all the consumers would be alerted about it.
So, is this the right way to implement the scenario
of having a producer activating N consumers or is there another type
of exchange/queue that would give me this functionality?
Thanks in advance,
Acácio
BODY { font-family:Arial, Helvetica, sans-serif;font-size:12px; }
Re: How to properly implement a 1-to-N pattern
Posted by Joshua Kramer <jo...@globalherald.net>.
Hello Acacio,
You can use the Fanout exchange. Or, you can use a publish-subscribe
model. Each server queue that is subscribed to a certain topic on the
Publish-Subscribe exchange will recieve a message with that topic. So if
you have 5 server queues subscribed to the topic "weather", then they will
all receive messages with the topic "weather".
In turn, you can have any number of local queues listening to the server
queue. The local queues are served in a round-robin fashion. So, if you
have three local queues listening to one server queue, then only one of
the local queues will receive any message bound for the server queue.
You can combine this. Suppose, for example, that you have 15 application
servers. You have five different events that you listen for; each event
has a cluster of three servers used for processing. To make this work,
you create 5 server queues and subscribe them each to the appropriate
topic... then, to each of the 5 server queues set up 3 local queues for
listening. In this way, each server queue will distribute requests to its
own three-server cluster in a round robin manner.
I have a Linux Journal article set to be published in November covering
these topics. Let me know if you have more questions.
best,
-JK
On Wed, 2 Sep 2009, acacio.centeno@terra.com.br wrote:
> Date: Wed, 2 Sep 2009 14:33:09 +0000
> From: acacio.centeno@terra.com.br
> Reply-To: users@qpid.apache.org, acacio.centeno@terra.com.br
> To: users@qpid.apache.org
> Subject: How to properly implement a 1-to-N pattern
>
>
> *
> *
> *
>
> Hi,
> By what I’ve read on the AMQP specification,
> should I have N consumers to a queue, for each message posted to it,
> one of the consumers is going to be activated in round-robin. The
> spec call this a store-and-forward-queue (section 2.1.1). What If I
> want to have all the consumers alerted about each message posted?
> I figured that I could manage to have each of them
> creating a queue and binding it to a fanout Exchange. Then, for each
> message posted to the Exchange, all the queues would get their copies
> and thus all the consumers would be alerted about it.
> So, is this the right way to implement the scenario
> of having a producer activating N consumers or is there another type
> of exchange/queue that would give me this functionality?
> Thanks in advance,
>
> Acácio
> BODY { font-family:Arial, Helvetica, sans-serif;font-size:12px; }
>
>
--
-----
http://www.globalherald.net/jb01
GlobalHerald.NET, the Smarter Social Network! (tm)
Re: RES: How to properly implement a 1-to-N pattern
Posted by Ted Ross <tr...@redhat.com>.
Luiz Cordeiro wrote:
> Thanks for the answers, they were precise and fast.
>
> We're considering using QPid in a corporate project but there're a couple of points I would like to make clearer:
>
> - Is there a way to specify a list of brokers for the C++ client lib and have the traffic automatically distributed among them? Such that, should one of them go down, the client lib would automatically redistribute the load for the remaining brokers? In other words, is the C++ client lib capable of load balancing and unavailability detection? I've read about the possibility to set a keepalive on the socket, and that the Connection would be cut in the occurrence of 2 failures in the heartbeat process, but that was all that I could find about it on the project's Docs.
>
The Qpid broker (c++) supports clustering whereby multiple broker nodes
are tied together to provide high availability. When using clustering,
a client connects to one of the nodes and will seamlessly fail over to
another if that node fails.
Qpid clustering is targeted at high availability and not load
balancing. The only resource that scales up is connection sockets. If
you are going to try the clustering capability, I recommend that you use
the code from the Subversion trunk. Clustering has been improved
significantly since the last release (0.5).
> - Are there SNMP consumers for QPid's event queue, such that we could monitor QPid by integrating it with our Nagios infrastructure?
>
I'm not sure what you mean by "event queue", but log events can be
directed to Syslog by using the --log-to-syslog command line option. Or
are you asking if the broker can be monitored via SNMP (it cannot
currently).
> Thanks,
> Acácio
-Ted
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
RES: How to properly implement a 1-to-N pattern
Posted by Luiz Cordeiro <Lu...@corp.terra.com.br>.
Thanks for the answers, they were precise and fast.
We're considering using QPid in a corporate project but there're a couple of points I would like to make clearer:
- Is there a way to specify a list of brokers for the C++ client lib and have the traffic automatically distributed among them? Such that, should one of them go down, the client lib would automatically redistribute the load for the remaining brokers? In other words, is the C++ client lib capable of load balancing and unavailability detection? I've read about the possibility to set a keepalive on the socket, and that the Connection would be cut in the occurrence of 2 failures in the heartbeat process, but that was all that I could find about it on the project's Docs.
- Are there SNMP consumers for QPid's event queue, such that we could monitor QPid by integrating it with our Nagios infrastructure?
Thanks,
Acácio.
-----Mensagem original-----
De: Ted Ross [mailto:tross@redhat.com]
Enviada em: quarta-feira, 2 de setembro de 2009 11:56
Para: users@qpid.apache.org
Assunto: Re: How to properly implement a 1-to-N pattern
acacio.centeno@terra.com.br wrote:
>
> *
> *
> *
>
> Hi,
> By what I’ve read on the AMQP specification,
> should I have N consumers to a queue, for each message posted to it,
> one of the consumers is going to be activated in round-robin. The
> spec call this a store-and-forward-queue (section 2.1.1). What If I
> want to have all the consumers alerted about each message posted?
> I figured that I could manage to have each of them
> creating a queue and binding it to a fanout Exchange. Then, for each
> message posted to the Exchange, all the queues would get their copies
> and thus all the consumers would be alerted about it.
> So, is this the right way to implement the scenario
> of having a producer activating N consumers or is there another type
> of exchange/queue that would give me this functionality?
> Thanks in advance,
>
> Acácio
> BODY { font-family:Arial, Helvetica, sans-serif;font-size:12px; }
>
>
>
Yes, that is correct. If multiple consumers are to receive the same
messages, each consumer should have its own queue for a copy of the message.
Note that this is still efficient since the broker only keeps one copy
of the message in memory and places references on the queues.
Regards,
-Ted
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: How to properly implement a 1-to-N pattern
Posted by Ted Ross <tr...@redhat.com>.
acacio.centeno@terra.com.br wrote:
>
> *
> *
> *
>
> Hi,
> By what I’ve read on the AMQP specification,
> should I have N consumers to a queue, for each message posted to it,
> one of the consumers is going to be activated in round-robin. The
> spec call this a store-and-forward-queue (section 2.1.1). What If I
> want to have all the consumers alerted about each message posted?
> I figured that I could manage to have each of them
> creating a queue and binding it to a fanout Exchange. Then, for each
> message posted to the Exchange, all the queues would get their copies
> and thus all the consumers would be alerted about it.
> So, is this the right way to implement the scenario
> of having a producer activating N consumers or is there another type
> of exchange/queue that would give me this functionality?
> Thanks in advance,
>
> Acácio
> BODY { font-family:Arial, Helvetica, sans-serif;font-size:12px; }
>
>
>
Yes, that is correct. If multiple consumers are to receive the same
messages, each consumer should have its own queue for a copy of the message.
Note that this is still efficient since the broker only keeps one copy
of the message in memory and places references on the queues.
Regards,
-Ted
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: How to properly implement a 1-to-N pattern
Posted by Joshua Boyd <jd...@etinternational.com>.
On Wed, 2009-09-02 at 14:33 +0000, acacio.centeno@terra.com.br wrote:
> By what I’ve read on the AMQP specification,
> should I have N consumers to a queue, for each message posted to it,
> one of the consumers is going to be activated in round-robin. The
> spec call this a store-and-forward-queue (section 2.1.1). What If I
> want to have all the consumers alerted about each message posted?
> I figured that I could manage to have each of them
> creating a queue and binding it to a fanout Exchange. Then, for each
> message posted to the Exchange, all the queues would get their copies
> and thus all the consumers would be alerted about it.
> So, is this the right way to implement the scenario
> of having a producer activating N consumers or is there another type
> of exchange/queue that would give me this functionality?
> Thanks in advance,
I'm using the pub/sub exchange for both what you are already doing (N
consumers in a round-robin) and what you want to do (broadcasting to all
N consumers)
For broadcasting to N consumers, they all subscribe to the same routing
key, but have a unique local queue name.
When I want to send a message and have only one of the N consumers get
it in more or less round-robin manner then each consumer has the same
local queue name.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org