message replication and consumer balancing

[Paweł Pierścionek <pa...@voiceworks.pl>]

Hi,

I am new to the MQ stuff. Tried parsing through AMQP specs and QPID  
docs and still don't get how to code this little two scenarios:

I have a single producer sending msgs to a queue with dynamic number  
of consumers.

scenario A - all consumers get all messages
scenario B - consumers "compete" for messages - each msg gets consumed  
by not more than one consumer.

Also how the picture changes for a durable version of scenario A -  
where number of consumers is constant and I want to guarantee that all  
messages get to all consumers even if one crashes and comes back after  
some time.

My question is what exchange,queue types and options to use for what  
scenario. Should I create a new exchange to handle scenario A or B?

TIA,
Pawel


---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org

Re: message replication and consumer balancing

[Ted Ross <tr...@redhat.com>]

Pawe? Piers'cionek wrote:
> Hi,
>
> I am new to the MQ stuff. Tried parsing through AMQP specs and QPID 
> docs and still don't get how to code this little two scenarios:
>
> I have a single producer sending msgs to a queue with dynamic number 
> of consumers.
>
> scenario A - all consumers get all messages
> scenario B - consumers "compete" for messages - each msg gets consumed 
> by not more than one consumer.
>
> Also how the picture changes for a durable version of scenario A - 
> where number of consumers is constant and I want to guarantee that all 
> messages get to all consumers even if one crashes and comes back after 
> some time.
>
> My question is what exchange,queue types and options to use for what 
> scenario. Should I create a new exchange to handle scenario A or B?
>
> TIA,
> Pawel
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
In scenario A, you want to create a queue per consumer (exclusive and 
auto-delete). Each queue could be bound to a fanout exchange so it 
receives a copy of every message sent to that exchange by the producer. 
You would most likely want to create your own fanout exchange for this.

The durable version of scenario A is similar except that queues must be 
created as durable (and not auto-delete) and the produced messages must 
also be durable. If a consumer disconnects, it would have to remember 
the name of the queue that it created so it could re-subscribe and pick 
up where it left off.

There's another possible pattern for scenario A. Create a single queue 
bound to a fanout exchange and have all consumers use browsing to 
receive messages. Browsing means that messages are "peeked" at but not 
consumed by the receiver and they remain for other receivers to view.

I'm not 100% sure but I think a "browsing" subscription uses 
accept-mode: none and acquire-mode: not-acquired.

Scenario B is done with a single queue where the queue is not exclusive. 
In this case, multiple consumers may subscribe to the queue and each 
message will be consumed by exactly one consumer. A consumer may even 
receive a message and reject it such that it can be consumed by a 
different consumer.

In the multi-queue scenario, you will want to use a fanout exchange. In 
the single queue cases, you can alternatively use a direct exchange 
where the binding key is the name of the queue (or any constant string 
known by the producer).

-Ted


---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org