You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Daryoush Mehrtash <dm...@gmail.com> on 2011/10/28 23:12:54 UTC
How can you do at-least-one in federated broker?
I have a distributed QPID broker scenario that need to make sure messages
are received at-least-once in the remote clients. I am trying to see if I
have to configure the brokers or not.
On the source broker I have:
qpid-config -a SRC_BROKER add queue SRC_QUEUE
On Destination broker I set up the route and queues to read from
> qpid-route queue add DEST_BROKER SRC_BROKER amq.fanout SRC_QUEUE
> qpid-config -a DEST_BROKER add queue DEST_QUEUE
> qpid-config -a DEST_BROKER bind amq.fanout DEST_QUEUE
I like to run the spout and drain apps in the python example.
spout SRC_QUEUE
drain "DEST_QUEUE; {link:{reliability:"at-least-once"}}"
Questions:
a) Do I have to do anything on the "spout" to make sure the message has
at-least-once reliability>
b) Do I have to configure the Destination borker, or the route to make sure
all messages are sent to the broker.
Thanks
Daryoush
Re: How can you do at-least-one in federated broker?
Posted by Gordon Sim <gs...@redhat.com>.
On 11/01/2011 09:09 PM, Daryoush Mehrtash wrote:
> Is there a notion of negative ack? For instance, if the destination
> broker gets a message that is corrupted, can it n-ack it?
There is such a mechanism in AMQP, but the inter-broker routing does not
use it. Brokers can't detect corruption of the message bodies - they are
treated as opaque byte sequences - and they treat any malformed frames
as a client error and terminate the connection.
> General question..... I am confused as to extend to which AQMP defines the
> interactions. It seems that it does defined the client to broker but I am
> not sure about inter-broker protocol. Is the protocol between the brokers
> also defined in AQMP?
The protocol used between the brokers is compliant AMQP 0-10[1]. One
broker is treated as a client of the other (in fact other than for
setting up the connection, each broker views the other as a regular
client[2]). Setting up the links is done through Qpid specific
management commands however.
[1] There is one minor exception to this rule, a defect in the
implementation rather than a concious design choice, whereby the broker
does not attach the session for 'push routes' in a compliant manner.
[2] There is a qpid specific property that is set on a connection to
indicate that it is a connection from another broker. This is used to
allow transfer of message with a userid other than that used for the link.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: How can you do at-least-one in federated broker?
Posted by Daryoush Mehrtash <dm...@gmail.com>.
Is there a notion of negative ack? For instance, if the destination
broker gets a message that is corrupted, can it n-ack it?
General question..... I am confused as to extend to which AQMP defines the
interactions. It seems that it does defined the client to broker but I am
not sure about inter-broker protocol. Is the protocol between the brokers
also defined in AQMP?
Daryoush
On Tue, Nov 1, 2011 at 3:03 AM, Gordon Sim <gs...@redhat.com> wrote:
> On 10/31/2011 09:29 PM, Daryoush Mehrtash wrote:
>
>>
>>> If an ack doesn't make it to the broker and the connection fails then any
>>> in-doubt messages will be requeued and will be redelivered when the route
>>> is re-established.
>>>
>>
>>
>> Would the sender ever terminate the connection because the ack has not
>> been received in some time period?
>>
>
> No, only if two heartbeats are skipped (these are sent every 2 minutes on
> federation links and at present that is not configurable).
>
>
> ------------------------------**------------------------------**---------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.**apache.org<us...@qpid.apache.org>
>
>
--
Re: How can you do at-least-one in federated broker?
Posted by Gordon Sim <gs...@redhat.com>.
On 10/31/2011 09:29 PM, Daryoush Mehrtash wrote:
>>
>> If an ack doesn't make it to the broker and the connection fails then any
>> in-doubt messages will be requeued and will be redelivered when the route
>> is re-established.
>
>
> Would the sender ever terminate the connection because the ack has not
> been received in some time period?
No, only if two heartbeats are skipped (these are sent every 2 minutes
on federation links and at present that is not configurable).
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: How can you do at-least-one in federated broker?
Posted by Daryoush Mehrtash <dm...@gmail.com>.
>
> If an ack doesn't make it to the broker and the connection fails then any
> in-doubt messages will be requeued and will be redelivered when the route
> is re-established.
Would the sender ever terminate the connection because the ack has not
been received in some time period?
daryoush
On Mon, Oct 31, 2011 at 1:10 PM, Gordon Sim <gs...@redhat.com> wrote:
> On 10/31/2011 07:01 PM, Daryoush Mehrtash wrote:
>
>> Thanks Gordon for your comments. It helped me a lot in understanding the
>> pieces. I have a few questions on the details your suggested:
>>
>>
>> You probably want to add in --ack N to the qpid-route command. That will
>>
>>> make transfer acknoweldeged, giving at-least-once guarantees between
>>> brokers.
>>>
>>
>>
>>
>> What would happen if that acks have not made it back to the broker (either
>> lost or inflight)? Would it stop transmission until a previous block
>> is
>> acknowledged?
>>
>
> No, flow control is not currently used between brokers, the source can
> send as many messages as it wants.
>
> If an ack doesn't make it to the broker and the connection fails then any
> in-doubt messages will be requeued and will be redelivered when the route
> is re-established.
>
>
> Are Acks "all-or-nothing" for whole block or is it selective? If an
>> ack
>> is not received, would the whole block be send back or are
>> there potential for intermediate acks and selective retransmission?
>>
>
> Assuming I understand you its all or nothing. I.e. an ack every N messages
> will accept all N messages. Generally the acknowledgement is done as a
> batch.
>
>
> I am potentially running brokers over lines with long delays
>> (e.g. satellites). Are there any timeouts that needs to be adjusted for
>> the brokers so as it wont give up?
>>
>
> Nothing I can think of.
>
>
> It depends on what types of failure you wish the guarantee to survive for.
>>> If you want durability (i.e. the ability to recover messages if either
>>> broker is restarted) then the messages should be sent as persistent (and
>>> all queues they pass through should be made durable).
>>>
>>
>>
>>
>> By "messages should be sent as persistent " do you mean that durable
>> argument is set on the message as in:
>>
>> msg = Message (....., durabule=True, ....)
>>
>> Correct?
>>
>
> Correct.
>
>
> If I understand Qpid correctly, messages are sent to an Exchange and read
>> from a Queue. If send is durable (assuming the stores are also installed)
>> but there is no queue at the time of the send, does this mean they get
>> persisted on the exchange until there is a queue? or will they get
>> discarded?
>>
>
> They will be discarded if there is no queue to route them to and there is
> no alternate-exchange defined for the exchange.
>
>
> Assuming durable message and durable queues and routes, what are
>> the mechanism in Qpid to clean out messages (potentially old messages)
>> from
>> the exchanges, queues and routes?
>>
>
> You really only need to worry about queues as that is the only place
> messages are stored. You can set a TTL on the message to have it expired at
> some point e.g. if the data will be stale.
>
>
> ------------------------------**------------------------------**---------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.**apache.org<us...@qpid.apache.org>
>
>
--
Daryoush
Weblog: http://onfp.blogspot.com/
Re: How can you do at-least-one in federated broker?
Posted by Gordon Sim <gs...@redhat.com>.
On 10/31/2011 07:01 PM, Daryoush Mehrtash wrote:
> Thanks Gordon for your comments. It helped me a lot in understanding the
> pieces. I have a few questions on the details your suggested:
>
>
> You probably want to add in --ack N to the qpid-route command. That will
>> make transfer acknoweldeged, giving at-least-once guarantees between
>> brokers.
>
>
>
> What would happen if that acks have not made it back to the broker (either
> lost or inflight)? Would it stop transmission until a previous block is
> acknowledged?
No, flow control is not currently used between brokers, the source can
send as many messages as it wants.
If an ack doesn't make it to the broker and the connection fails then
any in-doubt messages will be requeued and will be redelivered when the
route is re-established.
> Are Acks "all-or-nothing" for whole block or is it selective? If an ack
> is not received, would the whole block be send back or are
> there potential for intermediate acks and selective retransmission?
Assuming I understand you its all or nothing. I.e. an ack every N
messages will accept all N messages. Generally the acknowledgement is
done as a batch.
> I am potentially running brokers over lines with long delays
> (e.g. satellites). Are there any timeouts that needs to be adjusted for
> the brokers so as it wont give up?
Nothing I can think of.
>> It depends on what types of failure you wish the guarantee to survive for.
>> If you want durability (i.e. the ability to recover messages if either
>> broker is restarted) then the messages should be sent as persistent (and
>> all queues they pass through should be made durable).
>
>
>
> By "messages should be sent as persistent " do you mean that durable
> argument is set on the message as in:
>
> msg = Message (....., durabule=True, ....)
>
> Correct?
Correct.
> If I understand Qpid correctly, messages are sent to an Exchange and read
> from a Queue. If send is durable (assuming the stores are also installed)
> but there is no queue at the time of the send, does this mean they get
> persisted on the exchange until there is a queue? or will they get
> discarded?
They will be discarded if there is no queue to route them to and there
is no alternate-exchange defined for the exchange.
> Assuming durable message and durable queues and routes, what are
> the mechanism in Qpid to clean out messages (potentially old messages) from
> the exchanges, queues and routes?
You really only need to worry about queues as that is the only place
messages are stored. You can set a TTL on the message to have it expired
at some point e.g. if the data will be stale.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: How can you do at-least-one in federated broker?
Posted by Daryoush Mehrtash <dm...@gmail.com>.
Thanks Gordon for your comments. It helped me a lot in understanding the
pieces. I have a few questions on the details your suggested:
You probably want to add in --ack N to the qpid-route command. That will
> make transfer acknoweldeged, giving at-least-once guarantees between
> brokers.
What would happen if that acks have not made it back to the broker (either
lost or inflight)? Would it stop transmission until a previous block is
acknowledged?
Are Acks "all-or-nothing" for whole block or is it selective? If an ack
is not received, would the whole block be send back or are
there potential for intermediate acks and selective retransmission?
I am potentially running brokers over lines with long delays
(e.g. satellites). Are there any timeouts that needs to be adjusted for
the brokers so as it wont give up?
> It depends on what types of failure you wish the guarantee to survive for.
> If you want durability (i.e. the ability to recover messages if either
> broker is restarted) then the messages should be sent as persistent (and
> all queues they pass through should be made durable).
By "messages should be sent as persistent " do you mean that durable
argument is set on the message as in:
msg = Message (....., durabule=True, ....)
Correct?
If I understand Qpid correctly, messages are sent to an Exchange and read
from a Queue. If send is durable (assuming the stores are also installed)
but there is no queue at the time of the send, does this mean they get
persisted on the exchange until there is a queue? or will they get
discarded?
Assuming durable message and durable queues and routes, what are
the mechanism in Qpid to clean out messages (potentially old messages) from
the exchanges, queues and routes?
Thanks,
Daryoush
On Mon, Oct 31, 2011 at 4:28 AM, Gordon Sim <gs...@redhat.com> wrote:
> On 10/28/2011 10:12 PM, Daryoush Mehrtash wrote:
>
>> I have a distributed QPID broker scenario that need to make sure messages
>> are received at-least-once in the remote clients. I am trying to see if
>> I
>> have to configure the brokers or not.
>>
>> On the source broker I have:
>>
>> qpid-config -a SRC_BROKER add queue SRC_QUEUE
>>
>>
>> On Destination broker I set up the route and queues to read from
>>
>> qpid-route queue add DEST_BROKER SRC_BROKER amq.fanout SRC_QUEUE
>>>
>>
> You probably want to add in --ack N to the qpid-route command. That will
> make transfer acknoweldeged, giving at-least-once guarantees between
> brokers.
>
> [The N determines the frequency of acknowledgements; i.e. an
> acknowledgement will be sent every N messages. Lower values will reduce the
> size of the in-doubt window, higher values (up to a point, say around 200
> or so) will generally increase throughput of the link.]
>
>
> qpid-config -a DEST_BROKER add queue DEST_QUEUE
>>> qpid-config -a DEST_BROKER bind amq.fanout DEST_QUEUE
>>>
>>
>>
>> I like to run the spout and drain apps in the python example.
>>
>> spout SRC_QUEUE
>>
>> drain "DEST_QUEUE; {link:{reliability:"at-least-**once"}}"
>>
>>
>> Questions:
>>
>>
>> a) Do I have to do anything on the "spout" to make sure the message has
>> at-least-once reliability>
>>
>
> It depends on what types of failure you wish the guarantee to survive for.
> If you want durability (i.e. the ability to recover messages if either
> broker is restarted) then the messages should be sent as persistent (and
> all queues they pass through should be made durable).
>
> The spout example does not at present have a command line option for
> setting persistence but that is easy to add.
>
> Apart from that you want to specify --reconnect to have it attempt to
> re-establish connection to the broker if lost and re-send any in-doubt
> messages.
>
> For the queues you would add --durable to the qpid-config command for
> SRC_QUEUE and durable:True alongside the reliability option in your drain
> address.
>
> [You also need to have the store plugin loaded].
>
>
> b) Do I have to configure the Destination borker, or the route to make
>> sure
>> all messages are sent to the broker.
>>
>
> Make sure you have acknowldeged transfer enabled between brokers and
> reconnect enabled on your publisher (spout in this case).
>
> ------------------------------**------------------------------**---------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.**apache.org<us...@qpid.apache.org>
>
>
--
Daryoush
Weblog: http://onfp.blogspot.com/
Re: How can you do at-least-one in federated broker?
Posted by Gordon Sim <gs...@redhat.com>.
On 10/28/2011 10:12 PM, Daryoush Mehrtash wrote:
> I have a distributed QPID broker scenario that need to make sure messages
> are received at-least-once in the remote clients. I am trying to see if I
> have to configure the brokers or not.
>
> On the source broker I have:
>
> qpid-config -a SRC_BROKER add queue SRC_QUEUE
>
>
> On Destination broker I set up the route and queues to read from
>
>> qpid-route queue add DEST_BROKER SRC_BROKER amq.fanout SRC_QUEUE
You probably want to add in --ack N to the qpid-route command. That will
make transfer acknoweldeged, giving at-least-once guarantees between
brokers.
[The N determines the frequency of acknowledgements; i.e. an
acknowledgement will be sent every N messages. Lower values will reduce
the size of the in-doubt window, higher values (up to a point, say
around 200 or so) will generally increase throughput of the link.]
>> qpid-config -a DEST_BROKER add queue DEST_QUEUE
>> qpid-config -a DEST_BROKER bind amq.fanout DEST_QUEUE
>
>
> I like to run the spout and drain apps in the python example.
>
> spout SRC_QUEUE
>
> drain "DEST_QUEUE; {link:{reliability:"at-least-once"}}"
>
>
> Questions:
>
>
> a) Do I have to do anything on the "spout" to make sure the message has
> at-least-once reliability>
It depends on what types of failure you wish the guarantee to survive
for. If you want durability (i.e. the ability to recover messages if
either broker is restarted) then the messages should be sent as
persistent (and all queues they pass through should be made durable).
The spout example does not at present have a command line option for
setting persistence but that is easy to add.
Apart from that you want to specify --reconnect to have it attempt to
re-establish connection to the broker if lost and re-send any in-doubt
messages.
For the queues you would add --durable to the qpid-config command for
SRC_QUEUE and durable:True alongside the reliability option in your
drain address.
[You also need to have the store plugin loaded].
> b) Do I have to configure the Destination borker, or the route to make sure
> all messages are sent to the broker.
Make sure you have acknowldeged transfer enabled between brokers and
reconnect enabled on your publisher (spout in this case).
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org