You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Ilyushonak Barys <Ba...@troika.ru> on 2011/05/11 13:55:43 UTC
Sender Capacity and error handling
Greetings.
Could you please help me to answer on the following question: how can I handle exceptions while synchronous send?
The docs says:
If the connection fails and is transparently reconnected (see Section 2.10, "Connection Options" for details on how to control this feature), the unsettled messages for each sender over that connection will be re-transmitted.
http://qpid.apache.org/books/0.10/Programming-In-Apache-Qpid/html/ch02s05.html
But: what is happened if the connection do not reconnect? How can I handle such error?
I would like log it in the file.
I use MRG 1.3, C++ broker and C++ client, messaging API.
Regards, Boris
_______________________________________________________
The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia.
If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Sender Capacity and error handling
Posted by Gordon Sim <gs...@redhat.com>.
On 05/13/2011 11:21 AM, Ilyushonak Barys wrote:
> Gordon, thanks again for detailed explanation.
>
>>> 2. I would like to cache qpid connection and session objects to increase producer performance.
>> Not sure what you mean here. You mean you want to avoid recreating a connection/session each time you need it?
> You are right. I use infinite loop and in the loop I would like to avoid recreating a connection/session each time I need to send a message.
>
>> What would recovery consist of here for your case?
> I got execution reports form the exchange and I would like to store message number exactly which was sent to the broker. In the error case I would like to redeliver messages from exchange starting exactly from the number I stored.
In general, if you lose your connection for example, you can't know
exactly which message was the last the broker processed. You have a
window of in-doubt messages (of size 1 to n, depending on you sender
capacity).
That means you may get duplicates on redelivery. In the future we plane
to offer richer functionality in this area. But for now only an
at-least-once delivery is offered.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
RE: Sender Capacity and error handling
Posted by Ilyushonak Barys <Ba...@troika.ru>.
Gordon, thanks again for detailed explanation.
>> 2. I would like to cache qpid connection and session objects to increase producer performance.
>Not sure what you mean here. You mean you want to avoid recreating a connection/session each time you need it?
You are right. I use infinite loop and in the loop I would like to avoid recreating a connection/session each time I need to send a message.
>What would recovery consist of here for your case?
I got execution reports form the exchange and I would like to store message number exactly which was sent to the broker. In the error case I would like to redeliver messages from exchange starting exactly from the number I stored.
Thank you again for your time and detailed explanation. It really helps us.
Regards,
Boris
-----Original Message-----
From: Gordon Sim [mailto:gsim@redhat.com]
Sent: Friday, May 13, 2011 1:33 PM
To: users@qpid.apache.org
Subject: Re: Sender Capacity and error handling
On 05/12/2011 07:40 AM, Ilyushonak Barys wrote:
> Gordon,
>
> thank you very much for the answer. It really helpful.
>
> The case I would like to solve concern message producer written in C++. The main thing - do not loose messages (at least once delivery). The case is the following:
> 1. I would like to use async message sending on producer to increase performance. In this case I expect qpid messaging API cache my messages in memory buffer on the client (producer). I expect that qpid messaging API deliver the messages from the buffer to broker in other thread than my program (producer) works.
Yes, that is what it does.
> 2. I would like to cache qpid connection and session objects to increase producer performance.
Not sure what you mean here. You mean you want to avoid recreating a connection/session each time you need it?
> 3. I would like to use the reconnect=true feature from connection
> settings 4. I would like do not loose messages. That means I need to know the messages exactly delivered to the broker, and the messages which didn't. If something goes wrong I need to redeliver messages not delivered before.
You can track the buffered messages using the available() method on the sender. Messages are removed in order so if you send 4 messages, then when available drops to 3 you know the first messages has been confirmed.
> 5. I would like to determine the situation, when " something goes wrong" to start recovery process.
That involves getting an exception (TransportFailure). Given that you are using reconnect, this would mean that the retry logic has given up.
What would recovery consist of here for your case?
>
> Actually I would like develop assured high performance C++ producer with at least once delivery reliability.
>
> -----Original Message-----
> From: Gordon Sim [mailto:gsim@redhat.com]
> Sent: Wednesday, May 11, 2011 8:09 PM
> To: users@qpid.apache.org
> Subject: Re: Sender Capacity and error handling
>
> On 05/11/2011 02:26 PM, Ilyushonak Barys wrote:
>> Gordon,
>>
>> Thank you very much for quick response. We have decided to use reconnect_timeout and close session on exception.
>> Could you please provide some best practice for error handling with messaging API?
>
> Hard to answer that without more understanding of the specific applications.
>
>> PS. I expected something like callback error handler and so on...
>
> The API doesn't use callbacks at all at present (that style of interaction can be layered on top). Any call on the API may fail if the connection is lost and/or cannot be re-established. Catching these at some level and either exiting or doing some application specific handling would be required.
>
> Sorry I couldn't be more helpful. If I'm missing the point of the question please push back.
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>
> _______________________________________________________
>
> The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia.
> If you need assistance please contact our Contact Center (+7495) 258
> 0500 or go to www.troika.ru/eng/Contacts/system.wbp
>
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
_______________________________________________________
The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia.
If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Sender Capacity and error handling
Posted by Gordon Sim <gs...@redhat.com>.
On 05/12/2011 07:40 AM, Ilyushonak Barys wrote:
> Gordon,
>
> thank you very much for the answer. It really helpful.
>
> The case I would like to solve concern message producer written in C++. The main thing - do not loose messages (at least once delivery). The case is the following:
> 1. I would like to use async message sending on producer to increase performance. In this case I expect qpid messaging API cache my messages in memory buffer on the client (producer). I expect that qpid messaging API deliver the messages from the buffer to broker in other thread than my program (producer) works.
Yes, that is what it does.
> 2. I would like to cache qpid connection and session objects to increase producer performance.
Not sure what you mean here. You mean you want to avoid recreating a
connection/session each time you need it?
> 3. I would like to use the reconnect=true feature from connection settings
> 4. I would like do not loose messages. That means I need to know the messages exactly delivered to the broker, and the messages which didn't. If something goes wrong I need to redeliver messages not delivered before.
You can track the buffered messages using the available() method on the
sender. Messages are removed in order so if you send 4 messages, then
when available drops to 3 you know the first messages has been confirmed.
> 5. I would like to determine the situation, when " something goes wrong" to start recovery process.
That involves getting an exception (TransportFailure). Given that you
are using reconnect, this would mean that the retry logic has given up.
What would recovery consist of here for your case?
>
> Actually I would like develop assured high performance C++ producer with at least once delivery reliability.
>
> -----Original Message-----
> From: Gordon Sim [mailto:gsim@redhat.com]
> Sent: Wednesday, May 11, 2011 8:09 PM
> To: users@qpid.apache.org
> Subject: Re: Sender Capacity and error handling
>
> On 05/11/2011 02:26 PM, Ilyushonak Barys wrote:
>> Gordon,
>>
>> Thank you very much for quick response. We have decided to use reconnect_timeout and close session on exception.
>> Could you please provide some best practice for error handling with messaging API?
>
> Hard to answer that without more understanding of the specific applications.
>
>> PS. I expected something like callback error handler and so on...
>
> The API doesn't use callbacks at all at present (that style of interaction can be layered on top). Any call on the API may fail if the connection is lost and/or cannot be re-established. Catching these at some level and either exiting or doing some application specific handling would be required.
>
> Sorry I couldn't be more helpful. If I'm missing the point of the question please push back.
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
>
> _______________________________________________________
>
> The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia.
> If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp
>
>
>
> ---------------------------------------------------------------------
> Apache Qpid - AMQP Messaging Implementation
> Project: http://qpid.apache.org
> Use/Interact: mailto:users-subscribe@qpid.apache.org
>
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
RE: Sender Capacity and error handling
Posted by Ilyushonak Barys <Ba...@troika.ru>.
Gordon,
thank you very much for the answer. It really helpful.
The case I would like to solve concern message producer written in C++. The main thing - do not loose messages (at least once delivery). The case is the following:
1. I would like to use async message sending on producer to increase performance. In this case I expect qpid messaging API cache my messages in memory buffer on the client (producer). I expect that qpid messaging API deliver the messages from the buffer to broker in other thread than my program (producer) works.
2. I would like to cache qpid connection and session objects to increase producer performance.
3. I would like to use the reconnect=true feature from connection settings
4. I would like do not loose messages. That means I need to know the messages exactly delivered to the broker, and the messages which didn't. If something goes wrong I need to redeliver messages not delivered before.
5. I would like to determine the situation, when " something goes wrong" to start recovery process.
Actually I would like develop assured high performance C++ producer with at least once delivery reliability.
-----Original Message-----
From: Gordon Sim [mailto:gsim@redhat.com]
Sent: Wednesday, May 11, 2011 8:09 PM
To: users@qpid.apache.org
Subject: Re: Sender Capacity and error handling
On 05/11/2011 02:26 PM, Ilyushonak Barys wrote:
> Gordon,
>
> Thank you very much for quick response. We have decided to use reconnect_timeout and close session on exception.
> Could you please provide some best practice for error handling with messaging API?
Hard to answer that without more understanding of the specific applications.
> PS. I expected something like callback error handler and so on...
The API doesn't use callbacks at all at present (that style of interaction can be layered on top). Any call on the API may fail if the connection is lost and/or cannot be re-established. Catching these at some level and either exiting or doing some application specific handling would be required.
Sorry I couldn't be more helpful. If I'm missing the point of the question please push back.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
_______________________________________________________
The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia.
If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Sender Capacity and error handling
Posted by Gordon Sim <gs...@redhat.com>.
On 05/11/2011 02:26 PM, Ilyushonak Barys wrote:
> Gordon,
>
> Thank you very much for quick response. We have decided to use reconnect_timeout and close session on exception.
> Could you please provide some best practice for error handling with messaging API?
Hard to answer that without more understanding of the specific
applications.
> PS. I expected something like callback error handler and so on...
The API doesn't use callbacks at all at present (that style of
interaction can be layered on top). Any call on the API may fail if the
connection is lost and/or cannot be re-established. Catching these at
some level and either exiting or doing some application specific
handling would be required.
Sorry I couldn't be more helpful. If I'm missing the point of the
question please push back.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
RE: Sender Capacity and error handling
Posted by Ilyushonak Barys <Ba...@troika.ru>.
Gordon,
Thank you very much for quick response. We have decided to use reconnect_timeout and close session on exception.
Could you please provide some best practice for error handling with messaging API?
PS. I expected something like callback error handler and so on...
Regards,
Boris
-----Original Message-----
From: Gordon Sim [mailto:gsim@redhat.com]
Sent: Wednesday, May 11, 2011 5:08 PM
To: users@qpid.apache.org
Subject: Re: Sender Capacity and error handling
On 05/11/2011 12:55 PM, Ilyushonak Barys wrote:
> Greetings.
>
> Could you please help me to answer on the following question: how can I handle exceptions while synchronous send?
>
> The docs says:
> If the connection fails and is transparently reconnected (see Section 2.10, "Connection Options" for details on how to control this feature), the unsettled messages for each sender over that connection will be re-transmitted.
> http://qpid.apache.org/books/0.10/Programming-In-Apache-Qpid/html/ch02
> s05.html
>
> But: what is happened if the connection do not reconnect? How can I handle such error?
You can control the reconnection via the various related connection settings (reconnect, reconnect_timeout etc).
If reconnection is disabled, or unsuccessful, loss of a connection is signalled by throwing qpid::messaging::TransportFailure which you can then catch and handle appropriately.
> I would like log it in the file.
Loss of connection is already logged on the client side.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
_______________________________________________________
The information contained in this message may be privileged and conf idential and protected from disclosure. If you are not the original intended recipient, you are hereby notified that any review, retransmission, dissemination, or other use of, or taking of any action in reliance upon, this information is prohibited. If you have received this communication in error, please notify the sender immediately by replying to this message and delete it from your computer. Thank you for your cooperation. Troika Dialog, Russia.
If you need assistance please contact our Contact Center (+7495) 258 0500 or go to www.troika.ru/eng/Contacts/system.wbp
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org
Re: Sender Capacity and error handling
Posted by Gordon Sim <gs...@redhat.com>.
On 05/11/2011 12:55 PM, Ilyushonak Barys wrote:
> Greetings.
>
> Could you please help me to answer on the following question: how can I handle exceptions while synchronous send?
>
> The docs says:
> If the connection fails and is transparently reconnected (see Section 2.10, "Connection Options" for details on how to control this feature), the unsettled messages for each sender over that connection will be re-transmitted.
> http://qpid.apache.org/books/0.10/Programming-In-Apache-Qpid/html/ch02s05.html
>
> But: what is happened if the connection do not reconnect? How can I handle such error?
You can control the reconnection via the various related connection
settings (reconnect, reconnect_timeout etc).
If reconnection is disabled, or unsuccessful, loss of a connection is
signalled by throwing qpid::messaging::TransportFailure which you can
then catch and handle appropriately.
> I would like log it in the file.
Loss of connection is already logged on the client side.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org