You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by "La Fleur, Sebastiaan" <se...@ns.nl.INVALID> on 2021/06/03 15:18:47 UTC

RE: Question: How can I achieve the 'exactly-once' and 'at-least-once' guarantee using Python proton?

Thank you Robbie!

In the mean time I have been able to extend our implementation (based on Python Proton) with the at_least_once settling mechanism. We are now caching the unsettled messages using the delivery tag according to the AMQP 1.0 specification and retrying all of them whenever a link is opened and there are unsettled messages.

Am I correct that the unsettled map in the attach frame from the AMQP 1.0 specification is currently not exposed or implemented by the Proton client?
Source: http://docs.oasis-open.org/amqp/core/v1.0/amqp-core-complete-v1.0.pdf
Page 65, section 2.7.3 Attach:     <field name="unsettled" type="map"/>

Am I also correct that any resume link functionality currently is not implemented?
Same document, page 49, 2.63. Establishing or Resuming A Link

I have been unable to find these implementations in the (python) proton client so I would like to confirm if they are indeed missing or I overlooked a resource.

Thanks in advance!


Regards,

Sebastiaan la Fleur

-----Original Message-----
From: Robbie Gemmell <ro...@gmail.com>
Sent: vrijdag 23 april 2021 12:02
To: users <us...@qpid.apache.org>
Subject: Re: Question: How can I achieve the 'exactly-once' and 'at-least-once' guarantee using Python proton?

You can't do exactly-once, it requires complicated interactions neither the clients or servers support.

I dont believe the python client can ever re-send a message itself, as it doesnt retain messages once actually sent, but the 'buffer'
question is perhaps nuanced depending on what you meant by that precisely, and possibly also whether you are using the built in connection reconnect. Essentially when the client has actually sent things then it no longer has the payload and so it would be up to you to resend if required, but if for example it has a message to send that hasnt actually been transmitted yet (e.g send call was made before credit was given, or flow control meant it hadnt yet gone onto the wire, socket wasnt writable etc) before a disconnect, then that can mean it is still effectively not-sent and it can then be after it reconnected itself. If you dont want that you would handle disconnects yourself, recreate your sender, and begin sending again.

On Thu, 22 Apr 2021 at 15:34, La Fleur, Sebastiaan <se...@ns.nl.invalid> wrote:
>
> Dear all,
>
> Thank you for this software. Currently I am investigating proton as our AMQP 1.0 library of choice and as such I have reached a number of questions I hope you can help me with.
> To maintain simplicity on the mailing list index, I will try to keep 1 question to a single subject (area).
>
> With AMQP you can use the accepting and settling of messages to achieve 3 delivery guarantees: at-least once, at-most once and exactly-once.
> For the former 2, I have found the relevant Link options:
> https://qpid.apache.org/releases/qpid-proton-0.34.0/proton/python/docs
> /proton.reactor.html#proton.reactor.AtLeastOnce
>
> For the latter, exactly-once, I have been unable to find any pre-existing mechanisms in the API documentation except a small explanation of the steps associated with this guarantee:
> http://qpid.apache.org/releases/qpid-proton-0.34.0/proton/python/docs/
> overview.html#delivery-guarantees
>
> Can you confirm there is no mechanism for achieving the exactly-once guarantee? If not, how would I achieve this?
> Can you also confirm proton will NOT buffer any messages or resend them with the At-Least-once guarantee and these concerns are up to the user of the library?
>
> Thank you in advance!
>
>
> Sincere regards,
>
> Sebastiaan la Fleur
>
>
> ________________________________
>
> Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor (gebruik door) de geadresseerde. De e-mail kan persoonlijke of vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail (en eventuele bijlagen) te vernietigen.
>
> Informatie vennootschap<http://www.ns.nl/emaildisclaimer>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org For additional commands, e-mail: users-help@qpid.apache.org


________________________________

Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor (gebruik door) de geadresseerde. De e-mail kan persoonlijke of vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail (en eventuele bijlagen) te vernietigen.

Informatie vennootschap<http://www.ns.nl/emaildisclaimer>

RE: Question: How can I achieve the 'exactly-once' and 'at-least-once' guarantee using Python proton?

Posted by "La Fleur, Sebastiaan" <se...@ns.nl.INVALID>.
Cheers, thanks again for a very quick reply Robbie!

Have a nice night!


Regards,

Sebastiaan la Fleur

-----Original Message-----
From: Robbie Gemmell <ro...@gmail.com>
Sent: donderdag 3 juni 2021 17:45
To: users <us...@qpid.apache.org>
Subject: Re: Question: How can I achieve the 'exactly-once' and 'at-least-once' guarantee using Python proton?

It is not exposed no, as it is really only used for the exactly-once scenario which isnt supported.

For at-least-once, you would simply resend all messages for which an outcome was not received before the link/connection is lost. You typically wouldnt even need to know the delivery tag to do that, just have some list of 'unconfirmed' messages to send.

On Thu, 3 Jun 2021 at 16:19, La Fleur, Sebastiaan <se...@ns.nl.invalid> wrote:
>
> Thank you Robbie!
>
> In the mean time I have been able to extend our implementation (based on Python Proton) with the at_least_once settling mechanism. We are now caching the unsettled messages using the delivery tag according to the AMQP 1.0 specification and retrying all of them whenever a link is opened and there are unsettled messages.
>
> Am I correct that the unsettled map in the attach frame from the AMQP 1.0 specification is currently not exposed or implemented by the Proton client?
> Source: http://docs.oasis-open.org/amqp/core/v1.0/amqp-core-complete-v1.0.pdf
> Page 65, section 2.7.3 Attach:     <field name="unsettled" type="map"/>
>
> Am I also correct that any resume link functionality currently is not implemented?
> Same document, page 49, 2.63. Establishing or Resuming A Link
>
> I have been unable to find these implementations in the (python) proton client so I would like to confirm if they are indeed missing or I overlooked a resource.
>
> Thanks in advance!
>
>
> Regards,
>
> Sebastiaan la Fleur
>
> -----Original Message-----
> From: Robbie Gemmell <ro...@gmail.com>
> Sent: vrijdag 23 april 2021 12:02
> To: users <us...@qpid.apache.org>
> Subject: Re: Question: How can I achieve the 'exactly-once' and 'at-least-once' guarantee using Python proton?
>
> You can't do exactly-once, it requires complicated interactions neither the clients or servers support.
>
> I dont believe the python client can ever re-send a message itself, as it doesnt retain messages once actually sent, but the 'buffer'
> question is perhaps nuanced depending on what you meant by that precisely, and possibly also whether you are using the built in connection reconnect. Essentially when the client has actually sent things then it no longer has the payload and so it would be up to you to resend if required, but if for example it has a message to send that hasnt actually been transmitted yet (e.g send call was made before credit was given, or flow control meant it hadnt yet gone onto the wire, socket wasnt writable etc) before a disconnect, then that can mean it is still effectively not-sent and it can then be after it reconnected itself. If you dont want that you would handle disconnects yourself, recreate your sender, and begin sending again.
>
> On Thu, 22 Apr 2021 at 15:34, La Fleur, Sebastiaan <se...@ns.nl.invalid> wrote:
> >
> > Dear all,
> >
> > Thank you for this software. Currently I am investigating proton as our AMQP 1.0 library of choice and as such I have reached a number of questions I hope you can help me with.
> > To maintain simplicity on the mailing list index, I will try to keep 1 question to a single subject (area).
> >
> > With AMQP you can use the accepting and settling of messages to achieve 3 delivery guarantees: at-least once, at-most once and exactly-once.
> > For the former 2, I have found the relevant Link options:
> > https://qpid.apache.org/releases/qpid-proton-0.34.0/proton/python/do
> > cs /proton.reactor.html#proton.reactor.AtLeastOnce
> >
> > For the latter, exactly-once, I have been unable to find any pre-existing mechanisms in the API documentation except a small explanation of the steps associated with this guarantee:
> > http://qpid.apache.org/releases/qpid-proton-0.34.0/proton/python/doc
> > s/
> > overview.html#delivery-guarantees
> >
> > Can you confirm there is no mechanism for achieving the exactly-once guarantee? If not, how would I achieve this?
> > Can you also confirm proton will NOT buffer any messages or resend them with the At-Least-once guarantee and these concerns are up to the user of the library?
> >
> > Thank you in advance!
> >
> >
> > Sincere regards,
> >
> > Sebastiaan la Fleur
> >
> >
> > ________________________________
> >
> > Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor (gebruik door) de geadresseerde. De e-mail kan persoonlijke of vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail (en eventuele bijlagen) te vernietigen.
> >
> > Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org For
> additional commands, e-mail: users-help@qpid.apache.org
>
>
> ________________________________
>
> Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor (gebruik door) de geadresseerde. De e-mail kan persoonlijke of vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail (en eventuele bijlagen) te vernietigen.
>
> Informatie vennootschap<http://www.ns.nl/emaildisclaimer>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org For additional commands, e-mail: users-help@qpid.apache.org


________________________________

Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor (gebruik door) de geadresseerde. De e-mail kan persoonlijke of vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail (en eventuele bijlagen) te vernietigen.

Informatie vennootschap<http://www.ns.nl/emaildisclaimer>

Re: Question: How can I achieve the 'exactly-once' and 'at-least-once' guarantee using Python proton?

Posted by Robbie Gemmell <ro...@gmail.com>.
It is not exposed no, as it is really only used for the exactly-once
scenario which isnt supported.

For at-least-once, you would simply resend all messages for which an
outcome was not received before the link/connection is lost. You
typically wouldnt even need to know the delivery tag to do that, just
have some list of 'unconfirmed' messages to send.

On Thu, 3 Jun 2021 at 16:19, La Fleur, Sebastiaan
<se...@ns.nl.invalid> wrote:
>
> Thank you Robbie!
>
> In the mean time I have been able to extend our implementation (based on Python Proton) with the at_least_once settling mechanism. We are now caching the unsettled messages using the delivery tag according to the AMQP 1.0 specification and retrying all of them whenever a link is opened and there are unsettled messages.
>
> Am I correct that the unsettled map in the attach frame from the AMQP 1.0 specification is currently not exposed or implemented by the Proton client?
> Source: http://docs.oasis-open.org/amqp/core/v1.0/amqp-core-complete-v1.0.pdf
> Page 65, section 2.7.3 Attach:     <field name="unsettled" type="map"/>
>
> Am I also correct that any resume link functionality currently is not implemented?
> Same document, page 49, 2.63. Establishing or Resuming A Link
>
> I have been unable to find these implementations in the (python) proton client so I would like to confirm if they are indeed missing or I overlooked a resource.
>
> Thanks in advance!
>
>
> Regards,
>
> Sebastiaan la Fleur
>
> -----Original Message-----
> From: Robbie Gemmell <ro...@gmail.com>
> Sent: vrijdag 23 april 2021 12:02
> To: users <us...@qpid.apache.org>
> Subject: Re: Question: How can I achieve the 'exactly-once' and 'at-least-once' guarantee using Python proton?
>
> You can't do exactly-once, it requires complicated interactions neither the clients or servers support.
>
> I dont believe the python client can ever re-send a message itself, as it doesnt retain messages once actually sent, but the 'buffer'
> question is perhaps nuanced depending on what you meant by that precisely, and possibly also whether you are using the built in connection reconnect. Essentially when the client has actually sent things then it no longer has the payload and so it would be up to you to resend if required, but if for example it has a message to send that hasnt actually been transmitted yet (e.g send call was made before credit was given, or flow control meant it hadnt yet gone onto the wire, socket wasnt writable etc) before a disconnect, then that can mean it is still effectively not-sent and it can then be after it reconnected itself. If you dont want that you would handle disconnects yourself, recreate your sender, and begin sending again.
>
> On Thu, 22 Apr 2021 at 15:34, La Fleur, Sebastiaan <se...@ns.nl.invalid> wrote:
> >
> > Dear all,
> >
> > Thank you for this software. Currently I am investigating proton as our AMQP 1.0 library of choice and as such I have reached a number of questions I hope you can help me with.
> > To maintain simplicity on the mailing list index, I will try to keep 1 question to a single subject (area).
> >
> > With AMQP you can use the accepting and settling of messages to achieve 3 delivery guarantees: at-least once, at-most once and exactly-once.
> > For the former 2, I have found the relevant Link options:
> > https://qpid.apache.org/releases/qpid-proton-0.34.0/proton/python/docs
> > /proton.reactor.html#proton.reactor.AtLeastOnce
> >
> > For the latter, exactly-once, I have been unable to find any pre-existing mechanisms in the API documentation except a small explanation of the steps associated with this guarantee:
> > http://qpid.apache.org/releases/qpid-proton-0.34.0/proton/python/docs/
> > overview.html#delivery-guarantees
> >
> > Can you confirm there is no mechanism for achieving the exactly-once guarantee? If not, how would I achieve this?
> > Can you also confirm proton will NOT buffer any messages or resend them with the At-Least-once guarantee and these concerns are up to the user of the library?
> >
> > Thank you in advance!
> >
> >
> > Sincere regards,
> >
> > Sebastiaan la Fleur
> >
> >
> > ________________________________
> >
> > Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor (gebruik door) de geadresseerde. De e-mail kan persoonlijke of vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail (en eventuele bijlagen) te vernietigen.
> >
> > Informatie vennootschap<http://www.ns.nl/emaildisclaimer>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org For additional commands, e-mail: users-help@qpid.apache.org
>
>
> ________________________________
>
> Deze e-mail, inclusief eventuele bijlagen, is uitsluitend bestemd voor (gebruik door) de geadresseerde. De e-mail kan persoonlijke of vertrouwelijke informatie bevatten. Openbaarmaking, vermenigvuldiging, verspreiding en/of verstrekking van (de inhoud van) deze e-mail (en eventuele bijlagen) aan derden is uitdrukkelijk niet toegestaan. Indien u niet de bedoelde geadresseerde bent, wordt u vriendelijk verzocht degene die de e-mail verzond hiervan direct op de hoogte te brengen en de e-mail (en eventuele bijlagen) te vernietigen.
>
> Informatie vennootschap<http://www.ns.nl/emaildisclaimer>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org