You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@fineract.apache.org by VICTOR MANUEL ROMERO RODRIGUEZ <vi...@fintecheando.mx> on 2021/11/01 03:43:40 UTC
Re: Idempotency in Fineract
Hello Dev Community,
For this case that James presented a few days ago, this is the proposal. In
this way the suggested http verbs (PUT/POST/PATCH) handled by fineract can
implement the Idempotency and make Fineract a more robust platform having a
way to implement retry mechanism in the client application.
Local testing has been passed. Although it requires a new piece for the
cache (Redis).
This is the link in case of issues while receiving attachments
https://docs.google.com/presentation/d/13lw2JqE9ujHVFwmynjJcVIUA3NfcDkbs729VLgadIXE/edit?usp=sharing
Best regards.
Victor
El vie, 22 oct 2021 a las 12:58, James Dailey (<ja...@gmail.com>)
escribió:
> Devs -
>
> Through a conversation with a computer science person, I've learned that
> Fineract should have a clearly articulated approach with regard to
> idempotency. This is particularly true when thinking about how fineract
> interacts with payment systems.
>
> "When building a system to move money, it is paramount that operations
> that move money are idempotent. Failure to do this might result in
> accidentally double charging your customer, or paying a vendor multiple
> times. The risk of this happening is elevated based on the way software is
> typically built today, since developers take advantage of scalable systems
> that process multiple items in parallel. "
> (
> https://www.moderntreasury.com/journal/why-idempotency-matters-in-payments
> )
>
> Or this article on Stripe engineering calling for liberal use of
> Idempotency:
>
> "The easiest way to address inconsistencies in distributed state caused by
> failures is to implement server endpoints so that they’re idempotent, which
> means that they can be called any number of times while guaranteeing that
> side effects only occur once." (https://stripe.com/blog/idempotency)
>
> and this article in Apache Camel
>
> https://camel.apache.org/camel-kafka-connector/latest/user-guide/idempotency.html
>
> So, my understanding may be wrong and we've got this covered, or maybe we
> just consider this overkill. Perhaps there is something in the "magic"
> that's developed in the CQRS plus in-memory database queuing and our clever
> transaction state management, but... I don't know.
>
> Or perhaps we haven't really had to think about this because we haven't
> had that many large scale deployments and the scalability issues associated
> with multiple parallel calls haven't come to the fore, or haven't come back
> to the list. So, that's the intent of this email.
>
> Do we have a need now for layering on Idempotency checks? What is the
> right approach? Has anyone already implemented this, or decided not to for
> very good reasons? What would be the right approach to this? Implement
> where? Time limited to 3 hrs? Etc...
>
> So, this is a call to discuss. I'm far from an expert on this.
>
>
>
Re: Idempotency in Fineract
Posted by VICTOR MANUEL ROMERO RODRIGUEZ <vi...@fintecheando.mx>.
Hello Fineract Community,
This is an implementation for the previous proposal
https://github.com/apache/fineract/pull/1951
Entire presentation:
https://docs.google.com/presentation/d/13lw2JqE9ujHVFwmynjJcVIUA3NfcDkbs729VLgadIXE/edit?usp=sharing
El dom, 31 oct 2021 a las 21:47, VICTOR MANUEL ROMERO RODRIGUEZ (<
victor.romero@fintecheando.mx>) escribió:
> By the way this is the Jira ticket
>
> https://issues.apache.org/jira/browse/FINERACT-1420
>
> El dom, 31 oct 2021 a las 21:43, VICTOR MANUEL ROMERO RODRIGUEZ (<
> victor.romero@fintecheando.mx>) escribió:
>
>> Hello Dev Community,
>>
>> For this case that James presented a few days ago, this is the proposal.
>> In this way the suggested http verbs (PUT/POST/PATCH) handled by fineract
>> can implement the Idempotency and make Fineract a more robust platform
>> having a way to implement retry mechanism in the client application.
>>
>> Local testing has been passed. Although it requires a new piece for the
>> cache (Redis).
>>
>> This is the link in case of issues while receiving attachments
>>
>>
>> https://docs.google.com/presentation/d/13lw2JqE9ujHVFwmynjJcVIUA3NfcDkbs729VLgadIXE/edit?usp=sharing
>>
>> Best regards.
>>
>> Victor
>>
>> El vie, 22 oct 2021 a las 12:58, James Dailey (<ja...@gmail.com>)
>> escribió:
>>
>>> Devs -
>>>
>>> Through a conversation with a computer science person, I've learned that
>>> Fineract should have a clearly articulated approach with regard to
>>> idempotency. This is particularly true when thinking about how fineract
>>> interacts with payment systems.
>>>
>>> "When building a system to move money, it is paramount that operations
>>> that move money are idempotent. Failure to do this might result in
>>> accidentally double charging your customer, or paying a vendor multiple
>>> times. The risk of this happening is elevated based on the way software is
>>> typically built today, since developers take advantage of scalable systems
>>> that process multiple items in parallel. "
>>> (
>>> https://www.moderntreasury.com/journal/why-idempotency-matters-in-payments
>>> )
>>>
>>> Or this article on Stripe engineering calling for liberal use of
>>> Idempotency:
>>>
>>> "The easiest way to address inconsistencies in distributed state caused
>>> by failures is to implement server endpoints so that they’re idempotent,
>>> which means that they can be called any number of times while guaranteeing
>>> that side effects only occur once." (
>>> https://stripe.com/blog/idempotency)
>>>
>>> and this article in Apache Camel
>>>
>>> https://camel.apache.org/camel-kafka-connector/latest/user-guide/idempotency.html
>>>
>>> So, my understanding may be wrong and we've got this covered, or maybe
>>> we just consider this overkill. Perhaps there is something in the "magic"
>>> that's developed in the CQRS plus in-memory database queuing and our clever
>>> transaction state management, but... I don't know.
>>>
>>> Or perhaps we haven't really had to think about this because we haven't
>>> had that many large scale deployments and the scalability issues associated
>>> with multiple parallel calls haven't come to the fore, or haven't come back
>>> to the list. So, that's the intent of this email.
>>>
>>> Do we have a need now for layering on Idempotency checks? What is the
>>> right approach? Has anyone already implemented this, or decided not to for
>>> very good reasons? What would be the right approach to this? Implement
>>> where? Time limited to 3 hrs? Etc...
>>>
>>> So, this is a call to discuss. I'm far from an expert on this.
>>>
>>>
>>>
Re: Idempotency in Fineract
Posted by VICTOR MANUEL ROMERO RODRIGUEZ <vi...@fintecheando.mx>.
By the way this is the Jira ticket
https://issues.apache.org/jira/browse/FINERACT-1420
El dom, 31 oct 2021 a las 21:43, VICTOR MANUEL ROMERO RODRIGUEZ (<
victor.romero@fintecheando.mx>) escribió:
> Hello Dev Community,
>
> For this case that James presented a few days ago, this is the proposal.
> In this way the suggested http verbs (PUT/POST/PATCH) handled by fineract
> can implement the Idempotency and make Fineract a more robust platform
> having a way to implement retry mechanism in the client application.
>
> Local testing has been passed. Although it requires a new piece for the
> cache (Redis).
>
> This is the link in case of issues while receiving attachments
>
>
> https://docs.google.com/presentation/d/13lw2JqE9ujHVFwmynjJcVIUA3NfcDkbs729VLgadIXE/edit?usp=sharing
>
> Best regards.
>
> Victor
>
> El vie, 22 oct 2021 a las 12:58, James Dailey (<ja...@gmail.com>)
> escribió:
>
>> Devs -
>>
>> Through a conversation with a computer science person, I've learned that
>> Fineract should have a clearly articulated approach with regard to
>> idempotency. This is particularly true when thinking about how fineract
>> interacts with payment systems.
>>
>> "When building a system to move money, it is paramount that operations
>> that move money are idempotent. Failure to do this might result in
>> accidentally double charging your customer, or paying a vendor multiple
>> times. The risk of this happening is elevated based on the way software is
>> typically built today, since developers take advantage of scalable systems
>> that process multiple items in parallel. "
>> (
>> https://www.moderntreasury.com/journal/why-idempotency-matters-in-payments
>> )
>>
>> Or this article on Stripe engineering calling for liberal use of
>> Idempotency:
>>
>> "The easiest way to address inconsistencies in distributed state caused
>> by failures is to implement server endpoints so that they’re idempotent,
>> which means that they can be called any number of times while guaranteeing
>> that side effects only occur once." (https://stripe.com/blog/idempotency)
>>
>>
>> and this article in Apache Camel
>>
>> https://camel.apache.org/camel-kafka-connector/latest/user-guide/idempotency.html
>>
>> So, my understanding may be wrong and we've got this covered, or maybe we
>> just consider this overkill. Perhaps there is something in the "magic"
>> that's developed in the CQRS plus in-memory database queuing and our clever
>> transaction state management, but... I don't know.
>>
>> Or perhaps we haven't really had to think about this because we haven't
>> had that many large scale deployments and the scalability issues associated
>> with multiple parallel calls haven't come to the fore, or haven't come back
>> to the list. So, that's the intent of this email.
>>
>> Do we have a need now for layering on Idempotency checks? What is the
>> right approach? Has anyone already implemented this, or decided not to for
>> very good reasons? What would be the right approach to this? Implement
>> where? Time limited to 3 hrs? Etc...
>>
>> So, this is a call to discuss. I'm far from an expert on this.
>>
>>
>>