You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "Jeremy Ross (Jira)" <ji...@apache.org> on 2021/01/25 21:07:00 UTC
[jira] [Commented] (CAMEL-16078) IdempotentConsumer +
MemoryIdempotentRepository allows duplicates
[ https://issues.apache.org/jira/browse/CAMEL-16078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17271677#comment-17271677 ]
Jeremy Ross commented on CAMEL-16078:
-------------------------------------
I believe Caffeine's Window TinyLfu algorithm is the culprit here. If I exclude camel-caffeine-lrucache, the SimpleLRUCache is used, which seems to have a strict policy of eldest entry is removed. Considering the importance of idempotency and "once and only once" semantics, this is a far more preferable algorithm, even if it's not designed for massive scale.
Considering that Caffeine's LRU cache can drop entries almost as soon as they're added, I don't see this as an appropriate backing for MemoryIdempotencyRepository, unless perhaps if the use case is massive scale, and it's ok if some duplicate messages get through as long as performance is good. Overall, to me, this behavior violates the [Principle of Least Surprise|https://en.wikipedia.org/wiki/Principle_of_least_astonishment]. We might consider, if nothing else, documenting this behavior. Long term it may be better to find a backing that can truly support idempotency. Hoping others can weigh in as I haven't used this part of Camel much.
I'm also curious if excluding camel-caffeine-lrucache has any other side effects for other parts of camel.
> IdempotentConsumer + MemoryIdempotentRepository allows duplicates
> -----------------------------------------------------------------
>
> Key: CAMEL-16078
> URL: https://issues.apache.org/jira/browse/CAMEL-16078
> Project: Camel
> Issue Type: Bug
> Components: came-core
> Affects Versions: 3.7.1
> Reporter: Jeremy Ross
> Priority: Major
>
> In certain scenarios, it seems that IdempotentConsumer is allowing duplicate message to pass. In the test case below, with a cache size of 50, duplicate message within a few messages of each other are allowed to pass.
> Test case: https://gist.github.com/jeremyross/d76f658767b4f476f93a12f9067ed515
--
This message was sent by Atlassian Jira
(v8.3.4#803005)