[jira] [Commented] (CAMEL-18350) camel-kafka: combination of breakOnFirstError and maxPollRecords causes camel to reconsume all records on error

["Espen Andreassen (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CAMEL-18350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17609931#comment-17609931 ] 

Espen Andreassen commented on CAMEL-18350:
------------------------------------------

I think this bug is somewhat critical. It renders the only technique I know of for approximating "exactly once" useless. As stated in the description, it will cause redelivery of all records on topics in case of an error, which probably is pretty critical for users who have applied an exactly once technique.

Here is a post on stackoverflow demonstrating that this technique probably is used by more than me:
[https://stackoverflow.com/questions/61621755/how-to-implement-exactly-once-kafka-consumer-using-apache-kafka]

> camel-kafka: combination of breakOnFirstError and maxPollRecords causes camel to reconsume all records on error 
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-18350
>                 URL: https://issues.apache.org/jira/browse/CAMEL-18350
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-kafka
>    Affects Versions: 3.18.0
>            Reporter: Espen Andreassen
>            Priority: Minor
>             Fix For: 3.20.0
>
>
> *Summary:*
> When configuring the camel consumer with props "breakOnFirstError" = "true" and "maxPollRecords" = "1", camel reconsumes all messages on topic on error. Camel should only have reconsumed the failed message is such cases.
> I configure the component like this to approximate an "exactly once" delivery policy (in comibnation with manual commits).
> I've created a project on github with a failing test that recreates the issue: [https://github.com/espeandr/camel-kafka-incorrectly-reconsumes-entire-topic-demo]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)