You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "Otavio Rodolfo Piske (Jira)" <ji...@apache.org> on 2023/01/30 09:05:00 UTC
[jira] [Comment Edited] (CAMEL-18985) camel-kafka: messages are getting lost with "breakOnFirstError"
[ https://issues.apache.org/jira/browse/CAMEL-18985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17681968#comment-17681968 ]
Otavio Rodolfo Piske edited comment on CAMEL-18985 at 1/30/23 9:04 AM:
-----------------------------------------------------------------------
I am assigning this one to me, so I can keep it my radar. But I don't have an ETC yet about when I'll be able to fix it, as I have a lot on the queue right now.
Obs.: anyone with time/interest on working this, just drop a note here and feel free to work on it.
was (Author: orpiske):
I am assigning this one to me, so I can keep it my radar. But I don't have an ETC yet about when I'll be able to fix it, as I have a lot on the queue right now.
> camel-kafka: messages are getting lost with "breakOnFirstError"
> ---------------------------------------------------------------
>
> Key: CAMEL-18985
> URL: https://issues.apache.org/jira/browse/CAMEL-18985
> Project: Camel
> Issue Type: Bug
> Components: camel-kafka
> Affects Versions: 3.18.5, 3.19.0, 3.20.0, 3.20.1
> Reporter: Oleg Pershai
> Assignee: Otavio Rodolfo Piske
> Priority: Major
>
> *Description:*
> Messages are getting lost with "breakOnFirstError=true" when processing of a particular message failed several times in a raw.
>
> *Configuration:*
> * autoCommitEnable=false
> * allowManualCommit=true
> * breakOnFirstError=true
> * autoOffsetReset=earliest
> * maxPollRecords is greater than one (e.g. 5 in this test)
>
> *Test Scenario:*
> # inbound-topic contains 10 messages: 0,1,2,3,4,5,6,7,8,9
> # camel-route polls 5 messages: 0,1,2,3,4
> # camel-route successfully processes message=0 and manually commits offset
> # camel-route successfully processes message=1 and manually commits offset
> # camel-route fails with processing message=2 first time
> # breakOnFirstError=true causes camel-route to reconnect and poll from the committed offset
> # camel-route polls 5 messages: 2,3,4,5,6
> # camel-route fails with processing message=2 second time
> # *ACTUAL:* camel-route reconnects and polls messages 7,8,9 - as result messages 3,4,5,6 are never processed
> *EXPECTED:* camel-route should do the following on the step 9:
> * reconnect and poll the same 5 messages again: 2,3,4,5,6
> * process message=2 third time (the test is configured to succeed on the 3rd attempt)
> * continue with processing messages 3,4,5,6...
>
> *GitHub project reproducing the issue:*
> [https://github.com/opershai/camel-kafka-lost-messages-demo]
> Test-runs;
> * 3.18.5 (failing): [https://github.com/opershai/camel-kafka-lost-messages-demo/actions/runs/4039702564]
> * 3.19.0 (failing): [https://github.com/opershai/camel-kafka-lost-messages-demo/actions/runs/4039815226]
> * 3.20.0 (failing): [https://github.com/opershai/camel-kafka-lost-messages-demo/actions/runs/4039709753]
> * 3.20.1 (failing): [https://github.com/opershai/camel-kafka-lost-messages-demo/actions/runs/4039673222]
>
> *Impact:*
> It seems "breakOnFirstError" is not fully reliable in LTS versions of Camel Kafka component at the moment:
> * 3.20.x - due to this bug
> * 3.18.x - due to this bug
> * 3.14.x - due to another bug https://issues.apache.org/jira/browse/CAMEL-17925 as fix was not back-ported to the 3.14.x
--
This message was sent by Atlassian Jira
(v8.20.10#820010)