You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "Oleg Pershai (Jira)" <ji...@apache.org> on 2023/04/25 21:50:00 UTC
[jira] [Commented] (CAMEL-18985) camel-kafka: messages are getting lost with "breakOnFirstError"
[ https://issues.apache.org/jira/browse/CAMEL-18985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17716461#comment-17716461 ]
Oleg Pershai commented on CAMEL-18985:
--------------------------------------
Hi [~orpiske]
I've created [pull request|https://github.com/apache/camel/pull/9932]. Could you please take a look whether the suggested fix makes sense for you?
Thank you
> camel-kafka: messages are getting lost with "breakOnFirstError"
> ---------------------------------------------------------------
>
> Key: CAMEL-18985
> URL: https://issues.apache.org/jira/browse/CAMEL-18985
> Project: Camel
> Issue Type: Bug
> Components: camel-kafka
> Affects Versions: 3.18.5, 3.19.0, 3.20.0, 3.20.1
> Reporter: Oleg Pershai
> Assignee: Otavio Rodolfo Piske
> Priority: Major
>
> *Description:*
> Messages are getting lost with "breakOnFirstError=true" when processing of a particular message failed several times in a raw.
>
> *Configuration:*
> * autoCommitEnable=false
> * allowManualCommit=true
> * breakOnFirstError=true
> * autoOffsetReset=earliest
> * maxPollRecords is greater than one (e.g. 5 in this test)
>
> *Test Scenario:*
> # inbound-topic contains 10 messages: 0,1,2,3,4,5,6,7,8,9
> # camel-route polls 5 messages: 0,1,2,3,4
> # camel-route successfully processes message=0 and manually commits offset
> # camel-route successfully processes message=1 and manually commits offset
> # camel-route fails with processing message=2 first time
> # breakOnFirstError=true causes camel-route to reconnect and poll from the committed offset
> # camel-route polls 5 messages: 2,3,4,5,6
> # camel-route fails with processing message=2 second time
> # *ACTUAL:* camel-route reconnects and polls messages 7,8,9 - as result messages 3,4,5,6 are never processed
> *EXPECTED:* camel-route should do the following on the step 9:
> * reconnect and poll the same 5 messages again: 2,3,4,5,6
> * process message=2 third time (the test is configured to succeed on the 3rd attempt)
> * continue with processing messages 3,4,5,6...
>
> *GitHub project reproducing the issue:*
> [https://github.com/opershai/camel-kafka-lost-messages-demo]
> Test-runs:
> * 3.18.5 (failing): [https://github.com/opershai/camel-kafka-lost-messages-demo/actions/runs/4039702564]
> * 3.19.0 (failing): [https://github.com/opershai/camel-kafka-lost-messages-demo/actions/runs/4039815226]
> * 3.20.0 (failing): [https://github.com/opershai/camel-kafka-lost-messages-demo/actions/runs/4039709753]
> * 3.20.1 (failing): [https://github.com/opershai/camel-kafka-lost-messages-demo/actions/runs/4039673222]
> * 2.25.4 (passing): [https://github.com/opershai/camel-kafka-lost-messages-demo/actions/runs/4057277373]
>
> *Impact:*
> It seems "breakOnFirstError" is not fully reliable in LTS versions of Camel Kafka component at the moment:
> * 3.20.x - due to this bug
> * 3.18.x - due to this bug
> * 3.14.x - due to another bug https://issues.apache.org/jira/browse/CAMEL-17925 as fix was not back-ported to the 3.14.x
--
This message was sent by Atlassian Jira
(v8.20.10#820010)