[jira] [Created] (CAMEL-18985) camel-kafka: messages are getting lost with "breakOnFirstError"

["Oleg Pershai (Jira)" <ji...@apache.org>]

Oleg Pershai created CAMEL-18985:
------------------------------------

             Summary: camel-kafka: messages are getting lost with "breakOnFirstError"
                 Key: CAMEL-18985
                 URL: https://issues.apache.org/jira/browse/CAMEL-18985
             Project: Camel
          Issue Type: Bug
          Components: camel-kafka
    Affects Versions: 3.20.1, 3.20.0, 3.19.0, 3.18.5
            Reporter: Oleg Pershai


*Description:*

Messages are getting lost with "breakOnFirstError=true" when processing of a particular message failed several times in a raw.

 

*Configuration:*
 * autoCommitEnable=false
 * allowManualCommit=true
 * breakOnFirstError=true
 * autoOffsetReset=earliest
 * maxPollRecords is greater than one (e.g. 5 in this test)

 

*Test Scenario:*
 # inbound-topic contains 10 messages: 0,1,2,3,4,5,6,7,8,9
 # camel-route polls 5 messages: 0,1,2,3,4
 # camel-route successfully processes message=0 and manually commits offset
 # camel-route successfully processes message=1 and manually commits offset
 # camel-route fails with processing message=2 first time
 # breakOnFirstError=true causes camel-route to reconnect and poll from the committed offset
 # camel-route polls 5 messages: 2,3,4,5,6
 # camel-route fails with processing message=2 second time
 # *ACTUAL:* camel-route reconnects and polls messages 7,8,9 - as result messages 3,4,5,6 are never processed

*EXPECTED:* camel-route should do the following on the step 9:
 * reconnect and poll the same 5 messages again: 2,3,4,5,6
 * process message=2 third time (the test is configured to succeed on the 3rd attempt)
 * continue with processing messages 3,4,5,6...

 

*GitHub project reproducing the issue:*

[https://github.com/opershai/camel-kafka-lost-messages-demo] 

Test-runs;
 * 3.18.5 (failing): [https://github.com/opershai/camel-kafka-lost-messages-demo/actions/runs/4039702564] 
 * 3.19.0 (failing): [https://github.com/opershai/camel-kafka-lost-messages-demo/actions/runs/4039815226] 
 * 3.20.0 (failing): [https://github.com/opershai/camel-kafka-lost-messages-demo/actions/runs/4039709753] 
 * 3.20.1 (failing): [https://github.com/opershai/camel-kafka-lost-messages-demo/actions/runs/4039673222] 

 

*Impact:*

It seems "breakOnFirstError" is not fully reliable in LTS versions of Camel Kafka component at the moment:
 * 3.20.x - due to this bug
 * 3.18.x - due to this bug
 * 3.14.x - due to another bug https://issues.apache.org/jira/browse/CAMEL-17925 as fixed was not back-ported to the 3.14.x



--
This message was sent by Atlassian Jira
(v8.20.10#820010)