[jira] [Commented] (KAFKA-13185) Kafka Connect should clear messageBatch after rewind

["Yunseop Eom (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-13185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18101167#comment-18101167 ] 

Yunseop Eom commented on KAFKA-13185:
-------------------------------------

PR opened: https://github.com/apache/kafka/pull/23048 for KAFKA-13185. Fixed the Kafka Connect sink-task recovery path so a failed preCommit rewind clears both the pending messageBatch and its associated origOffsets before the next poll, preventing stale records and rewound offsets from being delivered again. Added a focused regression test using a retriable put followed by a failed preCommit; the test was RED before the fix and GREEN after it. WorkerSinkTaskTest, connect:runtime Spotless, Checkstyle, SpotBugs, and the full connect:runtime test task passed. Status: PR open and awaiting review.

> Kafka Connect should clear messageBatch after rewind
> ----------------------------------------------------
>
>                 Key: KAFKA-13185
>                 URL: https://issues.apache.org/jira/browse/KAFKA-13185
>             Project: Kafka
>          Issue Type: Bug
>          Components: connect
>            Reporter: Jiri Syrovy
>            Priority: Critical
>
> WorkerSinkTask contains logic to handle RetriableException that happened in either put or flush. If the same Exception happens for longer than poll or timeout interval and it's thrown as well in preCommit/flush it causes offset to rewind back to the last committed position. Meanwhile messageBatch (message buffer) is not cleared and during the next poll it's used passed to the put method. 
> During the next poll the same message is read from the broker and again passed to the put method. 
> This results into duplicated message sent by the connector after RetrieableException.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)