[jira] [Resolved] (KAFKA-14727) Connect EOS mode should periodically call task commit

["Chris Egerton (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-14727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Egerton resolved KAFKA-14727.
-----------------------------------
    Fix Version/s: 3.5.0
       Resolution: Fixed

> Connect EOS mode should periodically call task commit
> -----------------------------------------------------
>
>                 Key: KAFKA-14727
>                 URL: https://issues.apache.org/jira/browse/KAFKA-14727
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions: 3.3.0, 3.4.0, 3.3.1, 3.3.2
>            Reporter: Greg Harris
>            Assignee: Greg Harris
>            Priority: Major
>             Fix For: 3.5.0
>
>
> In non-EOS mode, there is a background thread which periodically commits offsets for a task. If this thread does not have resources to flush on the framework side (records, or offsets) it still calls the task's commit() method to update the internal state of the task.
> In EOS mode, there is no background thread, and all offset commits are performed on the main task thread in response to sending records to Kafka. This has the effect of only triggering the task's commit() method when there are records to send to Kafka, which is different than non-EOS mode.
> In order to bring the two modes into better alignment, and allow tasks reliant on the non-EOS empty commit() behavior to work in EOS mode out-of-the-box, EOS mode should provide offset commits periodically for tasks which do not produce records.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)