[jira] [Created] (KAFKA-13601) Add option to support sync offset commit in Kafka Connect Sink

["Anil Dasari (Jira)" <ji...@apache.org>]

Anil Dasari created KAFKA-13601:
-----------------------------------

             Summary: Add option to support sync offset commit in Kafka Connect Sink
                 Key: KAFKA-13601
                 URL: https://issues.apache.org/jira/browse/KAFKA-13601
             Project: Kafka
          Issue Type: New Feature
          Components: KafkaConnect
            Reporter: Anil Dasari


Exactly once in s3 connector with scheduled rotation and field partitioner can be achieved with consumer offset sync' commit. 

Currently, WorkerSinkTask committing the consumer offsets asynchronously. 
private void doCommit(Map<TopicPartition, OffsetAndMetadata> offsets, boolean closing, final int seqno) \{
        log.info("{} Committing offsets", this);
        if (closing) \{
            doCommitSync(offsets, seqno);
        } else \{
            OffsetCommitCallback cb = new OffsetCommitCallback() {
                @Override
                public void onComplete(Map<TopicPartition, OffsetAndMetadata> offsets, Exception error) {
                    lastCommittedOffsets = offsets;
                    onCommitCompleted(error, seqno);
                }
            };
            consumer.commitAsync(offsets, cb);
        }
    }
 

Add config to sink to chose sync' offset commit



--
This message was sent by Atlassian Jira
(v8.20.1#820001)