You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@kafka.apache.org by "Sagar Rao (Jira)" <ji...@apache.org> on 2023/08/02 11:17:00 UTC

[jira] [Created] (KAFKA-15296) Allow committing offsets for Dropped records via SMTs

Sagar Rao created KAFKA-15296:
---------------------------------

             Summary: Allow committing offsets for Dropped records via SMTs
                 Key: KAFKA-15296
                 URL: https://issues.apache.org/jira/browse/KAFKA-15296
             Project: Kafka
          Issue Type: Bug
          Components: KafkaConnect
            Reporter: Sagar Rao
            Assignee: Sagar Rao


Currently the connect Runtime doesn't commit the offsets of records which have been dropped due to SMT. This can lead to issues if the dropped record's partition reflects a source partition and the connector depends upon the committed offsets to make progress. In such cases, the connector might just stall. We should enable committing offsets for dropped records as well. Note that today if a record is dropped because exactly-once support is enabled and the connector chose to abort the batch containing the record, then its offset is still committed. So there already exists a discrepancy in the way the runtime treats these dropped records.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)