You are viewing a plain text version of this content. The canonical link for it is here.

Posted to jira@kafka.apache.org by "Chris Egerton (Jira)" <ji...@apache.org> on 2022/08/04 15:55:00 UTC

[jira] [Resolved] (KAFKA-10000) Atomic commit of source connector records and offsets

     [ https://issues.apache.org/jira/browse/KAFKA-10000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris Egerton resolved KAFKA-10000.
-----------------------------------
    Resolution: Done

> Atomic commit of source connector records and offsets
> -----------------------------------------------------
>
>                 Key: KAFKA-10000
>                 URL: https://issues.apache.org/jira/browse/KAFKA-10000
>             Project: Kafka
>          Issue Type: New Feature
>          Components: KafkaConnect
>            Reporter: Chris Egerton
>            Assignee: Chris Egerton
>            Priority: Blocker
>              Labels: needs-kip
>             Fix For: 3.3.0
>
>
> It'd be nice to be able to configure source connectors such that their offsets are committed if and only if all records up to that point have been ack'd by the producer. This would go a long way towards EOS for source connectors.
>  
> This differs from https://issues.apache.org/jira/browse/KAFKA-6079, which is marked as {{WONTFIX}} since it only concerns enabling the idempotent producer for source connectors and is not concerned with source connector offsets.
> This also differs from https://issues.apache.org/jira/browse/KAFKA-6080, which had a lot of discussion around allowing connector-defined transaction boundaries. The suggestion in this ticket is to only use source connector offset commits as the transaction boundaries for connectors; allowing connector-specified transaction boundaries can be addressed separately.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)