[jira] [Resolved] (KAFKA-6079) Idempotent production for source connectors

["Randall Hauch (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-6079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Randall Hauch resolved KAFKA-6079.
----------------------------------
    Resolution: Won't Fix

Marking this as WONTFIX since it is *already possible since AK 0.11* to configure the Connect worker producers [to use idempotent delivery|https://kafka.apache.org/documentation/#upgrade_11_exactly_once_semantics]:

{quote}
Idempotent delivery ensures that messages are delivered exactly once to a particular topic partition during the lifetime of a single producer.
{quote}

That eliminates duplicate events due to retries, but a failure of a Connector worker might still mean records are written to Kafka but the worker fails before it can commit the latest offsets; when it restarts, it begins from the last committed offsets and this may certainly result in duplicate messages that were written before the failure.

See KAFKA-6080 for changing Connect to support exactly-once semantics for source connectors.

> Idempotent production for source connectors
> -------------------------------------------
>
>                 Key: KAFKA-6079
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6079
>             Project: Kafka
>          Issue Type: New Feature
>          Components: KafkaConnect
>            Reporter: Antony Stubbs
>            Priority: Major
>
> Idempotent production for source connection to reduce duplicates at least from retires.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)