[jira] [Reopened] (KAFKA-5827) Allow configuring Kafka sink connectors to start processing records from the end of topics

["Greg Harris (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-5827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Greg Harris reopened KAFKA-5827:
--------------------------------

> Allow configuring Kafka sink connectors to start processing records from the end of topics
> ------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-5827
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5827
>             Project: Kafka
>          Issue Type: Improvement
>          Components: KafkaConnect
>            Reporter: Behrang Saeedzadeh
>            Priority: Major
>
> As far as I can see, Kafka connectors start exporting data of a topic from the beginning of its partitions. We have a topic that contains a few million old records that we don't need but we would like to start exporting new records that are added to the topic.
> Basically:
> * When the connector is started for the first time and it does not have a current offset stored, it should start consuming data from the end of topic partitions
> * When the connector is restarted and has a current offset for partitions stored somewhere, it should start from those offsets



--
This message was sent by Atlassian Jira
(v8.20.10#820010)