[jira] [Commented] (KAFKA-5696) SourceConnector does not commit offset on rebalance

["Victor Chicu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-5696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16291456#comment-16291456 ] 

Victor Chicu commented on KAFKA-5696:
-------------------------------------

Any news? I have the same issue in case of rebalance if task is killed before offsets are committed Kafka Connect does not commit on stop partitions and offsets from source record to the OffsetStorage. In this case the connector take once again the same records from external storage and hence duplicate the records in the topic. :(

> SourceConnector does not commit offset on rebalance
> ---------------------------------------------------
>
>                 Key: KAFKA-5696
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5696
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>            Reporter: Oleg Kuznetsov
>            Priority: Critical
>              Labels: newbie
>
> I'm running SourceConnector, that reads files from storage and put data in kafka. I want, in case of reconfiguration, offsets to be flushed. 
> Say, a file is completely processed, but source records are not yet committed and in case of reconfiguration their offsets might be missing in store.
> Is it possible to force committing offsets on reconfiguration?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)