You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Randall Hauch (JIRA)" <ji...@apache.org> on 2017/04/18 21:05:41 UTC
[jira] [Commented] (KAFKA-3821) Allow Kafka Connect source tasks to
produce offset without writing to topics
[ https://issues.apache.org/jira/browse/KAFKA-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15973506#comment-15973506 ]
Randall Hauch commented on KAFKA-3821:
--------------------------------------
[~criccomini] logged KAFKA-5084, but that is really another situation where the connector would like to provide new offsets without having to produce {{SourceRecord}}. As he says [here|https://confluentcommunity.slack.com/archives/C49L0V3L7/p1492547735471505], the connector sometimes need to tell Kafka Connect:
{quote}
i made some progress in my source, but i don’t want to send any of the data
{quote}
> Allow Kafka Connect source tasks to produce offset without writing to topics
> ----------------------------------------------------------------------------
>
> Key: KAFKA-3821
> URL: https://issues.apache.org/jira/browse/KAFKA-3821
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Affects Versions: 0.9.0.1
> Reporter: Randall Hauch
> Labels: needs-kip
>
> Provide a way for a {{SourceTask}} implementation to record a new offset for a given partition without necessarily writing a source record to a topic.
> Consider a connector task that uses the same offset when producing an unknown number of {{SourceRecord}} objects (e.g., it is taking a snapshot of a database). Once the task completes those records, the connector wants to update the offsets (e.g., the snapshot is complete) but has no more records to be written to a topic. With this change, the task could simply supply an updated offset.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)