You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Tzu-Li (Gordon) Tai (JIRA)" <ji...@apache.org> on 2016/07/29 16:55:20 UTC
[jira] [Comment Edited] (FLINK-3398) Flink Kafka consumer should
support auto-commit opt-outs
[ https://issues.apache.org/jira/browse/FLINK-3398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15399667#comment-15399667 ]
Tzu-Li (Gordon) Tai edited comment on FLINK-3398 at 7/29/16 4:54 PM:
---------------------------------------------------------------------
Would this be better?:
Have settings probably like {{flink.start-from-offsets}} (or {{flink.starting-position=external-offsets}} as proposed in FLINK-4280) and {{flink.commmit-offsets}}. If these settings are present, the Kafka settings supplied must contain the "group.id".
A specific "group" setting for Flink might not make sense, because the Kafka consumer doesn't really use any consumer group management functionality. The setting is implying that the user wants to trigger a Flink-specific behaviour that depends on a Kafka setting.
Does this make sense?
was (Author: tzulitai):
Would this be better?:
Have settings probably like "flink.start-from-offsets" (or "flink.starting-position=external-offsets" as proposed in FLINK-4280) and "flink.commmit-offsets". If these settings are present, the Kafka settings supplied must contain the "group.id".
A specific "group" setting for Flink might not make sense, because the Kafka consumer doesn't really use any consumer group management functionality. The setting is implying that the user wants to trigger a Flink-specific behaviour that depends on a Kafka setting.
Does this make sense?
> Flink Kafka consumer should support auto-commit opt-outs
> --------------------------------------------------------
>
> Key: FLINK-3398
> URL: https://issues.apache.org/jira/browse/FLINK-3398
> Project: Flink
> Issue Type: Improvement
> Components: Kafka Connector
> Reporter: Shikhar Bhushan
> Fix For: 1.2.0
>
>
> Currently the Kafka source will commit consumer offsets to Zookeeper, either upon a checkpoint if checkpointing is enabled, otherwise periodically based on {{auto.commit.interval.ms}}
> It should be possible to opt-out of committing consumer offsets to Zookeeper. Kafka has this config as {{auto.commit.enable}} (0.8) and {{enable.auto.commit}} (0.9).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)