You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Michael Noll (JIRA)" <ji...@apache.org> on 2016/07/05 12:34:11 UTC
[jira] [Updated] (KAFKA-3478) Finer Stream Flow Control
[ https://issues.apache.org/jira/browse/KAFKA-3478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael Noll updated KAFKA-3478:
--------------------------------
Description:
Today we have a event-time based flow control mechanism in order to synchronize multiple input streams in a best effort manner:
http://docs.confluent.io/3.0.0/streams/architecture.html#flow-control-with-timestamps
However, there are some use cases where users would like to have finer control of the input streams, for example, with two input streams, one of them always reading from offset 0 upon (re)-starting, and the other reading for log end offset.
Today we only have one consumer config "offset.auto.reset" to control that behavior, which means all streams are read either from "earliest" or "latest".
We should consider how to improve this settings to allow users have finer control over these frameworks.
was:
Today we have a event-time based flow control mechanism in order to synchronize multiple input streams in a best effort manner:
http://docs.confluent.io/2.1.0-alpha1/streams/architecture.html#flow-control-with-timestamps
However, there are some use cases where users would like to have finer control of the input streams, for example, with two input streams, one of them always reading from offset 0 upon (re)-starting, and the other reading for log end offset.
Today we only have one consumer config "offset.auto.reset" to control that behavior, which means all streams are read either from "earliest" or "latest".
We should consider how to improve this settings to allow users have finer control over these frameworks.
> Finer Stream Flow Control
> -------------------------
>
> Key: KAFKA-3478
> URL: https://issues.apache.org/jira/browse/KAFKA-3478
> Project: Kafka
> Issue Type: Sub-task
> Components: streams
> Reporter: Guozhang Wang
> Labels: user-experience
> Fix For: 0.10.1.0
>
>
> Today we have a event-time based flow control mechanism in order to synchronize multiple input streams in a best effort manner:
> http://docs.confluent.io/3.0.0/streams/architecture.html#flow-control-with-timestamps
> However, there are some use cases where users would like to have finer control of the input streams, for example, with two input streams, one of them always reading from offset 0 upon (re)-starting, and the other reading for log end offset.
> Today we only have one consumer config "offset.auto.reset" to control that behavior, which means all streams are read either from "earliest" or "latest".
> We should consider how to improve this settings to allow users have finer control over these frameworks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)