[jira] [Commented] (KAFKA-4114) Allow for different "auto.offset.reset" strategies for different input streams

["Bill Bejeck (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-4114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15513498#comment-15513498 ] 

Bill Bejeck commented on KAFKA-4114:
------------------------------------

Hi Matthias,  

I've started working on this and wanted to confirm my understanding.  

The addtional parameter for auto-reset will need to be the same for all calls to {{KStreamBuilder.stream}} and {{KStreamBuilder.table}} for a given single instance of {{KStreamBuilder}}, but can differ for separate instances.

For example given two {{KStreamBuilder}} instances {{kb1}} and {{kb2}} you could do the following:

{{kb1.stream("foo", "earliest")}}
{{kb1.stream("bar", "earliest")}}

{{kb2.stream("baz", "latest")}}
{{kb2.table("bar", "latest")}}

but not this: 

{{kb1.stream("foo", "latest")}}
{{kb1.stream("bar", "earliest")}}
as this second case above would reguire multiple consumers in the underlying {{StreamThread}} of the {{KafkaStreams}} instance obtained from using {{kb1}} as a construction parameter

Is this correct?

> Allow for different "auto.offset.reset" strategies for different input streams
> ------------------------------------------------------------------------------
>
>                 Key: KAFKA-4114
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4114
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: streams
>            Reporter: Matthias J. Sax
>            Assignee: Bill Bejeck
>
> Today we only have one consumer config "offset.auto.reset" to control that behavior, which means all streams are read either from "earliest" or "latest".
> However, it would be useful to improve this settings to allow users have finer control over different input stream. For example, with two input streams, one of them always reading from offset 0 upon (re)-starting, and the other reading for log end offset.
> This JIRA requires to extend {{KStreamBuilder}} API for methods {{.stream(...)}} and {{.table(...)}} to add a new parameter that indicate the initial offset to be used.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)