auto.offset.reset for Kafka streams 0.10.2.0

[Mahendra Kariya <ma...@go-jek.com>]

Hey All,

Is the auto offset reset set to "earliest" by default in Kafka streams
0.10.2.0? I thought default was "latest".

I started a new Kafka streams application with a fresh application id and
it started consuming messages from the beginning.

Re: auto.offset.reset for Kafka streams 0.10.2.0

[Mahendra Kariya <ma...@go-jek.com>]

Thanks for the clarification Matthais / Michael!

+1 to clear documentation around this because as far as I remember, default
for normal consumers is "latest" and since Streams internally use normal
consumers, the first intuition is that it will be "latest" for Streams as
well.


Best,
Mahendra



On Tue, Apr 11, 2017 at 12:31 PM, Michael Noll <mi...@confluent.io> wrote:

> It's also documented at
> http://docs.confluent.io/current/streams/developer-guide.html#non-streams-
> configuration-parameters
> .
>
> FYI: We have already begun syncing the Confluent docs for Streams into the
> Apache Kafka docs for Streams, but there's still quite some work left
> (volunteers are welcome :-P).
>
> -Michael
>
>
> On Tue, Apr 11, 2017 at 8:37 AM, Matthias J. Sax <ma...@confluent.io>
> wrote:
>
> > Default for Streams is "earliest"
> >
> > cf.
> > https://github.com/apache/kafka/blob/0.10.2.0/streams/
> > src/main/java/org/apache/kafka/streams/StreamsConfig.java#L405
> >
> >
> > -Matthias
> >
> > On 4/10/17 9:41 PM, Mahendra Kariya wrote:
> > > This was even my assumption. But I had to explicitly specify
> > > auto.offset.reset=latest. Without this config, it started from
> > "earliest"!
> > >
> > > On Tue, Apr 11, 2017 at 10:07 AM, Sachin Mittal <sj...@gmail.com>
> > wrote:
> > >
> > >> As far as I know default is latest, if no offsets are found. Otherwise
> > it
> > >> starts from the offset.
> > >>
> > >>
> > >> On Tue, Apr 11, 2017 at 8:51 AM, Mahendra Kariya <
> > >> mahendra.kariya@go-jek.com
> > >>> wrote:
> > >>
> > >>> Hey All,
> > >>>
> > >>> Is the auto offset reset set to "earliest" by default in Kafka
> streams
> > >>> 0.10.2.0? I thought default was "latest".
> > >>>
> > >>> I started a new Kafka streams application with a fresh application id
> > and
> > >>> it started consuming messages from the beginning.
> > >>>
> > >>
> > >
> >
> >
>

Re: auto.offset.reset for Kafka streams 0.10.2.0

[Michael Noll <mi...@confluent.io>]

It's also documented at
http://docs.confluent.io/current/streams/developer-guide.html#non-streams-configuration-parameters
.

FYI: We have already begun syncing the Confluent docs for Streams into the
Apache Kafka docs for Streams, but there's still quite some work left
(volunteers are welcome :-P).

-Michael


On Tue, Apr 11, 2017 at 8:37 AM, Matthias J. Sax <ma...@confluent.io>
wrote:

> Default for Streams is "earliest"
>
> cf.
> https://github.com/apache/kafka/blob/0.10.2.0/streams/
> src/main/java/org/apache/kafka/streams/StreamsConfig.java#L405
>
>
> -Matthias
>
> On 4/10/17 9:41 PM, Mahendra Kariya wrote:
> > This was even my assumption. But I had to explicitly specify
> > auto.offset.reset=latest. Without this config, it started from
> "earliest"!
> >
> > On Tue, Apr 11, 2017 at 10:07 AM, Sachin Mittal <sj...@gmail.com>
> wrote:
> >
> >> As far as I know default is latest, if no offsets are found. Otherwise
> it
> >> starts from the offset.
> >>
> >>
> >> On Tue, Apr 11, 2017 at 8:51 AM, Mahendra Kariya <
> >> mahendra.kariya@go-jek.com
> >>> wrote:
> >>
> >>> Hey All,
> >>>
> >>> Is the auto offset reset set to "earliest" by default in Kafka streams
> >>> 0.10.2.0? I thought default was "latest".
> >>>
> >>> I started a new Kafka streams application with a fresh application id
> and
> >>> it started consuming messages from the beginning.
> >>>
> >>
> >
>
>

Re: auto.offset.reset for Kafka streams 0.10.2.0

["Matthias J. Sax" <ma...@confluent.io>]

Default for Streams is "earliest"

cf.
https://github.com/apache/kafka/blob/0.10.2.0/streams/src/main/java/org/apache/kafka/streams/StreamsConfig.java#L405


-Matthias

On 4/10/17 9:41 PM, Mahendra Kariya wrote:
> This was even my assumption. But I had to explicitly specify
> auto.offset.reset=latest. Without this config, it started from "earliest"!
> 
> On Tue, Apr 11, 2017 at 10:07 AM, Sachin Mittal <sj...@gmail.com> wrote:
> 
>> As far as I know default is latest, if no offsets are found. Otherwise it
>> starts from the offset.
>>
>>
>> On Tue, Apr 11, 2017 at 8:51 AM, Mahendra Kariya <
>> mahendra.kariya@go-jek.com
>>> wrote:
>>
>>> Hey All,
>>>
>>> Is the auto offset reset set to "earliest" by default in Kafka streams
>>> 0.10.2.0? I thought default was "latest".
>>>
>>> I started a new Kafka streams application with a fresh application id and
>>> it started consuming messages from the beginning.
>>>
>>
> 

Re: auto.offset.reset for Kafka streams 0.10.2.0

[Mahendra Kariya <ma...@go-jek.com>]

This was even my assumption. But I had to explicitly specify
auto.offset.reset=latest. Without this config, it started from "earliest"!

On Tue, Apr 11, 2017 at 10:07 AM, Sachin Mittal <sj...@gmail.com> wrote:

> As far as I know default is latest, if no offsets are found. Otherwise it
> starts from the offset.
>
>
> On Tue, Apr 11, 2017 at 8:51 AM, Mahendra Kariya <
> mahendra.kariya@go-jek.com
> > wrote:
>
> > Hey All,
> >
> > Is the auto offset reset set to "earliest" by default in Kafka streams
> > 0.10.2.0? I thought default was "latest".
> >
> > I started a new Kafka streams application with a fresh application id and
> > it started consuming messages from the beginning.
> >
>

Re: auto.offset.reset for Kafka streams 0.10.2.0

[Sachin Mittal <sj...@gmail.com>]

As far as I know default is latest, if no offsets are found. Otherwise it
starts from the offset.


On Tue, Apr 11, 2017 at 8:51 AM, Mahendra Kariya <mahendra.kariya@go-jek.com
> wrote:

> Hey All,
>
> Is the auto offset reset set to "earliest" by default in Kafka streams
> 0.10.2.0? I thought default was "latest".
>
> I started a new Kafka streams application with a fresh application id and
> it started consuming messages from the beginning.
>