Reading messages from start - new job submission

[Avi Levi <av...@bluevoyant.com>]

I'm updating a job without savepoint.
The consumer properties is set to *  prop.setProperty("auto.offset.reset",
"earliest")*
The start strategy is not explicitly set (using the default
setStartFromGroupOffsets).
In this case I expect that the consumer will read the messages from the
beginning since there is no offset restored, but is actually reading from
latest. What am I missing?

Best
Avi

Re: Reading messages from start - new job submission

[Dian Fu <di...@gmail.com>]

Hi Avi,

As described in the documentation: "If offsets could not be found for a partition, the auto.offset.reset setting in the properties will be used.". For starting from GroupOffset, the property "auto.offset.reset" will ONLY be respected when the group offset cannot be found for a partition. Otherwise, it will use the offset found.

Regards,
Dian

> 在 2019年2月20日，上午1:49，avilevi <av...@bluevoyant.com> 写道：
> 
> Thanks for the answer,
> But my question is why do  I need to set
> /myConsumer.setStartFromEarliest();/  if I set this property
> /setProperty("auto.offset.reset", "earliest") /in consumer properties ? 
> I want the consumer to start reading from earliest only If offsets could not
> be found as stated in the documentation.
> isn't that the expected behaviour ?
> 
> Best wishes 
> Avi
> 
> 
> 
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Reading messages from start - new job submission

[avilevi <av...@bluevoyant.com>]

Thanks for the answer,
But my question is why do  I need to set
/myConsumer.setStartFromEarliest();/  if I set this property
/setProperty("auto.offset.reset", "earliest") /in consumer properties ? 
I want the consumer to start reading from earliest only If offsets could not
be found as stated in the documentation.
isn't that the expected behaviour ?

Best wishes 
Avi



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Reading messages from start - new job submission

[avilevi <av...@bluevoyant.com>]

Hi ,
I'm using 1.7.0

Cheers



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Reading messages from start - new job submission

[sohimankotia <so...@gmail.com>]

Hi,

Which version of flink you r using ?

Reset offset to earliest : 

https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/kafka.html#kafka-consumers-start-position-configuration


Thanks




--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/