You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Sergey Shelukhin <Se...@microsoft.com.INVALID> on 2020/01/30 02:30:54 UTC

offset reset on unavailability

Hi.
We've run into a situation where Kafka cluster was unstable, but some brokers were still up and responding. 
Some of the consumers restarted at that time and were not able to get their commit offset.
We run with auto.offset.reset earliest by default, for bootstrap; after some time, these consumers reset their commit offset to earliest and started reprocessing a bunch of events.

We are using Confluent.Kafka client.
Is that an expected behavior?
Is there an option to only reset offset on the positive ack that the offset is not stored for this consumer? 
We'd like the cases when the offset cannot be retrieved due to a transient condition to result in retries, or at least a failure.

Re: offset reset on unavailability

Posted by Puneet Saha <sa...@gmail.com>.
unsubscribe

On Wed, Jan 29, 2020 at 6:31 PM Sergey Shelukhin
<Se...@microsoft.com.invalid> wrote:

> Hi.
> We've run into a situation where Kafka cluster was unstable, but some
> brokers were still up and responding.
> Some of the consumers restarted at that time and were not able to get
> their commit offset.
> We run with auto.offset.reset earliest by default, for bootstrap; after
> some time, these consumers reset their commit offset to earliest and
> started reprocessing a bunch of events.
>
> We are using Confluent.Kafka client.
> Is that an expected behavior?
> Is there an option to only reset offset on the positive ack that the
> offset is not stored for this consumer?
> We'd like the cases when the offset cannot be retrieved due to a transient
> condition to result in retries, or at least a failure.
>