You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Marcin Kuthan (JIRA)" <ji...@apache.org> on 2016/10/06 20:10:20 UTC

[jira] [Commented] (KAFKA-3370) Add options to auto.offset.reset to reset offsets upon initialization only

    [ https://issues.apache.org/jira/browse/KAFKA-3370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15553047#comment-15553047 ] 

Marcin Kuthan commented on KAFKA-3370:
--------------------------------------

As I understand we need two separate properties:

auto.reset.offset: for new topic/partition scenario with two options (keep existing property name/value for backward compatiblity):
* earliest
* latest

auto.reset.offset.existing: for existing topic/partition scenario also with two options:
* nearest - move to earliest if offset too low, move to latest if offset too high
* none - throw an exception

I'm not sure which one should be treated as default, perhaps none because it seems to be safer.

What do you think?

> Add options to auto.offset.reset to reset offsets upon initialization only
> --------------------------------------------------------------------------
>
>                 Key: KAFKA-3370
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3370
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Guozhang Wang
>            Assignee: Vahid Hashemian
>             Fix For: 0.10.2.0
>
>
> Currently "auto.offset.reset" is applied in the following two cases:
> 1) upon starting the consumer for the first time (hence no committed offsets before);
> 2) upon fetching offsets out-of-range.
> For scenarios where case 2) needs to be avoid (i.e. people need to be notified upon offsets out-of-range rather than silently offset reset), "auto.offset.reset" need to be set to "none". However for case 1) setting "auto.offset.reset" to "none" will cause NoOffsetForPartitionException upon polling. And in this case, seekToBeginning/seekToEnd is mistakenly applied trying to set the offset at initialization, which are actually designed for during the life time of the consumer (in rebalance callback, for example).
> The fix proposal is to add two more options to "auto.offset.reset", "earliest-on-start", and "latest-on-start", whose semantics are "earliest" and "latest" for case 1) only, and "none" for case 2).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)