You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Swapnil Ghike (JIRA)" <ji...@apache.org> on 2013/07/23 01:56:48 UTC

[jira] [Updated] (KAFKA-987) Avoid checkpointing offsets in Kafka consumer that have not changed since the last commit

     [ https://issues.apache.org/jira/browse/KAFKA-987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Swapnil Ghike updated KAFKA-987:
--------------------------------

    Attachment: kafka-987.patch

The only case wherein the code does not have a bug and yet the consumer could end up consuming from an offset < the previously committed offset is when there is an unclean leader election. Hence, this patch will commit new offsets as long as they are different from the previously committed offsets.
                
> Avoid checkpointing offsets in Kafka consumer that have not changed since the last commit
> -----------------------------------------------------------------------------------------
>
>                 Key: KAFKA-987
>                 URL: https://issues.apache.org/jira/browse/KAFKA-987
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.8
>            Reporter: Swapnil Ghike
>            Assignee: Swapnil Ghike
>              Labels: improvement
>             Fix For: 0.8
>
>         Attachments: kafka-987.patch
>
>
> We need to fix the Kafka zookeeper consumer to avoid checkpointing offsets that have not changed since the last offset commit. This will help reduce the write load on zookeeper.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira