You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Swapnil Ghike (JIRA)" <ji...@apache.org> on 2013/07/23 01:56:48 UTC
[jira] [Updated] (KAFKA-987) Avoid checkpointing offsets in Kafka
consumer that have not changed since the last commit
[ https://issues.apache.org/jira/browse/KAFKA-987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Swapnil Ghike updated KAFKA-987:
--------------------------------
Attachment: kafka-987.patch
The only case wherein the code does not have a bug and yet the consumer could end up consuming from an offset < the previously committed offset is when there is an unclean leader election. Hence, this patch will commit new offsets as long as they are different from the previously committed offsets.
> Avoid checkpointing offsets in Kafka consumer that have not changed since the last commit
> -----------------------------------------------------------------------------------------
>
> Key: KAFKA-987
> URL: https://issues.apache.org/jira/browse/KAFKA-987
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Reporter: Swapnil Ghike
> Assignee: Swapnil Ghike
> Labels: improvement
> Fix For: 0.8
>
> Attachments: kafka-987.patch
>
>
> We need to fix the Kafka zookeeper consumer to avoid checkpointing offsets that have not changed since the last offset commit. This will help reduce the write load on zookeeper.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira