[jira] [Commented] (KAFKA-763) Add an option to replica from the largest offset during unclean leader election

["Jun Rao (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13600157#comment-13600157 ] 

Jun Rao commented on KAFKA-763:
-------------------------------

Thanks for the patch. Some comments:

1. We should try to reuse SimpleConsumer.earliestOrLatestOffset(). Currently, there is one version using broker and another using zkclient. The latter is actually not used. We can remove the latter and modify the former to take a SimpleConsumer, instead of Broker. Then we can reuse this method in SimpleConsumerShell, ReplicaFetcherThread, and ConsumerFetcherThread. The logic in SimpleConsumer.earliestOrLatestOffset() actually has a bug. It doesn't check the error code in the fetch offset response. We should fix that.

2. ReplicaFetcherThread:
2.1 In the following line, the test should be <= and >=, respectively. 
     if (log.logStartOffset < leaderEndOffset && log.logEndOffset > leaderEndOffset) {
In trunk, we probably don't need to check the first condition log.logStartOffset < leaderEndOffset. This is because in trunk, log.truncateTo() handles the case when targetOffset < log.logStartOffset already. It starts a new log with the targetOffset, which is what we want.
2.2 There are two places in the comment that have "the ISR goes down". It should be "the leader goes down".
2.3 The comment for "2. Unclean leader election" is redundant since it's covered by other comments already.
                
> Add an option to replica from the largest offset during unclean leader election
> -------------------------------------------------------------------------------
>
>                 Key: KAFKA-763
>                 URL: https://issues.apache.org/jira/browse/KAFKA-763
>             Project: Kafka
>          Issue Type: Improvement
>          Components: core
>    Affects Versions: 0.8
>            Reporter: Jun Rao
>            Assignee: Swapnil Ghike
>            Priority: Blocker
>              Labels: kafka-0.8, p2
>         Attachments: kafka-763-new-v1.patch, kafka-763_v1.patch
>
>
> If there is an unclean leader election, a follower may have an offset out of the range of the leader. Currently, the follower will delete all its data and refetch from the smallest offset of the leader. It would be useful to add an option to let the follower refetch from the largest offset of the leader since refetching from the smallest offset may take some time.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira