[jira] [Comment Edited] (KAFKA-1367) Broker topic metadata not kept in sync with ZooKeeper

["Aditya Auradkar (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-1367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577509#comment-14577509 ] 

Aditya Auradkar edited comment on KAFKA-1367 at 6/8/15 5:40 PM:
----------------------------------------------------------------

[~jjkoshy] [~junrao] KAFKA-2225, even if we leave the ISR in the TopicMetadataRequest, how do the consumers detect which of the replicas in ISR to fetch from right? The consumers need to know which "zone" each of the brokers live in and their own in order to fetch from the closest replica (which mitigates with the bandwidth issues described in 2225).

Couple of options:
1. Return it in BrokerMetadataRequest (KIP-24)
2. Piggyback it along with the ISR field in TMR. i.e. isr : {0: "zone1", 1: "zone2"}

If we choose to do (2), then the TMR will evolve anyway.


was (Author: aauradkar):
[~jjkoshy] [~junrao] KAFKA-2225, even if we leave the ISR in the TopicMetadataRequest, how do the consumers detect which of the replicas in ISR to fetch from right? The consumers need to know which "zone" each of the brokers live in and their own in order to fetch from the closest replica (which mitigates with the bandwidth issues described in 2225).

> Broker topic metadata not kept in sync with ZooKeeper
> -----------------------------------------------------
>
>                 Key: KAFKA-1367
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1367
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.8.0, 0.8.1
>            Reporter: Ryan Berdeen
>            Assignee: Ashish K Singh
>              Labels: newbie++
>             Fix For: 0.8.3
>
>         Attachments: KAFKA-1367.txt
>
>
> When a broker is restarted, the topic metadata responses from the brokers will be incorrect (different from ZooKeeper) until a preferred replica leader election.
> In the metadata, it looks like leaders are correctly removed from the ISR when a broker disappears, but followers are not. Then, when a broker reappears, the ISR is never updated.
> I used a variation of the Vagrant setup created by Joe Stein to reproduce this with latest from the 0.8.1 branch: https://github.com/also/kafka/commit/dba36a503a5e22ea039df0f9852560b4fb1e067c



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)