[jira] [Comment Edited] (KAFKA-3068) NetworkClient may connect to a different Kafka cluster than originally configured

["Eno Thereska (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-3068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15085618#comment-15085618 ] 

Eno Thereska edited comment on KAFKA-3068 at 1/6/16 3:15 PM:
-------------------------------------------------------------

[~ijuma], [~hachikuji]: I think a proper solution would make use of a cluster identity in addition to broker id+host. The problems with using bootstrap brokers is that 1) users might not specify enough bootstrap brokers and 2) bootstrap brokers can also move to a different cluster as [~ijuma] points out. While the windows of vulnerability will be small as [~junrao] points out, it will still be a problem. One option would have been to query zookeeper (which is the cluster identity in a way) and get the latest brokers from it instead of caching them in the client. With the new clients that is not an option anymore though.


was (Author: enothereska):
[~ijuma], [~hachikuji]: I think a proper solution would make use of a cluster identity in addition to broker id+host. The problems with using bootstrap brokers is that 1) users might not specify enough bootstrap brokers and 2) bootstrap brokers can also move to a different cluster as [~ijuma] points out. While the windows of vulnerability will be small as [~junrao] points out, it will still be a problem. One option would be to query zookeeper (which is the cluster identity in a way) and get the latest brokers from it instead of caching them in the client.

> NetworkClient may connect to a different Kafka cluster than originally configured
> ---------------------------------------------------------------------------------
>
>                 Key: KAFKA-3068
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3068
>             Project: Kafka
>          Issue Type: Bug
>          Components: clients
>    Affects Versions: 0.9.0.0
>            Reporter: Jun Rao
>
> In https://github.com/apache/kafka/pull/290, we added the logic to cache all brokers (id and ip) that the client has ever seen. If we can't find an available broker from the current Metadata, we will pick a broker that we have ever seen (in NetworkClient.leastLoadedNode()).
> One potential problem this logic can introduce is the following. Suppose that we have a broker with id 1 in a Kafka cluster. A producer client remembers this broker in nodesEverSeen. At some point, we bring down this broker and use the host in a different Kafka cluster. Then, the producer client uses this broker from nodesEverSeen to refresh metadata. It will find the metadata in a different Kafka cluster and start producing data there.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)