[jira] [Commented] (KAFKA-293) Allow to configure all broker ids at once similar to how zookeeper handles server ids

["Jay Kreps (Commented) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13226547#comment-13226547 ] 

Jay Kreps commented on KAFKA-293:
---------------------------------

The current approach is, admittedly, hacky; but the zookeeper approach is equally hacky if you ask me. I don't think it is good to put any host information in your server config.

If we want to fix this issue I think the right way to go would be to create a zk sequence of ids. Servers would store the broker id in their data directory (maybe we could add a ${data.dir}/meta.properties file to hold this kind of stuff). If a server starts and has no meta.properties file then it would increment the zk id counter and take a new id and store it persistently with its logs. On startup the broker would take its id from this file.

The reason this approach is better is because the node.id is not really configuration. That is if you swap the node.ids for two brokers you effectively corrupt the logs in the eyes of consumers. So the node.id is pinned to a particular set of logs. This would make any manual borking of your node id impossible and takes the node id out of the things the user has to configure.

Thoughts?
                
> Allow to configure all broker ids at once similar to how zookeeper handles server ids
> -------------------------------------------------------------------------------------
>
>                 Key: KAFKA-293
>                 URL: https://issues.apache.org/jira/browse/KAFKA-293
>             Project: Kafka
>          Issue Type: Improvement
>    Affects Versions: 0.7
>            Reporter: Thomas Dudziak
>
> Zookeeper allows to specify all server ids in the same configuration (https://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html#sc_configuration) which has the benefit that the configuration file is the same for all zookeeper instances. A similar approach for Kafka would be quite useful, e.g.
> brokerid.1=<host 1>
> brokerid.2=<host 2>
> etc.
> It'd still require per-instance configuration (myid file in the zookeeper case) but that can be created separately (e.g. by the deployment tool used).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira