[jira] [Created] (KAFKA-5319) Add a tool to make cluster replica and leader balance

["Ma Tianchi (JIRA)" <ji...@apache.org>]

Ma Tianchi created KAFKA-5319:
---------------------------------

             Summary: Add a tool to make cluster replica and leader balance
                 Key: KAFKA-5319
                 URL: https://issues.apache.org/jira/browse/KAFKA-5319
             Project: Kafka
          Issue Type: Improvement
          Components: admin
    Affects Versions: 0.10.2.1
            Reporter: Ma Tianchi
         Attachments: ClusterBalanceCommand.scala

When a new broker is added to cluster,there is not any topics in the new broker.When we use console command to create a topic without 'replicaAssignmentOpt',Kafka use AdminUtils.assignReplicasToBrokers to get replicaAssignment.Even though it is balance at the creating time if the cluster never change,with more and more brokers added to cluster the replica balanced will become not well. We also can use 'kafka-reassign-partitions.sh' to balance ,but the big amount of topics make it not easy.And at the topic created time , Kafka choose a  PreferredReplicaLeader which be put at the first position of  the AR to make leader balance.But the balance will be destroyed when cluster changed.Using  'kafka-reassign-partitions.sh' to make partition reassignment may be also destroy the leader balance ,because user can change the AR of the partition . It may be not balanced , but Kafka believe cluster leader balance is well with every leaders is the first on at AR.
So we create a tool to make the number of replicas and number of leaders on every brokers is balanced.It uses a algorithm to get a balanced replica and leader  reassignment,then uses ReassignPartitionsCommand to real balance the cluster.
It can be used to make balance when cluster added brokers or cluster is not balanced .It does not deal with moving replicas of a dead broker to an alive broker,it only makes replicas and leaders on alive brokers is balanced.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)