kafka git commit: MINOR: Documentation for Rack Awareness

[ij...@apache.org]

Repository: kafka
Updated Branches:
  refs/heads/trunk e20eba958 -> 697811551


MINOR: Documentation for Rack Awareness

Author: Ben Stopford <be...@gmail.com>

Reviewers: Ismael Juma <is...@juma.me.uk>

Closes #1369 from benstopford/rack-awareness-docs


Project: http://git-wip-us.apache.org/repos/asf/kafka/repo
Commit: http://git-wip-us.apache.org/repos/asf/kafka/commit/69781155
Tree: http://git-wip-us.apache.org/repos/asf/kafka/tree/69781155
Diff: http://git-wip-us.apache.org/repos/asf/kafka/diff/69781155

Branch: refs/heads/trunk
Commit: 6978115514fc74021e4f2c761402d6b0c954d50c
Parents: e20eba9
Author: Ben Stopford <be...@gmail.com>
Authored: Wed May 11 15:28:18 2016 +0100
Committer: Ismael Juma <is...@juma.me.uk>
Committed: Wed May 11 15:28:18 2016 +0100

----------------------------------------------------------------------
 docs/ops.html | 11 +++++++++++
 1 file changed, 11 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kafka/blob/69781155/docs/ops.html
----------------------------------------------------------------------
diff --git a/docs/ops.html b/docs/ops.html
index f64a701..faf5453 100644
--- a/docs/ops.html
+++ b/docs/ops.html
@@ -98,6 +98,17 @@ Since running this command can be tedious you can also configure Kafka to do thi
     auto.leader.rebalance.enable=true
 </pre>
 
+<h4><a id="basic_ops_racks" href="#basic_ops_racks">Balancing Replicas Across Racks</a></h4>
+The rack awareness feature spreads replicas of the same partition across different racks. This extends the guarantees Kafka provides for broker-failure to cover rack-failure, limiting the risk of data loss should all the brokers on a rack fail at once. The feature can also be applied to other broker groupings such as availability zones in EC2.
+<p></p>
+You can specify that a broker belongs to a particular rack by adding a property to the broker config:
+<pre>   broker.rack=my-rack-id</pre>
+When a topic is <a href="#basic_ops_add_topic">created</a>, <a href="#basic_ops_modify_topic">modified</a> or replicas are <a href="#basic_ops_cluster_expansion">redistributed</a>, the rack constraint will be honoured, ensuring replicas span as many racks as they can (a partition will span min(#racks, replication-factor) different racks).
+<p></p>
+The algorithm used to assign replicas to brokers ensures that the number of leaders per broker will be constant, regardless of how brokers are distributed across racks. This ensures balanced throughput.
+<p></p>
+However if racks are assigned different numbers of brokers, the assignment of replicas will not be even. Racks with fewer brokers will get more replicas, meaning they will use more storage and put more resources into replication. Hence it is sensible to configure an equal number of brokers per rack.
+
 <h4><a id="basic_ops_mirror_maker" href="#basic_ops_mirror_maker">Mirroring data between clusters</a></h4>
 
 We refer to the process of replicating data <i>between</i> Kafka clusters "mirroring" to avoid confusion with the replication that happens amongst the nodes in a single cluster. Kafka comes with a tool for mirroring data between Kafka clusters. The tool reads from a source cluster and writes to a destination cluster, like this: