You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by xi...@apache.org on 2018/08/14 18:57:25 UTC
hadoop git commit: HDFS-13788. Update EC documentation about rack
fault tolerance. Contributed by Kitti Nanasi.
Repository: hadoop
Updated Branches:
refs/heads/trunk 7e822ec24 -> cede33997
HDFS-13788. Update EC documentation about rack fault tolerance. Contributed by Kitti Nanasi.
Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/cede3399
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/cede3399
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/cede3399
Branch: refs/heads/trunk
Commit: cede33997f7ab09fc046017508b680e282289ce3
Parents: 7e822ec
Author: Xiao Chen <xi...@apache.org>
Authored: Tue Aug 14 11:56:51 2018 -0700
Committer: Xiao Chen <xi...@apache.org>
Committed: Tue Aug 14 11:57:22 2018 -0700
----------------------------------------------------------------------
.../hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hadoop/blob/cede3399/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md
----------------------------------------------------------------------
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md
index 6ae2086..2e8cbbd 100644
--- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md
@@ -107,10 +107,10 @@ Deployment
This means that when reading and writing striped files, most operations are off-rack.
Network bisection bandwidth is thus very important.
- For rack fault-tolerance, it is also important to have at least as many racks as the configured EC stripe width.
- For EC policy RS (6,3), this means minimally 9 racks, and ideally 10 or 11 to handle planned and unplanned outages.
- For clusters with fewer racks than the stripe width, HDFS cannot maintain rack fault-tolerance, but will still attempt
- to spread a striped file across multiple nodes to preserve node-level fault-tolerance.
+ For rack fault-tolerance, it is also important to have enough number of racks, so that on average, each rack holds number of blocks no more than the number of EC parity blocks. A formula to calculate this would be (data blocks + parity blocks) / parity blocks, rounding up.
+ For EC policy RS (6,3), this means minimally 3 racks (calculated by (6 + 3) / 3 = 3), and ideally 9 or more to handle planned and unplanned outages.
+ For clusters with fewer racks than the number of the parity cells, HDFS cannot maintain rack fault-tolerance, but will still attempt
+ to spread a striped file across multiple nodes to preserve node-level fault-tolerance. For this reason, it is recommended to setup racks with similar number of DataNodes.
### Configuration keys
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org