You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Yang Yun (Jira)" <ji...@apache.org> on 2021/03/15 07:00:00 UTC
[jira] [Created] (HDFS-15896) Add used percert limitation to
BlockPlacementPolicyDefault for choosing DataNode to write
Yang Yun created HDFS-15896:
-------------------------------
Summary: Add used percert limitation to BlockPlacementPolicyDefault for choosing DataNode to write
Key: HDFS-15896
URL: https://issues.apache.org/jira/browse/HDFS-15896
Project: Hadoop HDFS
Issue Type: Improvement
Components: block placement
Environment: {code:java}
{code}
Reporter: Yang Yun
Assignee: Yang Yun
Add used percent limitation to BlockPlacementPolicyDefault for choosing datanode to write.
The logical is similar with avoid stale node.
Default is disable, the high used percent is 100.0%
{code:java}
public static final String
DFS_NAMENODE_HIGH_USED_DATANODE_FOR_WRITE_PERCENT_KEY =
"dfs.namenode.high-used.datanode.for.wirte.percent";
public static final float
DFS_NAMENODE_HIGH_USED_DATANODE_FOR_WRITE_PERCENT_DEFAULT =
100.0f;{code}
The choosing will try two times, if the first failed for high used datanode, it will try again without high used limitation.
Also add a high used ratio, when the percentage of high used datanodes reaches this ratio, allow writing to high used nodes to prevent hotspots.
{code:java}
public static final String
DFS_NAMENODE_HIGH_USED_DATANODE_FOR_WRITE_RATIO_KEY =
"dfs.namenode.high-used.datanode.for.wirte.ratio";
public static final float
DFS_NAMENODE_HIGH_USED_DATANODE_FOR_WRITE_RATIO_DEFAULT = 0.6f;{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org