You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2021/11/03 08:22:36 UTC
[GitHub] [hadoop] haiyang1987 commented on a change in pull request #3596: HDFS-16287. Support to make dfs.namenode.avoid.read.slow.datanode reconfigurable
haiyang1987 commented on a change in pull request #3596:
URL: https://github.com/apache/hadoop/pull/3596#discussion_r741696525
##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java
##########
@@ -260,17 +257,14 @@
final Timer timer = new Timer();
this.slowPeerTracker = dataNodePeerStatsEnabled ?
new SlowPeerTracker(conf, timer) : null;
- this.excludeSlowNodesEnabled = conf.getBoolean(
- DFS_NAMENODE_BLOCKPLACEMENTPOLICY_EXCLUDE_SLOW_NODES_ENABLED_KEY,
- DFS_NAMENODE_BLOCKPLACEMENTPOLICY_EXCLUDE_SLOW_NODES_ENABLED_DEFAULT);
this.maxSlowPeerReportNodes = conf.getInt(
DFSConfigKeys.DFS_NAMENODE_MAX_SLOWPEER_COLLECT_NODES_KEY,
DFSConfigKeys.DFS_NAMENODE_MAX_SLOWPEER_COLLECT_NODES_DEFAULT);
this.slowPeerCollectionInterval = conf.getTimeDuration(
DFSConfigKeys.DFS_NAMENODE_SLOWPEER_COLLECT_INTERVAL_KEY,
DFSConfigKeys.DFS_NAMENODE_SLOWPEER_COLLECT_INTERVAL_DEFAULT,
TimeUnit.MILLISECONDS);
- if (slowPeerTracker != null && excludeSlowNodesEnabled) {
Review comment:
@tomscut Thank you for your review.
1.Current parameter 'dataNodePeerStatsEnabled' and 'excludeSlowNodesEnabled' decision SlowPeerCollector thread whether to start ,But it didn't take into account avoid SlowDataNodesForRead logic
2.So think about two phases:
a.The first is to start SlowPeerCollector thread
b.Second, you can control whether to enable read/write avoid slow datanode according to dynamic parameters
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org