You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "liaoyuxiangqin (JIRA)" <ji...@apache.org> on 2018/12/10 08:33:00 UTC
[jira] [Commented] (HDFS-14133) Add limit of replication streams on
a given node at one time for erasurecode blocks resconstruction
[ https://issues.apache.org/jira/browse/HDFS-14133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16714424#comment-16714424 ]
liaoyuxiangqin commented on HDFS-14133:
---------------------------------------
Hi [~xchen12138] can you help me review this issuse is need to do for ec block resconstrution. thanks!
> Add limit of replication streams on a given node at one time for erasurecode blocks resconstruction
> ----------------------------------------------------------------------------------------------------
>
> Key: HDFS-14133
> URL: https://issues.apache.org/jira/browse/HDFS-14133
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: erasure-coding
> Affects Versions: 3.3.0
> Reporter: liaoyuxiangqin
> Priority: Major
>
> When i testing blocks reconstruction triger by data disk offline, i found that the Replicated blocks reconstruction speed limit by {color:#FF0000}dfs.namenode.replication.work.multiplier.per.iteration{color}
> in namenode compute work, and for a given node the maximum number of outgoing replication streams at one time limit by used {color:#FF0000}dfs.namenode.replication.max-streams{color} and {color:#FF0000}dfs.namenode.replication.max-streams-hard-limit{color}, but for Erasure Coded blocks only limit total blocks to reconstruct one time by {color:#FF0000}dfs.namenode.replication.work.multiplier.per.iteration{color}
> but not limit maximum of outgoing replication streams per node, so that maybe cause node load imbalance, i think also ec need add node limit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org