You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2024/11/01 13:54:00 UTC

[jira] [Commented] (HDFS-17550) EC: Support decommissioning DataNode by EC block reconstruction

    [ https://issues.apache.org/jira/browse/HDFS-17550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17894853#comment-17894853 ] 

ASF GitHub Bot commented on HDFS-17550:
---------------------------------------

zhengchenyu opened a new pull request, #7142:
URL: https://github.com/apache/hadoop/pull/7142

   
   ### Description of PR
   
   By enabling reconstruction to decommissioning datanode, the speed increased by 12 times on a small-scale cluster with only 100 machines dedicated to storing EC data. I think in prodcution cluster with thousands of clusters, there will be hundreds of times growth.
   
   
   ### How was this patch tested?
   
   unit test and test in cluster
   
   ### For code changes:
   
   - [x] Support decommissioning DataNode by EC block reconstruction.
   - [x] `liveBusyBlockIndices` rename to `liveAndDecommissioningBusyIndices`.
   - [x] remove `excludeReconstructedIndices` which is duplicate with `liveAndDecommissioningBusyIndices`
   - [x] add new `liveAndDecommissioningBusyIndices` which is used when reconstruct the busy decommissioning block.
   
   




> EC: Support decommissioning DataNode by EC block reconstruction
> ---------------------------------------------------------------
>
>                 Key: HDFS-17550
>                 URL: https://issues.apache.org/jira/browse/HDFS-17550
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Chenyu Zheng
>            Priority: Major
>
> By enabling reconstruction to decommissioning datanode, the speed increased by 12 times on a small-scale cluster with only 100 machines dedicated to storing EC data. I think in prodcution cluster with thousands of clusters, there will be hundreds of times growth.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org