[GitHub] [hadoop] RuinanGu commented on a diff in pull request #4252: HDFS-16566 Erasure Coding: Recovery may causes excess replicas when busy DN exsits

[GitBox <gi...@apache.org>]

RuinanGu commented on code in PR #4252:
URL: https://github.com/apache/hadoop/pull/4252#discussion_r863162214


##########
hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/erasurecoding.proto:
##########
@@ -107,7 +107,8 @@ message BlockECReconstructionInfoProto {
   required StorageUuidsProto targetStorageUuids = 4;
   required StorageTypesProto targetStorageTypes = 5;
   required bytes liveBlockIndices = 6;
-  required ErasureCodingPolicyProto ecPolicy = 7;
+  required bytes excludeReconstructedIndices = 7;

Review Comment:
   Thanks for review! Should I respect the order of the old parameters and add the new ones to the last to make it compatible?



##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/erasurecode/StripedReconstructionInfo.java:
##########
@@ -41,26 +41,28 @@ public class StripedReconstructionInfo {
   private final DatanodeInfo[] targets;
   private final StorageType[] targetStorageTypes;
   private final String[] targetStorageIds;
+  private final byte[] excludeReconstructedIndices;
 
   public StripedReconstructionInfo(ExtendedBlock blockGroup,
       ErasureCodingPolicy ecPolicy, byte[] liveIndices, DatanodeInfo[] sources,
       byte[] targetIndices) {
     this(blockGroup, ecPolicy, liveIndices, sources, targetIndices, null,
-        null, null);
+        null, null, new byte[0]);
   }
 
   StripedReconstructionInfo(ExtendedBlock blockGroup,
       ErasureCodingPolicy ecPolicy, byte[] liveIndices, DatanodeInfo[] sources,
       DatanodeInfo[] targets, StorageType[] targetStorageTypes,
-      String[] targetStorageIds) {
+      String[] targetStorageIds, byte[] excludeReconstructedIndices) {

Review Comment:
   When the DN receive the command of EC reconstruction, we can get BlockECReconstructionCommand-> BlockECReconstructionInfo->StripedReconstructionInfo->StripedBlockReconstructor->StripedWriter, and StripedWriter.initTargetIndices() need to be modified to exclude the indices of busy DN.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org