[GitHub] [ozone] JacksonYao287 commented on a change in pull request #2291: HDDS-5278. replication manager will delete over-replicated container with storage utilization priority

[GitBox <gi...@apache.org>]

JacksonYao287 commented on a change in pull request #2291:
URL: https://github.com/apache/ozone/pull/2291#discussion_r642197460



##########
File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ReplicationManager.java
##########
@@ -878,6 +871,24 @@ private void handleOverReplicatedContainer(final ContainerInfo container,
         Set<ContainerReplica> eligibleSet = new HashSet<>(eligibleReplicas);
         ContainerPlacementStatus ps =
             getPlacementStatus(eligibleSet, replicationFactor);
+
+        // Iterate replicas in deterministic order to avoid potential data loss.
+        // See https://issues.apache.org/jira/browse/HDDS-4589.
+        // N.B., sort replicas by (containerID, datanodeDetails).
+        // what is more , for all the deletion candidate replicas,
+        // we always delete the one with highest storage.
+        // See https://issues.apache.org/jira/browse/HDDS-5278
+        eligibleReplicas.sort((r1, r2) -> {
+          int result = nodeManager.getNodeStatInternal(r1.getDatanodeDetails())
+              .compareByRemainingRatio(nodeManager.

Review comment:
       @GlenGeng Thanks for the review! i think you are right and maybe i do not think enough about this problem. Storage usage ratio is a dynamic value, which may change continuously. so if scm1 make a decision to delete CR1 according to the current node storage usage ratio it sees, when scm fail over happens,  scm2 may make another choice according to the up-to-date usage ratio.  i will close this pr for now !




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org