You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@helix.apache.org by GitBox <gi...@apache.org> on 2022/02/15 18:27:09 UTC

[GitHub] [helix] NealSun96 commented on a change in pull request #1961: Use final remaining capacity when computing weighted score

NealSun96 commented on a change in pull request #1961:
URL: https://github.com/apache/helix/pull/1961#discussion_r807166800



##########
File path: helix-core/src/main/java/org/apache/helix/controller/rebalancer/waged/constraints/ConstraintBasedAlgorithm.java
##########
@@ -275,6 +262,43 @@ public int compareTo(AssignableReplicaWithScore replica2) {
     }
   }
 
+  /**
+   * Update the overallClusterUtilMap with newly placed replica
+   */
+  private Map<String, Integer> computeFinalClusterRemainingCapacity(ClusterModel clusterModel)
+      throws HelixRebalanceException {
+
+    // compute current remaining capacity.
+    Map<String, Integer> utilizationMap = new HashMap<>();
+    for (AssignableNode node : clusterModel.getAssignableNodes().values()) {
+      for (String capacityKey : node.getMaxCapacity().keySet()) {
+        utilizationMap.compute(capacityKey,
+            (k, v) -> v == null ? node.getRemainingCapacity().get(capacityKey)
+                : v + node.getRemainingCapacity().get(capacityKey));
+      }
+    }
+    // compute final remaining capacity after all replicas are assigned.
+    for (Map.Entry<String, Set<AssignableReplica>> replicasByResource : clusterModel
+        .getAssignableReplicaMap().entrySet()) {
+      for (AssignableReplica replica : replicasByResource.getValue()) {
+        for (Map.Entry<String, Integer> resourceUsage : replica.getCapacity().entrySet()) {
+          int finalRemainCapacity =
+              utilizationMap.get(resourceUsage.getKey()) - resourceUsage.getValue();
+          if (finalRemainCapacity < 0) {
+            // all replicas' assignment will fail if there is one dimension's remain capacity <0.
+            String errorMessage = String.format(
+                "The cluster does not have enough %s capacity for all partitions. ",
+                resourceUsage.getKey());
+            throw new HelixRebalanceException(errorMessage,
+                HelixRebalanceException.Type.FAILED_TO_CALCULATE);
+          }
+          utilizationMap.put(resourceUsage.getKey(), finalRemainCapacity);
+        }
+      }
+    }
+    return utilizationMap;

Review comment:
       @xyuanlu It occurred to me that this entire calculation may be accessed from ClusterContext._estimatedMaxUtilization, which can save us a lot of computation. Could you please verify and assess if we can use that data? 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org