[GitHub] [ozone] lokeshj1703 commented on a diff in pull request #3485: HDDS-6243. sourceToTargetMap in ContainerBalancer doesn't support multiple entries with same source.

[GitBox <gi...@apache.org>]

lokeshj1703 commented on code in PR #3485:
URL: https://github.com/apache/ozone/pull/3485#discussion_r905885380


##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancer.java:
##########
@@ -103,8 +103,18 @@ public class ContainerBalancer extends StatefulService {
   private volatile Thread currentBalancingThread;
   private Lock lock;
   private ContainerBalancerSelectionCriteria selectionCriteria;
-  private Map<DatanodeDetails, ContainerMoveSelection> sourceToTargetMap;
-  private Set<ContainerID> selectedContainers;
+
+  /*
+  Since a container can be selected only once during an iteration, these maps
+   use it as a primary key to track source to target pairings.
+  */
+  private final Map<ContainerID, DatanodeDetails> containerFromSourceMap;
+  private final Map<ContainerID, DatanodeDetails> containerToTargetMap;
+
+  private Set<DatanodeDetails> selectedTargets;
+  private Set<DatanodeDetails> selectedSources;
+  // maintaining a list of containers for testing
+  private final List<ContainerID> selectedContainersList;

Review Comment:
   Do we need this? We can probably get this from containerFromSourceMap and containerToTargetMap in the test.



##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancer.java:
##########
@@ -103,8 +103,18 @@ public class ContainerBalancer extends StatefulService {
   private volatile Thread currentBalancingThread;
   private Lock lock;
   private ContainerBalancerSelectionCriteria selectionCriteria;
-  private Map<DatanodeDetails, ContainerMoveSelection> sourceToTargetMap;
-  private Set<ContainerID> selectedContainers;
+
+  /*
+  Since a container can be selected only once during an iteration, these maps
+   use it as a primary key to track source to target pairings.
+  */
+  private final Map<ContainerID, DatanodeDetails> containerFromSourceMap;

Review Comment:
   Rename to containerToSourceMap



##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancer.java:
##########
@@ -672,31 +679,31 @@ private boolean moveContainer(DatanodeDetails source,
   }
 
   /**
-   * Update targets and selection criteria after a move.
+   * Update targets, sources, and selection criteria after a move.
    *
-   * @param selectedTargets  selected target datanodes
-   * @param moveSelection    the target datanode and container that has been
-   *                         just selected
-   * @param source           the source datanode
-   * @return List of updated potential targets
+   * @param moveSelection the target datanode and container that has been
+   *                      just selected
+   * @param source        the source datanode
    */
   private void updateTargetsAndSelectionCriteria(
-      Set<DatanodeDetails> selectedTargets,
       ContainerMoveSelection moveSelection, DatanodeDetails source) {
     // count source if it has not been involved in move earlier
-    if (!sourceToTargetMap.containsKey(source) &&
-        !selectedTargets.contains(source)) {
+    if (!selectedSources.contains(source)) {
       countDatanodesInvolvedPerIteration += 1;
     }
     // count target if it has not been involved in move earlier
     if (!selectedTargets.contains(moveSelection.getTargetNode())) {
       countDatanodesInvolvedPerIteration += 1;
     }
     incSizeSelectedForMoving(source, moveSelection);
-    sourceToTargetMap.put(source, moveSelection);
+    selectedContainersList.add(moveSelection.getContainerID());
+    containerFromSourceMap.put(moveSelection.getContainerID(), source);
+    containerToTargetMap.put(moveSelection.getContainerID(),
+        moveSelection.getTargetNode());

Review Comment:
   Add a precondition that one container cant be mapped to multiple source/targets.



##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancer.java:
##########
@@ -815,10 +822,12 @@ private void resetState() {
     this.clusterCapacity = 0L;
     this.clusterUsed = 0L;
     this.clusterRemaining = 0L;
-    this.selectedContainers.clear();
     this.overUtilizedNodes.clear();
     this.underUtilizedNodes.clear();
     this.unBalancedNodes.clear();
+    this.containerFromSourceMap.clear();
+    this.containerToTargetMap.clear();
+    this.selectedContainersList.clear();

Review Comment:
   Also reset selectedSources and targets?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org