[GitHub] [ozone] sodonnel commented on a diff in pull request #4556: HDDS-8369. Decommissioning with rack aware placement policy does not replicate to correct rack.

["sodonnel (via GitHub)" <gi...@apache.org>]

sodonnel commented on code in PR #4556:
URL: https://github.com/apache/ozone/pull/4556#discussion_r1162950612


##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/SCMContainerPlacementRackAware.java:
##########
@@ -200,6 +200,11 @@ protected List<DatanodeDetails> chooseDatanodesInternal(
       // are on different racks.
       for (int i = 0; i < excludedNodesCount; i++) {
         for (int j = i + 1; j < excludedNodesCount; j++) {
+          if (excludedNodes.get(j).isDecomissioned()) {

Review Comment:
   When picking new nodes, you have to pass in all the nodes that are not allowed to be used for a new replica, and that should include all the nodes that have a replica already, as they cannot be used for a new replica. The issue here, is that SCMContainerPlacementRackAware is using "excluded" nodes to mean something like "existing nodes I want to replace", but nodes could be excluded for other reasons too, such as being overloaded etc.
   
   We changed the interface to pass "usedNodes" and "ExcludedNodes" separately, but until https://issues.apache.org/jira/browse/HDDS-7226 is implemented, the usedNodes are not used in SCMContainerPlacementRackAware.
   
   Looking around the code, it is not strictly needed to exclude a decommissioning node, as the placement policies check if any selected node is valid, and one of those checks is the ensure the node is IN_SERVICE.
   
   Therefore I don't think we need the check for "isDecommissioned" here. Either we should fix the placement policy to use usedNodes correctly, or we should just not pass the decommissioned node in the exclude list to begin with.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org