You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2026/10/04 14:48:00 UTC
[jira] [Updated] (HDDS-16628) SCMContainerPlacementRackAware gives up before trying the other used node's rack
[ https://issues.apache.org/jira/browse/HDDS-16628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated HDDS-16628:
----------------------------------
Labels: pull-request-available (was: )
> SCMContainerPlacementRackAware gives up before trying the other used node's rack
> --------------------------------------------------------------------------------
>
> Key: HDDS-16628
> URL: https://issues.apache.org/jira/browse/HDDS-16628
> Project: Apache Ozone
> Issue Type: Bug
> Components: SCM
> Reporter: Chu Cheng Li
> Priority: Major
> Labels: pull-request-available
>
> When ReplicationManager adds a replica for a RATIS container whose used replicas are on two racks, {{SCMContainerPlacementRackAware.chooseNode}} looks for a node on the rack of each used node (affinity node) in turn. It picks random nodes and gives up after {{MAX_RETRY}} (3) of them fail {{isValidNode}}. Nodes that are not writable, such as DECOMMISSIONING, DECOMMISSIONED or IN_MAINTENANCE nodes, stay in the network topology until they go dead, and each one picked uses up a retry.
> If the first used node's rack has 3 or more such nodes and no valid node, every attempt runs out of retries on that rack and throws "No satisfied datanode to meet the space constrains". The other used node's rack and the fallback are never tried. The container stays under-replicated, and decommissioning a node that holds one of its replicas never completes. The same happens when the nodes on that rack are out of space.
> An SCM simulation found this: three decommissioned but still running nodes on one rack left 15-26 containers under-replicated and a node stuck in DECOMMISSIONING.
> Fix: give each affinity node's rack its own retries. When they run out, move on to the next affinity node's rack, and then to the fallback. Also reword the error, which blamed space even when the nodes were out of service.
> Tests: two new tests in {{TestSCMContainerPlacementRackAware}}, which fail without the fix.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org