You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Siddhant Sangwan (Jira)" <ji...@apache.org> on 2026/03/25 06:23:00 UTC
[jira] [Updated] (HDDS-14614) Improve handling for REPLICATION_NOT_HEALTHY_BEFORE_MOVE in Container Balancer
[ https://issues.apache.org/jira/browse/HDDS-14614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Siddhant Sangwan updated HDDS-14614:
------------------------------------
Description:
I've seen cases where a container passes the first replication/deletion check:
{code}
private boolean isContainerReplicatingOrDeleting(ContainerID containerID) {
return replicationManager.isContainerReplicatingOrDeleting(containerID);
}
{code}
but fails later in MoveManager, which does a second check for the same thing in a different way:
{code}
/*
If the container is under, over, or mis replicated, we should let
replication manager solve these issues first. Fail move for such a
container.
*/
ContainerHealthResult healthBeforeMove =
replicationManager.getContainerReplicationHealth(containerInfo,
currentReplicas);
if (healthBeforeMove.getHealthState() !=
ContainerHealthResult.HealthState.HEALTHY) {
ret.complete(MoveResult.REPLICATION_NOT_HEALTHY_BEFORE_MOVE);
return ret;
}
{code}
The SCM leader log will contain "REPLICATION_NOT_HEALTHY_BEFORE_MOVE".
Since the MoveManager check is a stricter check, we should use the same check first in ContainerBalancerSelectionCriteria as well. And regardless, if move fails because of this reason, we still need to add the source datanode back to the priority queue so it can be considered for another move. See https://issues.apache.org/jira/browse/HDDS-7252.
h1. Workaround
The workaround is to exclude all over/under/mis replicated container IDs from balancing using the configuration hdds.container.balancer.exclude.containers. For example, pass a comma separated list of container IDs, like "1, 2, 5, 6".
was:
I've seen cases where a container passes the first replication/deletion check:
{code}
private boolean isContainerReplicatingOrDeleting(ContainerID containerID) {
return replicationManager.isContainerReplicatingOrDeleting(containerID);
}
{code}
but fails later in MoveManager, which does a second check for the same thing in a different way:
{code}
/*
If the container is under, over, or mis replicated, we should let
replication manager solve these issues first. Fail move for such a
container.
*/
ContainerHealthResult healthBeforeMove =
replicationManager.getContainerReplicationHealth(containerInfo,
currentReplicas);
if (healthBeforeMove.getHealthState() !=
ContainerHealthResult.HealthState.HEALTHY) {
ret.complete(MoveResult.REPLICATION_NOT_HEALTHY_BEFORE_MOVE);
return ret;
}
{code}
Since the MoveManager check is a stricter check, we should use the same check first in ContainerBalancerSelectionCriteria as well. And regardless, if move fails because of this reason, we still need to add the source datanode back to the priority queue so it can be considered for another move. See https://issues.apache.org/jira/browse/HDDS-7252.
> Improve handling for REPLICATION_NOT_HEALTHY_BEFORE_MOVE in Container Balancer
> ------------------------------------------------------------------------------
>
> Key: HDDS-14614
> URL: https://issues.apache.org/jira/browse/HDDS-14614
> Project: Apache Ozone
> Issue Type: Improvement
> Components: SCM
> Affects Versions: 2.1.0
> Reporter: Siddhant Sangwan
> Assignee: Siddhant Sangwan
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.2.0
>
>
> I've seen cases where a container passes the first replication/deletion check:
> {code}
> private boolean isContainerReplicatingOrDeleting(ContainerID containerID) {
> return replicationManager.isContainerReplicatingOrDeleting(containerID);
> }
> {code}
> but fails later in MoveManager, which does a second check for the same thing in a different way:
> {code}
> /*
> If the container is under, over, or mis replicated, we should let
> replication manager solve these issues first. Fail move for such a
> container.
> */
> ContainerHealthResult healthBeforeMove =
> replicationManager.getContainerReplicationHealth(containerInfo,
> currentReplicas);
> if (healthBeforeMove.getHealthState() !=
> ContainerHealthResult.HealthState.HEALTHY) {
> ret.complete(MoveResult.REPLICATION_NOT_HEALTHY_BEFORE_MOVE);
> return ret;
> }
> {code}
> The SCM leader log will contain "REPLICATION_NOT_HEALTHY_BEFORE_MOVE".
> Since the MoveManager check is a stricter check, we should use the same check first in ContainerBalancerSelectionCriteria as well. And regardless, if move fails because of this reason, we still need to add the source datanode back to the priority queue so it can be considered for another move. See https://issues.apache.org/jira/browse/HDDS-7252.
> h1. Workaround
> The workaround is to exclude all over/under/mis replicated container IDs from balancing using the configuration hdds.container.balancer.exclude.containers. For example, pass a comma separated list of container IDs, like "1, 2, 5, 6".
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org