You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "mingchao zhao (Jira)" <ji...@apache.org> on 2022/12/21 08:30:00 UTC
[jira] [Commented] (HDDS-3724) BackgroundPipelineCreator should not throw Pipeline creation failed exception due to insufficient nodes
[ https://issues.apache.org/jira/browse/HDDS-3724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17650310#comment-17650310 ]
mingchao zhao commented on HDDS-3724:
-------------------------------------
Ozone 1.3.0 had been released and we currently have more than 600 open issues targeted for 1.3.0. I am moving the target field to 1.4.0.
If there is anything needs to be discussed about the Target Version, Please reach out to me via Apache email or Slack.
> BackgroundPipelineCreator should not throw Pipeline creation failed exception due to insufficient nodes
> -------------------------------------------------------------------------------------------------------
>
> Key: HDDS-3724
> URL: https://issues.apache.org/jira/browse/HDDS-3724
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Hanisha Koneru
> Priority: Major
>
> BackgroundPipelineCreator runs a scheduler to keep creating pipelines if possible. But in a healthy cluster where all the datanodes are part of pipelines and there are no more nodes available for creating new pipelines, the BackgroundPipelineCreator should not log an error message. In this scenario, it is not an error that a pipeline couldn't be created.
> SCM logs were flooded with "Failed to create pipeline" error messages because of this:
> {code:java}
> 2020-06-04 11:03:24,229 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor THREE. Exception: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:05:24,231 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor ONE. Exception: Cannot create pipeline of factor 1 using 0 nodes. Used 1 nodes. Healthy nodes 1
> 2020-06-04 11:05:24,231 [RatisPipelineUtilsThread] WARN org.apache.hadoop.hdds.scm.pipeline.PipelinePlacementPolicy: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:05:24,231 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor THREE. Exception: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:05:40,705 [ReplicationMonitor] INFO org.apache.hadoop.hdds.scm.container.ReplicationManager: Replication Monitor Thread took 0 milliseconds for processing 7 containers.
> 2020-06-04 11:07:24,234 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor ONE. Exception: Cannot create pipeline of factor 1 using 0 nodes. Used 1 nodes. Healthy nodes 1
> 2020-06-04 11:07:24,234 [RatisPipelineUtilsThread] WARN org.apache.hadoop.hdds.scm.pipeline.PipelinePlacementPolicy: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:07:24,234 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor THREE. Exception: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:09:24,242 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor ONE. Exception: Cannot create pipeline of factor 1 using 0 nodes. Used 1 nodes. Healthy nodes 1
> 2020-06-04 11:09:24,242 [RatisPipelineUtilsThread] WARN org.apache.hadoop.hdds.scm.pipeline.PipelinePlacementPolicy: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:09:24,242 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor THREE. Exception: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:10:40,700 [ReplicationMonitor] INFO org.apache.hadoop.hdds.scm.container.ReplicationManager: Replication Monitor Thread took 0 milliseconds for processing 7 containers.
> 2020-06-04 11:11:24,234 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor ONE. Exception: Cannot create pipeline of factor 1 using 0 nodes. Used 1 nodes. Healthy nodes 1
> 2020-06-04 11:11:24,235 [RatisPipelineUtilsThread] WARN org.apache.hadoop.hdds.scm.pipeline.PipelinePlacementPolicy: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:11:24,235 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor THREE. Exception: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:13:24,240 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor ONE. Exception: Cannot create pipeline of factor 1 using 0 nodes. Used 1 nodes. Healthy nodes 1
> 2020-06-04 11:13:24,240 [RatisPipelineUtilsThread] WARN org.apache.hadoop.hdds.scm.pipeline.PipelinePlacementPolicy: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:13:24,240 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor THREE. Exception: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:15:24,242 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor ONE. Exception: Cannot create pipeline of factor 1 using 0 nodes. Used 1 nodes. Healthy nodes 1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org