You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Ethan Rose (Jira)" <ji...@apache.org> on 2021/10/20 20:40:07 UTC
[jira] [Updated] (HDDS-3724) BackgroundPipelineCreator should not
throw Pipeline creation failed exception due to insufficient nodes
[ https://issues.apache.org/jira/browse/HDDS-3724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ethan Rose updated HDDS-3724:
-----------------------------
Target Version/s: 1.3.0 (was: 1.2.0)
I am managing the 1.2.0 release and we currently have more than 600 issues targeted for 1.2.0. I am moving the target field to 1.3.0.
If you are actively working on this jira and believe this should be targeted for the 1.2.0 release, Please reach out to me via Apache email or Slack.
> BackgroundPipelineCreator should not throw Pipeline creation failed exception due to insufficient nodes
> -------------------------------------------------------------------------------------------------------
>
> Key: HDDS-3724
> URL: https://issues.apache.org/jira/browse/HDDS-3724
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Hanisha Koneru
> Priority: Major
>
> BackgroundPipelineCreator runs a scheduler to keep creating pipelines if possible. But in a healthy cluster where all the datanodes are part of pipelines and there are no more nodes available for creating new pipelines, the BackgroundPipelineCreator should not log an error message. In this scenario, it is not an error that a pipeline couldn't be created.
> SCM logs were flooded with "Failed to create pipeline" error messages because of this:
> {code:java}
> 2020-06-04 11:03:24,229 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor THREE. Exception: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:05:24,231 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor ONE. Exception: Cannot create pipeline of factor 1 using 0 nodes. Used 1 nodes. Healthy nodes 1
> 2020-06-04 11:05:24,231 [RatisPipelineUtilsThread] WARN org.apache.hadoop.hdds.scm.pipeline.PipelinePlacementPolicy: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:05:24,231 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor THREE. Exception: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:05:40,705 [ReplicationMonitor] INFO org.apache.hadoop.hdds.scm.container.ReplicationManager: Replication Monitor Thread took 0 milliseconds for processing 7 containers.
> 2020-06-04 11:07:24,234 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor ONE. Exception: Cannot create pipeline of factor 1 using 0 nodes. Used 1 nodes. Healthy nodes 1
> 2020-06-04 11:07:24,234 [RatisPipelineUtilsThread] WARN org.apache.hadoop.hdds.scm.pipeline.PipelinePlacementPolicy: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:07:24,234 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor THREE. Exception: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:09:24,242 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor ONE. Exception: Cannot create pipeline of factor 1 using 0 nodes. Used 1 nodes. Healthy nodes 1
> 2020-06-04 11:09:24,242 [RatisPipelineUtilsThread] WARN org.apache.hadoop.hdds.scm.pipeline.PipelinePlacementPolicy: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:09:24,242 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor THREE. Exception: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:10:40,700 [ReplicationMonitor] INFO org.apache.hadoop.hdds.scm.container.ReplicationManager: Replication Monitor Thread took 0 milliseconds for processing 7 containers.
> 2020-06-04 11:11:24,234 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor ONE. Exception: Cannot create pipeline of factor 1 using 0 nodes. Used 1 nodes. Healthy nodes 1
> 2020-06-04 11:11:24,235 [RatisPipelineUtilsThread] WARN org.apache.hadoop.hdds.scm.pipeline.PipelinePlacementPolicy: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:11:24,235 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor THREE. Exception: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:13:24,240 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor ONE. Exception: Cannot create pipeline of factor 1 using 0 nodes. Used 1 nodes. Healthy nodes 1
> 2020-06-04 11:13:24,240 [RatisPipelineUtilsThread] WARN org.apache.hadoop.hdds.scm.pipeline.PipelinePlacementPolicy: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:13:24,240 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor THREE. Exception: Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 1.
> 2020-06-04 11:15:24,242 [RatisPipelineUtilsThread] ERROR org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager: Failed to create pipeline of type RATIS and factor ONE. Exception: Cannot create pipeline of factor 1 using 0 nodes. Used 1 nodes. Healthy nodes 1
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org