You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Attila Doroszlai (Jira)" <ji...@apache.org> on 2021/06/07 12:55:00 UTC
[jira] [Created] (HDDS-5312) Intermittent failure in SCM Ratis
integration test
Attila Doroszlai created HDDS-5312:
--------------------------------------
Summary: Intermittent failure in SCM Ratis integration test
Key: HDDS-5312
URL: https://issues.apache.org/jira/browse/HDDS-5312
Project: Apache Ozone
Issue Type: Bug
Components: SCM HA, test
Affects Versions: 1.2.0
Reporter: Attila Doroszlai
Assignee: Attila Doroszlai
Some integration tests intermittently fail due to mini cluster not existing safe mode within 2 minutes timeout. The problem is that pipeline creation interval is also 2 minutes. It may happen that pipeline is created only while the cluster is being shut down due to timeout.
{noformat:title=https://github.com/elek/ozone-build-results/blob/master/2021/06/02/8191/it-filesystem-hdds/hadoop-ozone/integration-test/org.apache.hadoop.hdds.scm.TestSCMInstallSnapshot-output.txt}
2021-06-02 03:21:03,005 [RatisPipelineUtilsThread - 0] WARN pipeline.PipelinePlacementPolicy (PipelinePlacementPolicy.java:filterViableNodes(151)) - Pipeline creation failed due to no sufficient healthy datanodes. Required 3. Found 2.
...
2021-06-02 03:21:04,007 [Listener at 127.0.0.1/40677] INFO ozone.MiniOzoneClusterImpl (MiniOzoneClusterImpl.java:lambda$waitForClusterToBeReady$0(224)) - Nodes are ready. Got 3 of 3 DN Heartbeats.
...
2021-06-02 03:22:59,107 [Listener at 127.0.0.1/40677] INFO ozone.MiniOzoneClusterImpl (MiniOzoneClusterImpl.java:shutdown(443)) - Shutting down the Mini Ozone Cluster
...
2021-06-02 03:23:03,031 [6d4e3dd1-e161-4c07-861b-817db46a0427@group-0D81E0660BF9-StateMachineUpdater] INFO pipeline.PipelineStateManager (PipelineStateManagerV2Impl.java:addPipeline(101)) - Created pipeline Pipeline ... RATIS/THREE ...
{noformat}
{noformat:title=https://github.com/elek/ozone-build-results/blob/master/2021/06/02/8191/it-filesystem-hdds/hadoop-ozone/integration-test/org.apache.hadoop.hdds.scm.TestSCMInstallSnapshot.txt}
org.apache.hadoop.hdds.scm.TestSCMInstallSnapshot Time elapsed: 146.994 s <<< ERROR!
java.util.concurrent.TimeoutException:
...
at org.apache.hadoop.ozone.MiniOzoneClusterImpl.waitForClusterToBeReady(MiniOzoneClusterImpl.java:217)
{noformat}
Related test failures:
{noformat}
2021/05/26/8113/it-client/hadoop-ozone/integration-test/org.apache.hadoop.ozone.client.rpc.TestOzoneRpcClientWithRatis.txt
2021/05/26/8118/it-ozone/hadoop-ozone/integration-test/org.apache.hadoop.ozone.scm.TestStorageContainerManagerHA.txt
2021/05/27/8142/it-filesystem-hdds/hadoop-ozone/integration-test/org.apache.hadoop.hdds.scm.TestSCMInstallSnapshot.txt
2021/05/30/8164/it-ozone/hadoop-ozone/integration-test/org.apache.hadoop.ozone.om.TestOzoneManagerRestInterface.txt
2021/05/31/8166/it-filesystem-hdds/hadoop-ozone/integration-test/org.apache.hadoop.hdds.scm.TestSCMSnapshot.txt
2021/05/31/8177/it-filesystem-hdds/hadoop-ozone/integration-test/org.apache.hadoop.hdds.scm.pipeline.TestPipelineClose.txt
2021/06/02/8191/it-filesystem-hdds/hadoop-ozone/integration-test/org.apache.hadoop.hdds.scm.TestSCMInstallSnapshot.txt
2021/06/02/8193/it-ozone/hadoop-ozone/integration-test/org.apache.hadoop.ozone.scm.TestStorageContainerManagerHA.txt
2021/06/02/8211/it-filesystem-hdds/hadoop-ozone/integration-test/org.apache.hadoop.hdds.scm.TestSCMSnapshot.txt
2021/06/02/8217/it-filesystem-hdds/hadoop-ozone/integration-test/org.apache.hadoop.hdds.scm.TestSCMInstallSnapshot.txt
2021/06/07/8299/it-filesystem-hdds/hadoop-ozone/integration-test/org.apache.hadoop.hdds.scm.TestSCMInstallSnapshot.txt
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org