You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2020/07/16 18:28:31 UTC
[GitHub] [hadoop-ozone] avijayanhwx opened a new pull request #1210: HDDS-3965. SCM failed to start up for duplicated pipeline detected.
avijayanhwx opened a new pull request #1210:
URL: https://github.com/apache/hadoop-ozone/pull/1210
## What changes were proposed in this pull request?
After the patch from HDDS-3925, SCM failed to start up for duplicated pipeline detected.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-3965
## How was this patch tested?
Manually tested.
Adding unit tests (WIP).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] nandakumar131 merged pull request #1210: HDDS-3965. SCM failed to start up for duplicated pipeline detected.
Posted by GitBox <gi...@apache.org>.
nandakumar131 merged pull request #1210:
URL: https://github.com/apache/hadoop-ozone/pull/1210
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] avijayanhwx commented on pull request #1210: HDDS-3965. SCM failed to start up for duplicated pipeline detected.
Posted by GitBox <gi...@apache.org>.
avijayanhwx commented on pull request #1210:
URL: https://github.com/apache/hadoop-ozone/pull/1210#issuecomment-659593212
First Restart with HDDS-3925.
`2020-07-16 18:22:21,133 [main] INFO pipeline.SCMPipelineManager: Processing pipeline PipelineID=096c944b-07d9-4a5b-af11-1102b46cf457
2020-07-16 18:22:21,133 [main] INFO pipeline.SCMPipelineManager: Found pipeline in old format key : PipelineID=096c944b-07d9-4a5b-af11-1102b46cf457
2020-07-16 18:22:21,133 [main] INFO db.RDBStoreIterator: Deleting pipelineID from DB : 117e0edf-ae20-4d43-8f8f-68edab965925
2020-07-16 18:22:21,141 [main] INFO pipeline.PipelineStateManager: Created pipeline Pipeline[ Id: 096c944b-07d9-4a5b-af11-1102b46cf457, Nodes: 690d95ba-9af3-41fe-a4e8-fd635072d724{ip: 172.24.0.7, host: ozone_datanode_3.ozone_default, networkLocation: /default-rack, certSerialId: null}, Type:RATIS, Factor:ONE, State:ALLOCATED, leaderId:, CreationTimestamp2020-07-16T18:22:21.132927Z]
2020-07-16 18:22:21,142 [main] INFO pipeline.SCMPipelineManager: Processing pipeline PipelineID=117e0edf-ae20-4d43-8f8f-68edab965925
2020-07-16 18:22:21,142 [main] INFO pipeline.SCMPipelineManager: Found pipeline in old format key : PipelineID=117e0edf-ae20-4d43-8f8f-68edab965925
2020-07-16 18:22:21,143 [main] INFO db.RDBStoreIterator: Deleting pipelineID from DB : 33950078-664e-44f5-b8e8-a53120a0ace4
2020-07-16 18:22:21,143 [main] INFO pipeline.PipelineStateManager: Created pipeline Pipeline[ Id: 117e0edf-ae20-4d43-8f8f-68edab965925, Nodes: d3827311-dda3-40a3-9a8c-10ab2d9bcc66{ip: 172.24.0.6, host: ozone_datanode_1.ozone_default, networkLocation: /default-rack, certSerialId: null}, Type:RATIS, Factor:ONE, State:ALLOCATED, leaderId:, CreationTimestamp2020-07-16T18:22:21.142554Z]
2020-07-16 18:22:21,144 [main] INFO pipeline.SCMPipelineManager: Processing pipeline PipelineID=33950078-664e-44f5-b8e8-a53120a0ace4
2020-07-16 18:22:21,144 [main] INFO pipeline.SCMPipelineManager: Found pipeline in old format key : PipelineID=33950078-664e-44f5-b8e8-a53120a0ace4
2020-07-16 18:22:21,144 [main] INFO db.RDBStoreIterator: Deleting pipelineID from DB : 5ddf942e-ccbe-4482-a3b4-49fa6af1f34c
2020-07-16 18:22:21,145 [main] INFO pipeline.PipelineStateManager: Created pipeline Pipeline[ Id: 33950078-664e-44f5-b8e8-a53120a0ace4, Nodes: 690d95ba-9af3-41fe-a4e8-fd635072d724{ip: 172.24.0.7, host: ozone_datanode_3.ozone_default, networkLocation: /default-rack, certSerialId: null}d3827311-dda3-40a3-9a8c-10ab2d9bcc66{ip: 172.24.0.6, host: ozone_datanode_1.ozone_default, networkLocation: /default-rack, certSerialId: null}1e6d6f78-d8e3-4e31-9408-8eecf44d3d90{ip: 172.24.0.2, host: ozone_datanode_2.ozone_default, networkLocation: /default-rack, certSerialId: null}, Type:RATIS, Factor:THREE, State:ALLOCATED, leaderId:, CreationTimestamp2020-07-16T18:22:21.144300Z]
2020-07-16 18:22:21,146 [main] INFO pipeline.SCMPipelineManager: Processing pipeline PipelineID=5ddf942e-ccbe-4482-a3b4-49fa6af1f34c
2020-07-16 18:22:21,146 [main] INFO pipeline.SCMPipelineManager: Found pipeline in old format key : PipelineID=5ddf942e-ccbe-4482-a3b4-49fa6af1f34c
2020-07-16 18:22:21,146 [main] INFO db.RDBStoreIterator: Unable to delete key as it does not exist.`
Second Restart with HDDS-3925
`2020-07-16 18:22:54,026 [main] ERROR server.StorageContainerManagerStarter: SCM start failed with exception
java.io.IOException: Duplicate pipeline ID PipelineID=096c944b-07d9-4a5b-af11-1102b46cf457 detected.
at org.apache.hadoop.hdds.scm.pipeline.PipelineStateMap.addPipeline(PipelineStateMap.java:86)
at org.apache.hadoop.hdds.scm.pipeline.PipelineStateManager.addPipeline(PipelineStateManager.java:53)
at org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager.initializePipelineState(SCMPipelineManager.java:165)
at org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager.<init>(SCMPipelineManager.java:100)
at org.apache.hadoop.hdds.scm.server.StorageContainerManager.initializeSystemManagers(StorageContainerManager.java:410)
at org.apache.hadoop.hdds.scm.server.StorageContainerManager.<init>(StorageContainerManager.java:281)
at org.apache.hadoop.hdds.scm.server.StorageContainerManager.<init>(StorageContainerManager.java:213)
at org.apache.hadoop.hdds.scm.server.StorageContainerManager.createSCM(StorageContainerManager.java:624)
at org.apache.hadoop.hdds.scm.server.StorageContainerManagerStarter$SCMStarterHelper.start(StorageContainerManagerStarter.java:144)
at org.apache.hadoop.hdds.scm.server.StorageContainerManagerStarter.startScm(StorageContainerManagerStarter.java:119)
at org.apache.hadoop.hdds.scm.server.StorageContainerManagerStarter.call(StorageContainerManagerStarter.java:67)
at org.apache.hadoop.hdds.scm.server.StorageContainerManagerStarter.call(StorageContainerManagerStarter.java:43)
at picocli.CommandLine.execute(CommandLine.java:1173)
at picocli.CommandLine.access$800(CommandLine.java:141)
at picocli.CommandLine$RunLast.handle(CommandLine.java:1367)
at picocli.CommandLine$RunLast.handle(CommandLine.java:1335)
at picocli.CommandLine$AbstractParseResultHandler.handleParseResult(CommandLine.java:1243)
at picocli.CommandLine.parseWithHandlers(CommandLine.java:1526)
at picocli.CommandLine.parseWithHandler(CommandLine.java:1465)
at org.apache.hadoop.hdds.cli.GenericCli.execute(GenericCli.java:75)
at org.apache.hadoop.hdds.cli.GenericCli.run(GenericCli.java:66)
at org.apache.hadoop.hdds.scm.server.StorageContainerManagerStarter.main(StorageContainerManagerStarter.java:56)
Duplicate pipeline ID PipelineID=096c944b-07d9-4a5b-af11-1102b46cf457 detected.
2020-07-16 18:22:54,040 [shutdown-hook-0] INFO server.StorageContainerManagerStarter: SHUTDOWN_MSG:`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] avijayanhwx removed a comment on pull request #1210: HDDS-3965. SCM failed to start up for duplicated pipeline detected.
Posted by GitBox <gi...@apache.org>.
avijayanhwx removed a comment on pull request #1210:
URL: https://github.com/apache/hadoop-ozone/pull/1210#issuecomment-659593212
First Restart with HDDS-3925.
`2020-07-16 18:22:21,133 [main] INFO pipeline.SCMPipelineManager: Processing pipeline PipelineID=096c944b-07d9-4a5b-af11-1102b46cf457
2020-07-16 18:22:21,133 [main] INFO pipeline.SCMPipelineManager: Found pipeline in old format key : PipelineID=096c944b-07d9-4a5b-af11-1102b46cf457
2020-07-16 18:22:21,133 [main] INFO db.RDBStoreIterator: Deleting pipelineID from DB : 117e0edf-ae20-4d43-8f8f-68edab965925
2020-07-16 18:22:21,141 [main] INFO pipeline.PipelineStateManager: Created pipeline Pipeline[ Id: 096c944b-07d9-4a5b-af11-1102b46cf457, Nodes: 690d95ba-9af3-41fe-a4e8-fd635072d724{ip: 172.24.0.7, host: ozone_datanode_3.ozone_default, networkLocation: /default-rack, certSerialId: null}, Type:RATIS, Factor:ONE, State:ALLOCATED, leaderId:, CreationTimestamp2020-07-16T18:22:21.132927Z]
2020-07-16 18:22:21,142 [main] INFO pipeline.SCMPipelineManager: Processing pipeline PipelineID=117e0edf-ae20-4d43-8f8f-68edab965925
2020-07-16 18:22:21,142 [main] INFO pipeline.SCMPipelineManager: Found pipeline in old format key : PipelineID=117e0edf-ae20-4d43-8f8f-68edab965925
2020-07-16 18:22:21,143 [main] INFO db.RDBStoreIterator: Deleting pipelineID from DB : 33950078-664e-44f5-b8e8-a53120a0ace4
2020-07-16 18:22:21,143 [main] INFO pipeline.PipelineStateManager: Created pipeline Pipeline[ Id: 117e0edf-ae20-4d43-8f8f-68edab965925, Nodes: d3827311-dda3-40a3-9a8c-10ab2d9bcc66{ip: 172.24.0.6, host: ozone_datanode_1.ozone_default, networkLocation: /default-rack, certSerialId: null}, Type:RATIS, Factor:ONE, State:ALLOCATED, leaderId:, CreationTimestamp2020-07-16T18:22:21.142554Z]
2020-07-16 18:22:21,144 [main] INFO pipeline.SCMPipelineManager: Processing pipeline PipelineID=33950078-664e-44f5-b8e8-a53120a0ace4
2020-07-16 18:22:21,144 [main] INFO pipeline.SCMPipelineManager: Found pipeline in old format key : PipelineID=33950078-664e-44f5-b8e8-a53120a0ace4
2020-07-16 18:22:21,144 [main] INFO db.RDBStoreIterator: Deleting pipelineID from DB : 5ddf942e-ccbe-4482-a3b4-49fa6af1f34c
2020-07-16 18:22:21,145 [main] INFO pipeline.PipelineStateManager: Created pipeline Pipeline[ Id: 33950078-664e-44f5-b8e8-a53120a0ace4, Nodes: 690d95ba-9af3-41fe-a4e8-fd635072d724{ip: 172.24.0.7, host: ozone_datanode_3.ozone_default, networkLocation: /default-rack, certSerialId: null}d3827311-dda3-40a3-9a8c-10ab2d9bcc66{ip: 172.24.0.6, host: ozone_datanode_1.ozone_default, networkLocation: /default-rack, certSerialId: null}1e6d6f78-d8e3-4e31-9408-8eecf44d3d90{ip: 172.24.0.2, host: ozone_datanode_2.ozone_default, networkLocation: /default-rack, certSerialId: null}, Type:RATIS, Factor:THREE, State:ALLOCATED, leaderId:, CreationTimestamp2020-07-16T18:22:21.144300Z]
2020-07-16 18:22:21,146 [main] INFO pipeline.SCMPipelineManager: Processing pipeline PipelineID=5ddf942e-ccbe-4482-a3b4-49fa6af1f34c
2020-07-16 18:22:21,146 [main] INFO pipeline.SCMPipelineManager: Found pipeline in old format key : PipelineID=5ddf942e-ccbe-4482-a3b4-49fa6af1f34c
2020-07-16 18:22:21,146 [main] INFO db.RDBStoreIterator: Unable to delete key as it does not exist.`
Second Restart with HDDS-3925
`2020-07-16 18:22:54,026 [main] ERROR server.StorageContainerManagerStarter: SCM start failed with exception
java.io.IOException: Duplicate pipeline ID PipelineID=096c944b-07d9-4a5b-af11-1102b46cf457 detected.
at org.apache.hadoop.hdds.scm.pipeline.PipelineStateMap.addPipeline(PipelineStateMap.java:86)
at org.apache.hadoop.hdds.scm.pipeline.PipelineStateManager.addPipeline(PipelineStateManager.java:53)
at org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager.initializePipelineState(SCMPipelineManager.java:165)
at org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager.<init>(SCMPipelineManager.java:100)
at org.apache.hadoop.hdds.scm.server.StorageContainerManager.initializeSystemManagers(StorageContainerManager.java:410)
at org.apache.hadoop.hdds.scm.server.StorageContainerManager.<init>(StorageContainerManager.java:281)
at org.apache.hadoop.hdds.scm.server.StorageContainerManager.<init>(StorageContainerManager.java:213)
at org.apache.hadoop.hdds.scm.server.StorageContainerManager.createSCM(StorageContainerManager.java:624)
at org.apache.hadoop.hdds.scm.server.StorageContainerManagerStarter$SCMStarterHelper.start(StorageContainerManagerStarter.java:144)
at org.apache.hadoop.hdds.scm.server.StorageContainerManagerStarter.startScm(StorageContainerManagerStarter.java:119)
at org.apache.hadoop.hdds.scm.server.StorageContainerManagerStarter.call(StorageContainerManagerStarter.java:67)
at org.apache.hadoop.hdds.scm.server.StorageContainerManagerStarter.call(StorageContainerManagerStarter.java:43)
at picocli.CommandLine.execute(CommandLine.java:1173)
at picocli.CommandLine.access$800(CommandLine.java:141)
at picocli.CommandLine$RunLast.handle(CommandLine.java:1367)
at picocli.CommandLine$RunLast.handle(CommandLine.java:1335)
at picocli.CommandLine$AbstractParseResultHandler.handleParseResult(CommandLine.java:1243)
at picocli.CommandLine.parseWithHandlers(CommandLine.java:1526)
at picocli.CommandLine.parseWithHandler(CommandLine.java:1465)
at org.apache.hadoop.hdds.cli.GenericCli.execute(GenericCli.java:75)
at org.apache.hadoop.hdds.cli.GenericCli.run(GenericCli.java:66)
at org.apache.hadoop.hdds.scm.server.StorageContainerManagerStarter.main(StorageContainerManagerStarter.java:56)
Duplicate pipeline ID PipelineID=096c944b-07d9-4a5b-af11-1102b46cf457 detected.
2020-07-16 18:22:54,040 [shutdown-hook-0] INFO server.StorageContainerManagerStarter: SHUTDOWN_MSG:`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org
[GitHub] [hadoop-ozone] avijayanhwx commented on pull request #1210: HDDS-3965. SCM failed to start up for duplicated pipeline detected.
Posted by GitBox <gi...@apache.org>.
avijayanhwx commented on pull request #1210:
URL: https://github.com/apache/hadoop-ozone/pull/1210#issuecomment-659832842
cc @fapifta
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org