You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Sammi Chen (Jira)" <ji...@apache.org> on 2020/07/15 12:57:00 UTC
[jira] [Commented] (HDDS-3965) SCM failed to start up for
duplicated pipeline detected
[ https://issues.apache.org/jira/browse/HDDS-3965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17158136#comment-17158136 ]
Sammi Chen commented on HDDS-3965:
----------------------------------
cc [~pifta], could you take a look at this issue.
I did a rough investigation. It seems " it.removeFromDB()" in SCMPipelineManager finished successfully while the pipeline key is still in DB. So next time, when SCM restart, it fails with duplicated pipeline detected.
> SCM failed to start up for duplicated pipeline detected
> -------------------------------------------------------
>
> Key: HDDS-3965
> URL: https://issues.apache.org/jira/browse/HDDS-3965
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Reporter: Sammi Chen
> Priority: Critical
>
> SCM LOG：
> 2020-07-15 19:25:09,768 [main] ERROR org.apache.hadoop.hdds.scm.server.StorageContainerManagerStarter: SCM start failed with exception
> java.io.IOException: Duplicate pipeline ID PipelineID=db5966ec-140f-48d8-b0d6-e6f2ff777a77 detected.
> at org.apache.hadoop.hdds.scm.pipeline.PipelineStateMap.addPipeline(PipelineStateMap.java:89)
> at org.apache.hadoop.hdds.scm.pipeline.PipelineStateManager.addPipeline(PipelineStateManager.java:53)
> at org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager.initializePipelineState(SCMPipelineManager.java:165)
> at org.apache.hadoop.hdds.scm.pipeline.SCMPipelineManager.<init>(SCMPipelineManager.java:100)
> at org.apache.hadoop.hdds.scm.server.StorageContainerManager.initializeSystemManagers(StorageContainerManager.java:410)
> at org.apache.hadoop.hdds.scm.server.StorageContainerManager.<init>(StorageContainerManager.java:281)
> at org.apache.hadoop.hdds.scm.server.StorageContainerManager.<init>(StorageContainerManager.java:213)
> at org.apache.hadoop.hdds.scm.server.StorageContainerManager.createSCM(StorageContainerManager.java:624)
> at org.apache.hadoop.hdds.scm.server.StorageContainerManagerStarter$SCMStarterHelper.start(StorageContainerManagerStarter.java:144)
> at org.apache.hadoop.hdds.scm.server.StorageContainerManagerStarter.startScm(StorageContainerManagerStarter.java:119)
> RocksDB dump, string,
> rocksdb_ldb --db=scm.db scan --column_family=pipelines
> $db5966ec-140f-48d8-b0d6-e6f2ff777a77ؑ????٬??????? :
> ?
> $02d3c9b4-7972-4471-a520-fff108b8d32e
> 10.73.33.62
> 10.73.33.62"
> RATIS?M"
> /default-rack???Ƕ?????Ő???? *?71-a520-fff108b8d32e:
> $db5966ec-140f-48d8-b0d6-e6f2ff777a77ؑ????٬???????2
> ?Yf?Hذ????wzw :
> ?
> $02d3c9b4-7972-4471-a520-fff108b8d32e
> 10.73.33.62
> 10.73.33.62"
> RATIS?M"
> HEX:
> 0x0A2464623539363665632D313430662D343864382D623064362D653666326666373737613737A2061608D891BDA0C1DDD9ACDB0110F7F4DDFBAFDEB9EBB001 : 0x0AAA010A2430326433633962342D373937322D343437312D613532302D666666313038623864333265120B31302E37332E33332E36321A0B31302E37332E33332E3632220A0A05524154495310824D220F0A0A5354414E44414C4F4E4510834D322430326433633962342D373937322D343437312D613532302D6666663130386238643332653A0D2F64656661756C742D7261636BA2061508F188C9CBC7B6F2E90210AEA6E3C590FEBF90A5011001180120012A3F0A2464623539363665632D313430662D343864382D623064362D653666326666373737613737A2061608D891BDA0C1DDD9ACDB0110F7F4DDFBAFDEB9EBB00132004085A7C1E5B42E
> 0xDB5966EC140F48D8B0D6E6F2FF777A77 : 0x0AAC010A2430326433633962342D373937322D343437312D613532302D666666313038623864333265120B31302E37332E33332E36321A0B31302E37332E33332E3632220A0A05524154495310824D220F0A0A5354414E44414C4F4E4510834D322430326433633962342D373937322D343437312D613532302D6666663130386238643332653A0D2F64656661756C742D7261636B4800A2061508F188C9CBC7B6F2E90210AEA6E3C590FEBF90A5011001180120012A3F0A2464623539363665632D313430662D343864382D623064362D653666326666373737613737A2061608D891BDA0C1DDD9ACDB0110F7F4DDFBAFDEB9EBB0013200409DFCAF8BB52E
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org