[jira] [Comment Edited] (HDDS-399) Handle pipeline discovery on SCM restart.

["Anu Engineer (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDDS-399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16611416#comment-16611416 ] 

Anu Engineer edited comment on HDDS-399 at 9/12/18 12:59 AM:
-------------------------------------------------------------

Thanks for updating the patch. It is much more easier to understand. I really appreciate it.

Something that is not very clear to me from the patch. I will dig deeper into the code to see if I can puzzle this out.

Let us say that I tried to create a pipeline and I timed-out. We will set the status to closed. However, what is the state of pipeline in the cluster ? what happens if we get a pipeline report little later ? how is that handled ? 

I think I found it, in {{processNodeReport(Pipeline pipeline, DatanodeDetails dn)}} in {{PipelineManager.java}} we will set that to open, and we will set the pipeline state to {{Open}. If we had done that via the newly introduced {{statemanager}} in {{PipelineStateManager}}, then this would have thrown, becomes there is no path in the state machine to go from {{closed}} to {{open}}.

Now if we reboot, the reload will think this datanode/pipeline is closed, but in a little while when the NodeReport/Pipeline report arrives we will set it to {{open}}. 

Please let me know if I am mistaken about this, it is  pretty complex code,  so I might be mistaken about this.


was (Author: anu):
Thanks for updating the patch. It is much more easier to understand. I really appreciate it.

Something that is not very clear to me from the patch. I will dig deeper into the code to see if I can puzzle this out.

Let us say that I tried to create a pipeline and I timedout. We will set the status to closed. However, what is the state of pipeline in the cluster ? what happens if we get a pipeline report little later ? how is that handled ? 

> Handle pipeline discovery on SCM restart.
> -----------------------------------------
>
>                 Key: HDDS-399
>                 URL: https://issues.apache.org/jira/browse/HDDS-399
>             Project: Hadoop Distributed Data Store
>          Issue Type: Bug
>          Components: SCM
>    Affects Versions: 0.2.1
>            Reporter: Mukul Kumar Singh
>            Assignee: Mukul Kumar Singh
>            Priority: Blocker
>             Fix For: 0.2.1
>
>         Attachments: HDDS-399.001.patch, HDDS-399.002.patch, HDDS-399.003.patch
>
>
> On SCM restart, as part on node registration, SCM should find out the list on open pipeline on the node. Once all the nodes of the pipeline have reported back, they should be added as active pipelines for further allocations.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org