You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Hairong Kuang (JIRA)" <ji...@apache.org> on 2009/09/22 03:38:16 UTC

[jira] Updated: (HDFS-624) Client support pipeline recovery

     [ https://issues.apache.org/jira/browse/HDFS-624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hairong Kuang updated HDFS-624:
-------------------------------

    Attachment: pipelineRecovery.patch

This patch implemented the new pipeline recovery algorithm described in the design doc. With this patch, a client does not go through a primary data to perform pipeline recovery.

Major changes include:
1.  if setting up a pipeline for append, data streamer calls setupPipelineForAppendOrRecovery(); If setting up a pipeline for error recovery, data streamer calls processDatanodeError() which in turn calls setupPipelineForAppendOrRecovery().
2. When send a opWrite request to datanode to set up a pipeline, a client sends the reason, i.e, construction stage (create, append, append recovery, or streaming recovery), along with newGS, minBytesRcvd, and maxBytesRcvd etc.
In above two case, a client gets a new generation stamp and an access token from NameNode before setting up a pipeline and sends the pipeline update to NameNode after a successful pipeline setup.
3. Depending on the reason for setting up a pipeline, Datanode calls createTemporary, createRbw, append, recoverAppend, or recoverRbw to handle the request respectively.

> Client support pipeline recovery
> --------------------------------
>
>                 Key: HDFS-624
>                 URL: https://issues.apache.org/jira/browse/HDFS-624
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>    Affects Versions: Append Branch
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: Append Branch
>
>         Attachments: pipelineRecovery.patch
>
>
> This jira aims to
> 1. set up initial pipeline for append;
> 2. recover failed pipeline setup for append;
> 2. set up pipeline to recover failed data streaming.
> The algorithm is described in the design document in the pipeline recovery and pipeline set up sections. Pipeline close and failed pipeline close are not included in this jira. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.