You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Hairong Kuang (JIRA)" <ji...@apache.org> on 2009/09/22 03:38:16 UTC
[jira] Updated: (HDFS-624) Client support pipeline recovery
[ https://issues.apache.org/jira/browse/HDFS-624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hairong Kuang updated HDFS-624:
-------------------------------
Attachment: pipelineRecovery.patch
This patch implemented the new pipeline recovery algorithm described in the design doc. With this patch, a client does not go through a primary data to perform pipeline recovery.
Major changes include:
1. if setting up a pipeline for append, data streamer calls setupPipelineForAppendOrRecovery(); If setting up a pipeline for error recovery, data streamer calls processDatanodeError() which in turn calls setupPipelineForAppendOrRecovery().
2. When send a opWrite request to datanode to set up a pipeline, a client sends the reason, i.e, construction stage (create, append, append recovery, or streaming recovery), along with newGS, minBytesRcvd, and maxBytesRcvd etc.
In above two case, a client gets a new generation stamp and an access token from NameNode before setting up a pipeline and sends the pipeline update to NameNode after a successful pipeline setup.
3. Depending on the reason for setting up a pipeline, Datanode calls createTemporary, createRbw, append, recoverAppend, or recoverRbw to handle the request respectively.
> Client support pipeline recovery
> --------------------------------
>
> Key: HDFS-624
> URL: https://issues.apache.org/jira/browse/HDFS-624
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: Append Branch
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: Append Branch
>
> Attachments: pipelineRecovery.patch
>
>
> This jira aims to
> 1. set up initial pipeline for append;
> 2. recover failed pipeline setup for append;
> 2. set up pipeline to recover failed data streaming.
> The algorithm is described in the design document in the pipeline recovery and pipeline set up sections. Pipeline close and failed pipeline close are not included in this jira.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.