You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "sam rash (JIRA)" <ji...@apache.org> on 2010/08/22 05:37:16 UTC
[jira] Commented: (HDFS-1350) make datanodes do graceful shutdown
[ https://issues.apache.org/jira/browse/HDFS-1350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12901129#action_12901129 ]
sam rash commented on HDFS-1350:
--------------------------------
My understanding of how lease recovery works in 20-append is that on cluster restart, an open file will be recovered by the Namenode. Datanodes will send the longest valid length of the block (ie, if there are 8 bytes of checksum and 1500 data, the valid length is 1024 assuming 512 byte chunk size). The block is then truncated to a valid length.
20-append seems to have a bug that any block where data + checksum length don't match, the block isn't use in lease recovery. the work here might be to fix that?
> make datanodes do graceful shutdown
> -----------------------------------
>
> Key: HDFS-1350
> URL: https://issues.apache.org/jira/browse/HDFS-1350
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: data-node
> Reporter: sam rash
> Assignee: sam rash
>
> we found that the Datanode doesn't do a graceful shutdown and a block can be corrupted (data + checksum amounts off)
> we can make the DN do a graceful shutdown in case there are open files. if this presents a problem to a timely shutdown, we can make a it a parameter of how long to wait for the full graceful shutdown before just exiting
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.