You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "dhruba borthakur (JIRA)" <ji...@apache.org> on 2010/02/14 10:47:29 UTC
[jira] Created: (HDFS-978) Record every new block allocation of a
file into the transaction log.
Record every new block allocation of a file into the transaction log.
---------------------------------------------------------------------
Key: HDFS-978
URL: https://issues.apache.org/jira/browse/HDFS-978
Project: Hadoop HDFS
Issue Type: Improvement
Components: name-node
Reporter: dhruba borthakur
HDFS should record every new block allocation (of a file) into its transaction logs. In the current code, block allocations are persisted only when a file is closed or hflush-ed. This feature will enable HDFS writers to survive namenode restarts.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-978) Record every new block allocation of a
file into the transaction log.
Posted by "Wang Xu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833562#action_12833562 ]
Wang Xu commented on HDFS-978:
------------------------------
Do you mean log it after a block is written by datanodes or just allocated?
> Record every new block allocation of a file into the transaction log.
> ---------------------------------------------------------------------
>
> Key: HDFS-978
> URL: https://issues.apache.org/jira/browse/HDFS-978
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Reporter: dhruba borthakur
>
> HDFS should record every new block allocation (of a file) into its transaction logs. In the current code, block allocations are persisted only when a file is closed or hflush-ed. This feature will enable HDFS writers to survive namenode restarts.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-978) Record every new block allocation of a
file into the transaction log.
Posted by "dhruba borthakur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833622#action_12833622 ]
dhruba borthakur commented on HDFS-978:
---------------------------------------
When a writer requests a new block for a file, the NN should record the new blockid in the edits log. This persists the mapping from a filename to the list of its blocks. Now, the writer can write to this block (directly to the DataNode(s)) while the NN can be restarted without interrupting/invalidating that write.
> Record every new block allocation of a file into the transaction log.
> ---------------------------------------------------------------------
>
> Key: HDFS-978
> URL: https://issues.apache.org/jira/browse/HDFS-978
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: name-node
> Reporter: dhruba borthakur
>
> HDFS should record every new block allocation (of a file) into its transaction logs. In the current code, block allocations are persisted only when a file is closed or hflush-ed. This feature will enable HDFS writers to survive namenode restarts.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.