You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Ivan Kelly (JIRA)" <ji...@apache.org> on 2011/07/02 13:54:28 UTC
[jira] [Updated] (HDFS-2018) Move all journal stream management
code into one place
[ https://issues.apache.org/jira/browse/HDFS-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ivan Kelly updated HDFS-2018:
-----------------------------
Attachment: HDFS-2018.diff
new patch rebased on top of moved hdfs-common/hdfs.
I haven't tackled archiving yet, and storage inspector tests are simply commented out now, but I will sort all that stuff out on monday.
For archiving, I think the entry point for the FSEditLog should simply pass in the txnid. StorageArchiver is file specific. I propose we make the StorageArchiver a member of NNStorage, we can make it configurable if need be. Then the FileJournalManager should request it from its storage object and archive that way.
> Move all journal stream management code into one place
> ------------------------------------------------------
>
> Key: HDFS-2018
> URL: https://issues.apache.org/jira/browse/HDFS-2018
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: Edit log branch (HDFS-1073)
>
> Attachments: HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff
>
>
> Currently in the HDFS-1073 branch, the code for creating output streams is in FileJournalManager and the code for input streams is in the inspectors. This change does a number of things.
> - Input and Output streams are now created by the JournalManager.
> - FSImageStorageInspectors now deals with URIs when referring to edit logs
> - Recovery of inprogress logs is performed by counting the number of transactions instead of looking at the length of the file.
> The patch for this applies on top of the HDFS-1073 branch + HDFS-2003 patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira