You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Matt Foley (JIRA)" <ji...@apache.org> on 2011/04/23 00:29:05 UTC
[jira] [Updated] (HDFS-1798) saveNamespace should background the
writing of the FSImage files when in SafeMode
[ https://issues.apache.org/jira/browse/HDFS-1798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matt Foley updated HDFS-1798:
-----------------------------
Attachment: 1798_asyncSaveNamespace_v1.patch
This is a preliminary upload for design review. Still working on the unit tests. Patch contains the following:
* FSNamesystem
** added ability to place "hold" on Safe Mode
** also fixed several left-over uses of "synchronized" instead of read/write lock
* FSImage - created async thread to run saveNamespace(), guarded by a hold on Safe Mode.
* FSDirectory - loadFSImage() now calls asyncSaveNamespace() instead of saveNamespace().
It would have been very difficult to restructure FSDirectory.loadFSImage() to split into separate read and write phases, as initially proposed. Instead, I provided a lenient initializer for startup safe mode, initializeSafeMode(). It is called from both asyncSaveNamespace() and FSNamesystem.initialize(), and handles the possible combinations of events correctly.
> saveNamespace should background the writing of the FSImage files when in SafeMode
> ---------------------------------------------------------------------------------
>
> Key: HDFS-1798
> URL: https://issues.apache.org/jira/browse/HDFS-1798
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Matt Foley
> Assignee: Matt Foley
> Attachments: 1798_asyncSaveNamespace_v1.patch
>
>
> Even with parallel writes of the FSImage files (HDFS-1071), large clusters still typically spend 5 minutes writing the files. When in SafeMode, it should be safe to background these writes instead of waiting for them. To enable this, the namenode must be prevented from leaving SafeMode until the writes are done.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira