[jira] Created: (HDFS-893) Synchronization other FSNamesystem information to BackupNode

["Wang Xu (JIRA)" <ji...@apache.org>]

Synchronization other FSNamesystem information to BackupNode
------------------------------------------------------------

                 Key: HDFS-893
                 URL: https://issues.apache.org/jira/browse/HDFS-893
             Project: Hadoop HDFS
          Issue Type: New Feature
          Components: name-node
            Reporter: Wang Xu


Backup Node (BN) in trunk brings "warm" standby (HADOOP-4539), it synchronizes namespace information from Main NameNode, and leaves DataNode information, Block Operation (such as replication), and lease information unsynchronized. 

We (team from China Mobile Research Instatitute) had also some experimental on Namesystem schronization ( some information has been post at: http://gnawux.info/hadoop/2010/01/pratice-of-namenode-cluster-for-hdfs-ha/ ). Currently, we are thinking about migrate our code to re-based on Backup Node, and we think it might convenience the maintenance of code and contributing back to community.

As NN in current trunk can log all the namespace changes as journals to BN, the rest information needed to be synchronized includes:

 - Changes in DataNode Map: information collected from DataNode heartbeats and reports. Together with namespace info, the blocks map can also be calculated.
 - Lease information: leases are updated by clients periodically or when it lock/unlock file for write.
 - Block operation's status, such as pending replication block list: these information are depends on random number, thus it may lead to inconsistency if do not synchronize them.



-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.