[jira] [Created] (HDFS-5389) Remove INode limitations in Namenode

["Lin Xiao (JIRA)" <ji...@apache.org>]

Lin Xiao created HDFS-5389:
------------------------------

             Summary: Remove INode limitations in Namenode
                 Key: HDFS-5389
                 URL: https://issues.apache.org/jira/browse/HDFS-5389
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: namenode
    Affects Versions: 0.23.1
            Reporter: Lin Xiao
            Priority: Minor


Current HDFS Namenode stores all of its metadata in RAM. This has allowed Hadoop clusters to scale to 100K concurrent tasks. However, the memory limits the total number of files that a single NN can store. While Federation allows one to create multiple volumes with additional Namenodes, there is a need to scale a single namespace and also to store multiple namespaces in a single Namenode. When inodes are also stored on persistent storage, the system's boot time can be significantly reduced because there is no need to replay edit logs. It also provides the potential to support extended attributes once the memory size is not the bottleneck.




--
This message was sent by Atlassian JIRA
(v6.1#6144)