You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Amir Langer (JIRA)" <ji...@apache.org> on 2014/07/10 13:09:04 UTC
[jira] [Created] (HDFS-6658) Namenode memory optimization - Block
replicas list
Amir Langer created HDFS-6658:
---------------------------------
Summary: Namenode memory optimization - Block replicas list
Key: HDFS-6658
URL: https://issues.apache.org/jira/browse/HDFS-6658
Project: Hadoop HDFS
Issue Type: Improvement
Components: namenode
Affects Versions: 2.4.1
Reporter: Amir Langer
Fix For: 3.0.0, 2.5.0
Part of the memory consumed by every BlockInfo object in the Namenode is a linked list of block references for every DatanodeStorageInfo (called "triplets").
We propose to change the way we store the list in memory.
Using primitive integer indexes instead of object references will reduce the memory needed for every block replica (when compressed oops is disabled) and in our new design the list overhead will be per DatanodeStorageInfo and not per block replica.
see attached design doc. for details and evaluation results.
--
This message was sent by Atlassian JIRA
(v6.2#6252)