You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Sanjay Radia (JIRA)" <ji...@apache.org> on 2017/12/19 21:10:02 UTC
[jira] [Updated] (HDFS-7240) Object store in HDFS
[ https://issues.apache.org/jira/browse/HDFS-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sanjay Radia updated HDFS-7240:
-------------------------------
Attachment: Evolving NN using new block-container layer.pdf
I have attached a doc that describes how the existing NN can be modified to plug in the new block-container layer provided by HDFS-7240. Two key milestone are describe: First milestone is where the Container Map is kept in NN (gets us to almost 2x scalability since container map is 1/40th of original block map assuming an *average actual* block size of 50MB); this milestone does NOT require removing the FSN/BM lock. The 2nd milestone is where the container map and block management is completely removed which gets us to 2x scalability. After the 2nd milestone, the NN can be evolved in several directions for further scalability.
> Object store in HDFS
> --------------------
>
> Key: HDFS-7240
> URL: https://issues.apache.org/jira/browse/HDFS-7240
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Jitendra Nath Pandey
> Assignee: Jitendra Nath Pandey
> Attachments: Evolving NN using new block-container layer.pdf, HDFS Scalability and Ozone.pdf, HDFS-7240.001.patch, HDFS-7240.002.patch, HDFS-7240.003.patch, HDFS-7240.003.patch, HDFS-7240.004.patch, HDFS-7240.005.patch, HDFS-7240.006.patch, MeetingMinutes.pdf, Ozone-architecture-v1.pdf, Ozonedesignupdate.pdf, ozone_user_v0.pdf
>
>
> This jira proposes to add object store capabilities into HDFS.
> As part of the federation work (HDFS-1052) we separated block storage as a generic storage layer. Using the Block Pool abstraction, new kinds of namespaces can be built on top of the storage layer i.e. datanodes.
> In this jira I will explore building an object store using the datanode storage, but independent of namespace metadata.
> I will soon update with a detailed design document.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org