You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Virajith Jalaparti (JIRA)" <ji...@apache.org> on 2017/03/16 00:17:41 UTC
[jira] [Comment Edited] (HDFS-11190) Namenode support for data
stored in external stores.
[ https://issues.apache.org/jira/browse/HDFS-11190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15927238#comment-15927238 ]
Virajith Jalaparti edited comment on HDFS-11190 at 3/16/17 12:16 AM:
---------------------------------------------------------------------
Posting a patch that enables this functionality on the NN. The main changes include:
a) Adding a {{LocatedBlockBuilder}} that is used to build {{LocatedBlocks}}.
b) Adding a {{ProvidedBlocksBuilder}} as a subclass of {{LocatedBlockBuilder}}, which is used to build {{LocatedBlocks}} for provided blocks -- the {{ProvidedBlocksBuilder}} assigns a randomly chosen Datanode (which reports as provided volume) as the location of the provided blocks.
c) Adding a {{ProvidedStorageMap}} which is used to keep track of datanodes with provided volumes
d) Modifications to the {{BlockManager}} to use {{LocatedBlockBuilder}} to build {{LocatedBlocks}} and use {{ProvidedStorageMap}} while processing reports from datanodes.
Most of this code was contributed by [~chris.douglas].
was (Author: virajith):
Posting a patch that enables this functionality on the NN. The main changes include:
a) Adding a {{LocatedBlockBuilder}} that is used to build {{LocatedBlocks}}.
b) Adding a {{ProvidedBlocksBuilder}} as a subclass of {{LocatedBlockBuilder}}, which is used to build {{LocatedBlocks}} for provided blocks -- the {{ProvidedBlocksBuilder}} assigns a randomly chosen Datanode (which reports as provided volume) as the location of the provided blocks.
c) Adding a {{ProvidedStorageMap}} which is used to keep track of datanodes with provided volumes
d) Modifications to the {{BlockManager}} to use {{LocatedBlockBuilder}} to build {{LocatedBlocks}} and use {{ProvidedStorageMap}} while processing reports from datanodes.
> Namenode support for data stored in external stores.
> ----------------------------------------------------
>
> Key: HDFS-11190
> URL: https://issues.apache.org/jira/browse/HDFS-11190
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Virajith Jalaparti
> Attachments: HDFS-11190-HDFS-9806.001.patch
>
>
> The goal of this JIRA is to enable the Namenode to know about blocks that are in {{PROVIDED}} stores and are not necessarily stored on any Datanodes.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org