You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Surendra Singh Lilhore (JIRA)" <ji...@apache.org> on 2015/12/02 15:23:11 UTC
[jira] [Updated] (HDFS-9355) Add client API to get Datanode list
based on storage type
[ https://issues.apache.org/jira/browse/HDFS-9355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Surendra Singh Lilhore updated HDFS-9355:
-----------------------------------------
Description:
HDFS-2576 give option to hint namenode about favored nodes, but in heterogeneous cluster this will not work out. Suppose client wants to write his data in directory which have COLD policy, but he don't know which DN have ARCHIVE storage, So he will not able to give favoredNodes list.
was:
Through this feature client can give suggestion to HDFS to write his all the blocks on same set of datanodes. Currently this we can achieve through HDFS-2576. HDFS-2576 give option to hint namenode about favored nodes, but in heterogeneous cluster this will not work out. Support client wants to write his data in directory which have COLD policy, but he don't know which DN have ARCHIVE storage, So he will not able to give favoredNodes list.
*Implementation*
Colocation can enable by setting "dfs.colocation.enable" true in client configuration. If colocation is enable and favoredNodes list is empty then {{DataStreamer}} will set first set of datanodes as favoredNodes which is chosen for first block and subsequent block will use the same datanodes for write. Before closing file client can get the favoredNodes list and same he can use for writing new file.
Issue Type: Improvement (was: New Feature)
Summary: Add client API to get Datanode list based on storage type (was: Support colocation in HDFS.)
Updated jira summary and description based on [~nijel] suggestion..
> Add client API to get Datanode list based on storage type
> ---------------------------------------------------------
>
> Key: HDFS-9355
> URL: https://issues.apache.org/jira/browse/HDFS-9355
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Reporter: Surendra Singh Lilhore
> Assignee: Surendra Singh Lilhore
>
> HDFS-2576 give option to hint namenode about favored nodes, but in heterogeneous cluster this will not work out. Suppose client wants to write his data in directory which have COLD policy, but he don't know which DN have ARCHIVE storage, So he will not able to give favoredNodes list.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)