You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-dev@hadoop.apache.org by "Sameer Paranjpye (JIRA)" <ji...@apache.org> on 2006/03/24 23:16:04 UTC

[jira] Updated: (HADOOP-74) hash blocks into dfs.data.dirs

     [ http://issues.apache.org/jira/browse/HADOOP-74?page=all ]

Sameer Paranjpye updated HADOOP-74:
-----------------------------------

    Fix Version: 0.2
                     (was: 0.1)
        Version: 0.2
                     (was: 0.1)

> hash blocks into dfs.data.dirs
> ------------------------------
>
>          Key: HADOOP-74
>          URL: http://issues.apache.org/jira/browse/HADOOP-74
>      Project: Hadoop
>         Type: Improvement
>   Components: dfs
>     Versions: 0.2
>  Environment: large clusters
>     Reporter: Doug Cutting
>     Assignee: Konstantin Shvachko
>      Fix For: 0.2

>
> When dfs.data.dir has multiple values, we currently start a DataNode for each (all in the same JVM).  Instead we should run a single DataNode that stores block files into the different directories.  This will reduce the number of connections to the namenode.  We cannot hash because different devices might be different amounts full.  So the datanode will need to keep a table mapping from block id to file location, and add new blocks to less full devices.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira