[jira] Updated: (HADOOP-3799) Design a pluggable interface to place replicas of blocks in HDFS

["dhruba borthakur (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

dhruba borthakur updated HADOOP-3799:
-------------------------------------

    Attachment: BlockPlacementPluggable.txt

This patch makes the HDFS block placement algorithm pluggable. The Namenode uses the name of the configuration parameter called dfs.block.replicator.classname to find the name of the class that chooses block locations. The default policy remains the same as before and can be found in ReplicationTargetChooser.java.

A new block placement policy has to implement the interface specified in BlockPlacementInterface.java

I would have liked the interface to specify the name of the file for which the block belongs to. But, the name of the file cannot be materialized cheaply when the block manager is doing block replication. 

> Design a pluggable interface to place replicas of blocks in HDFS
> ----------------------------------------------------------------
>
>                 Key: HADOOP-3799
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3799
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: BlockPlacementPluggable.txt
>
>
> The current HDFS code typically places one replica on local rack, the second replica on remote random rack and the third replica on a random node of that remote rack. This algorithm is baked in the NameNode's code. It would be nice to make the block placement algorithm a pluggable interface. This will allow experimentation of different placement algorithms based on workloads, availability guarantees and failure models.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.