You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Vishal Ghawate <vi...@persistent.co.in> on 2009/05/11 08:20:39 UTC
datanode replication
hi,
i just cant resolve how the namenode decide on where to place the replicas of the datanode
can u also send me the example of the topology.script.file
and how exactly it works
thanks for referring my problem
Vishal S. Ghawate
DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails.
Re: datanode replication
Posted by Steve Loughran <st...@apache.org>.
Jeff Hammerbacher wrote:
> Hey Vishal,
>
> Check out the chooseTarget() method(s) of ReplicationTargetChooser.java in
> the org.apache.hadoop.hdfs.server.namenode package:
> http://svn.apache.org/viewvc/hadoop/core/trunk/src/hdfs/org/apache/hadoop/hdfs/server/namenode/ReplicationTargetChooser.java?view=markup
> .
>
> In words: assuming you're using the default replication level (3), the
> default strategy will put one block on the local node, one on a node in a
> remote rack, and another on that same remote rack.
>
> Note that HADOOP-3799 (http://issues.apache.org/jira/browse/HADOOP-3799)
> proposes making this strategy pluggable.
>
Yes, there's some good reasons for having different placement algorithms
for different datacentres, and I could even imagine different MR
sequences providing hints about where they want data, depending on what
they want to do afterwards
Re: datanode replication
Posted by Jeff Hammerbacher <ha...@cloudera.com>.
Hey Vishal,
Check out the chooseTarget() method(s) of ReplicationTargetChooser.java in
the org.apache.hadoop.hdfs.server.namenode package:
http://svn.apache.org/viewvc/hadoop/core/trunk/src/hdfs/org/apache/hadoop/hdfs/server/namenode/ReplicationTargetChooser.java?view=markup
.
In words: assuming you're using the default replication level (3), the
default strategy will put one block on the local node, one on a node in a
remote rack, and another on that same remote rack.
Note that HADOOP-3799 (http://issues.apache.org/jira/browse/HADOOP-3799)
proposes making this strategy pluggable.
As for the example of a topology.script file, I'll let others handle that.
Regards,
Jeff
On Sun, May 10, 2009 at 11:20 PM, Vishal Ghawate <
vishal_ghawate@persistent.co.in> wrote:
>
>
> hi,
> i just cant resolve how the namenode decide on where to place the replicas
> of the datanode
> can u also send me the example of the topology.script.file
> and how exactly it works
> thanks for referring my problem
>
> Vishal S. Ghawate
>
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which is
> the property of Persistent Systems Ltd. It is intended only for the use of
> the individual or entity to which it is addressed. If you are not the
> intended recipient, you are not authorized to read, retain, copy, print,
> distribute or use this message. If you have received this communication in
> error, please notify the sender and delete all copies of this message.
> Persistent Systems Ltd. does not accept any liability for virus infected
> mails.
>