You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by ArunKumar <ar...@gmail.com> on 2011/12/20 06:29:24 UTC

Controlling the placement of data in Hadoop Cluster nodes ?

Hi guys !

I don't want to use automatic placement of HDFS for some reasons.
Can i control the placement of data in Hadoop Cluster nodes ?
Say placing data1 on Node3 , data2 on Node5 ,etc

Can anyone guide me in this regard ?


Arun


--
View this message in context: http://lucene.472066.n3.nabble.com/Controlling-the-placement-of-data-in-Hadoop-Cluster-nodes-tp3600410p3600410.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.

Re: Controlling the placement of data in Hadoop Cluster nodes ?

Posted by Harsh J <ha...@cloudera.com>.
Moving to hdfs-dev@hadoop.apache.org. In future, please use the appropriate lists to post your questions, when they are about HDFS: http://hadoop.apache.org/hdfs/mailing_lists.html

You can implement a custom BlockPlacementPolicy in 0.21+ versions of Hadoop, and use that to achieve this. Should be fairly straight forward to do for test purposes -- just take a look at the interface.

On 20-Dec-2011, at 10:59 AM, ArunKumar wrote:

> Hi guys !
> 
> I don't want to use automatic placement of HDFS for some reasons.
> Can i control the placement of data in Hadoop Cluster nodes ?
> Say placing data1 on Node3 , data2 on Node5 ,etc
> 
> Can anyone guide me in this regard ?
> 
> 
> Arun
> 
> 
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Controlling-the-placement-of-data-in-Hadoop-Cluster-nodes-tp3600410p3600410.html
> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.

Re: Controlling the placement of data in Hadoop Cluster nodes ?

Posted by Harsh J <ha...@cloudera.com>.
Moving to hdfs-dev@hadoop.apache.org. In future, please use the appropriate lists to post your questions, when they are about HDFS: http://hadoop.apache.org/hdfs/mailing_lists.html

You can implement a custom BlockPlacementPolicy in 0.21+ versions of Hadoop, and use that to achieve this. Should be fairly straight forward to do for test purposes -- just take a look at the interface.

On 20-Dec-2011, at 10:59 AM, ArunKumar wrote:

> Hi guys !
> 
> I don't want to use automatic placement of HDFS for some reasons.
> Can i control the placement of data in Hadoop Cluster nodes ?
> Say placing data1 on Node3 , data2 on Node5 ,etc
> 
> Can anyone guide me in this regard ?
> 
> 
> Arun
> 
> 
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Controlling-the-placement-of-data-in-Hadoop-Cluster-nodes-tp3600410p3600410.html
> Sent from the Hadoop lucene-users mailing list archive at Nabble.com.