Anyone knows how to input the cluster racks architecture information?

[Richard Zhang <ri...@gmail.com>]

Hi Guys:
I am running map reduce job on the 20 nodes cluster. I have the architecture
of the cluster such as how the racks are organized. How to input these
information into hadoop?
I suppose these information should be useful for hadoop. But, when searching
the code, I did not find any class/methods are using such information.
Does anyone knows how to input the cluster racks architecture information
into hadoop and how hadoop uses this?
Thanks.
Richard

Re: Anyone knows how to input the cluster racks architecture information?

[Steve Loughran <st...@apache.org>]

Richard Zhang wrote:
> Hi Guys:
> I am running map reduce job on the 20 nodes cluster. I have the architecture
> of the cluster such as how the racks are organized. How to input these
> information into hadoop?
> I suppose these information should be useful for hadoop. But, when searching
> the code, I did not find any class/methods are using such information.
> Does anyone knows how to input the cluster racks architecture information
> into hadoop and how hadoop uses this?
> Thanks.
> Richard
> 

There;s something you can configure in the site.xml, that names a shell 
script that returns the name of the rack a node lives in; I forget the 
details, but it should be documented in the configuration docs. There is 
some stuff being developed that works out the rack from the IPv4 
address; if you can determine rack by some kind of mask on the address 
then  that would meet your needs -once its checked in

-- 
Steve Loughran                  http://www.1060.org/blogxter/publish/5
Author: Ant in Action           http://antbook.org/

Re: Anyone knows how to input the cluster racks architecture information?

[Shihua Ma <be...@gmail.com>]

Richard Zhang wrote:


Richard Zhang-2 wrote:
> 
> Hi Guys:
> I am running map reduce job on the 20 nodes cluster. I have the
> architecture
> of the cluster such as how the racks are organized. How to input these
> information into hadoop?
> I suppose these information should be useful for hadoop. But, when
> searching
> the code, I did not find any class/methods are using such information.
> Does anyone knows how to input the cluster racks architecture information
> into hadoop and how hadoop uses this?
> Thanks.
> Richard
> 
> 

The document of  
https://issues.apache.org/jira/secure/attachment/12345251/Rack_aware_HDFS_proposal.pdf
rack awareness  can answer your question that how to input the cluster racks
architecture information into hadoop.You can specifies a script through
configuration of "dfs.network.script" that print the network location path
of the current machine.


Josh ma :  http://www.hadoop.org.cn http://www.hadoop.org.cn 



-- 
View this message in context: http://www.nabble.com/Anyone-knows-how-to-input-the-cluster-racks-architecture-information--tp18145164p18170704.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.