problem with rack-awareness

[elton sky <el...@gmail.com>]

hello,

I am trying to separate my 6 nodes onto 2 different racks.
For test purpose, I wrote a bash file which smply returns "rack0" all the
time. And I add property "topology.script.file.name" in core-site.xml.

When I restart by start-dfs.sh, the namenode could not find any datanode at
all. All datanodes are lost somehow. If I remove "topology.script.file.name"
from conf, things back to normal, i.e. all datanodes are under
"default-rack".

I don't why datanode couldn't register to namenode when using rack. Any
ideas?

Re: problem with rack-awareness

[Edward Capriolo <ed...@gmail.com>]

On Fri, Jul 2, 2010 at 2:27 PM, Allen Wittenauer
<aw...@linkedin.com> wrote:
>
> On Jul 1, 2010, at 7:50 PM, elton sky wrote:
>
>> hello,
>>
>> I am trying to separate my 6 nodes onto 2 different racks.
>> For test purpose, I wrote a bash file which smply returns "rack0" all the
>> time. And I add property "topology.script.file.name" in core-site.xml.
>
> rack0 or /rack0?
>
> I think the leading / is required.
>
>
>>
>> When I restart by start-dfs.sh, the namenode could not find any datanode at
>> all. All datanodes are lost somehow. If I remove "topology.script.file.name"
>> from conf, things back to normal, i.e. all datanodes are under
>> "default-rack".
>>
>> I don't why datanode couldn't register to namenode when using rack. Any
>> ideas?
>
> It sounds like your topology script isn't returning the data in the correct format.

If you want a clear cut example refer to the hadoop wiki:

http://wiki.apache.org/hadoop/topology_rack_awareness_scripts

Re: problem with rack-awareness

[Allen Wittenauer <aw...@linkedin.com>]

On Jul 1, 2010, at 7:50 PM, elton sky wrote:

> hello,
> 
> I am trying to separate my 6 nodes onto 2 different racks.
> For test purpose, I wrote a bash file which smply returns "rack0" all the
> time. And I add property "topology.script.file.name" in core-site.xml.

rack0 or /rack0?

I think the leading / is required.


> 
> When I restart by start-dfs.sh, the namenode could not find any datanode at
> all. All datanodes are lost somehow. If I remove "topology.script.file.name"
> from conf, things back to normal, i.e. all datanodes are under
> "default-rack".
> 
> I don't why datanode couldn't register to namenode when using rack. Any
> ideas?

It sounds like your topology script isn't returning the data in the correct format.

RE: problem with rack-awareness

[Michael Segel <mi...@hotmail.com>]

A couple of things...

Does your script have a default rack defined? So if it can't find your machine, you default to it being on rack_default ?
(You could use rack0, but then you have a problem will you know what's really in rack0 or what's kicking out the default value?)

The other issue is that you may have to put in your machine name, the fully qualified name and the IP address.
I'm not sure which is getting passed in so I have 3 lists that I maintain in the script.

HTH

-Mike


> Date: Fri, 2 Jul 2010 12:50:26 +1000
> Subject: problem with rack-awareness
> From: eltonsky9404@gmail.com
> To: common-user@hadoop.apache.org
> 
> hello,
> 
> I am trying to separate my 6 nodes onto 2 different racks.
> For test purpose, I wrote a bash file which smply returns "rack0" all the
> time. And I add property "topology.script.file.name" in core-site.xml.
> 
> When I restart by start-dfs.sh, the namenode could not find any datanode at
> all. All datanodes are lost somehow. If I remove "topology.script.file.name"
> from conf, things back to normal, i.e. all datanodes are under
> "default-rack".
> 
> I don't why datanode couldn't register to namenode when using rack. Any
> ideas?
 		 	   		  
_________________________________________________________________
Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox.
http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_1