You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Rui Shi <sh...@yahoo.com> on 2008/04/06 03:00:20 UTC

Datanode not found

Hi,

I am configuring a cluster using 0.16.2. But when the cluster starts, it can not find any live datanode. I check the logs of one datanodes, which seems hanging in startup:
2008-04-05 17:48:17,967 INFO org.apache.hadoop.dfs.DataNode: Registered FSDatasetStatusMBean
2008-04-05 17:48:17,968 INFO org.apache.hadoop.dfs.DataNode: Opened server at 50010
2008-04-05 17:48:17,971 INFO org.apache.hadoop.dfs.DataNode: Balancing bandwith is 1048576 bytes/s
2008-04-05 17:48:18,465 INFO org.mortbay.util.Credential: Checking Resource aliases
2008-04-05 17:48:18,653 INFO org.mortbay.http.HttpServer: Version Jetty/5.1.4
2008-04-05 17:48:19,784 INFO org.mortbay.util.Container: Started org.mortbay.jetty.servlet.WebApplicationHandler@112e7f7
2008-04-05 17:48:19,849 INFO org.mortbay.util.Container: Started WebApplicationContext[/,/]
2008-04-05 17:48:19,850 INFO org.mortbay.util.Container: Started HttpContext[/logs,/logs]
2008-04-05 17:48:19,851 INFO org.mortbay.util.Container: Started HttpContext[/static,/static]
2008-04-05 17:48:19,854 INFO org.mortbay.http.SocketListener: Started SocketListener on 0.0.0.0:50075
2008-04-05 17:48:19,855 INFO org.mortbay.util.Container: Started org.mortbay.jetty.Server@13f991Does anyone know how to resolve it?

Thanks,

Rui



      ____________________________________________________________________________________
You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost.  
http://tc.deals.yahoo.com/tc/blockbuster/text5.com

Re: Datanode not found

Posted by Tim Nelson <ha...@enigmasecurity.com>.
I had been getting this exact same problem on my cluster. The solution I 
*finally* came up with (after 1-2 weeks of messing) was to manually 
insert a unique datanode ID into the datadir/current/VERSION file. If 
your problem is the same as mine your VERSION file may look like:

#Fri Apr 04 00:43:15 CDT 2008
namespaceID=1031383270
storageID=
cTime=0
storageType=DATA_NODE
layoutVersion=-11

Or something like that with a blank storageID field. The normal value 
for this field is DS-(random ~10 digit number)-IP-dataport-(random ~13 
digit number). I don't think this value really matters *what* it is so 
long as it's unique for each datanode (it could just be an IP address 
since usually in production you only run 1 datanode per machine). After 
I filled the storageID field in my datanodes started connecting to my 
namenode properly. I plan to try and debug this and submit a patch but 
I'm having problems getting my development environment to pass all the 
junit tests on vanilla code. Please let me know if this hack works for you.

Regards,
Tim Nelson

Rui Shi wrote:

>Hi,
>
>I am configuring a cluster using 0.16.2. But when the cluster starts, it can not find any live datanode. I check the logs of one datanodes, which seems hanging in startup:
>2008-04-05 17:48:17,967 INFO org.apache.hadoop.dfs.DataNode: Registered FSDatasetStatusMBean
>2008-04-05 17:48:17,968 INFO org.apache.hadoop.dfs.DataNode: Opened server at 50010
>2008-04-05 17:48:17,971 INFO org.apache.hadoop.dfs.DataNode: Balancing bandwith is 1048576 bytes/s
>2008-04-05 17:48:18,465 INFO org.mortbay.util.Credential: Checking Resource aliases
>2008-04-05 17:48:18,653 INFO org.mortbay.http.HttpServer: Version Jetty/5.1.4
>2008-04-05 17:48:19,784 INFO org.mortbay.util.Container: Started org.mortbay.jetty.servlet.WebApplicationHandler@112e7f7
>2008-04-05 17:48:19,849 INFO org.mortbay.util.Container: Started WebApplicationContext[/,/]
>2008-04-05 17:48:19,850 INFO org.mortbay.util.Container: Started HttpContext[/logs,/logs]
>2008-04-05 17:48:19,851 INFO org.mortbay.util.Container: Started HttpContext[/static,/static]
>2008-04-05 17:48:19,854 INFO org.mortbay.http.SocketListener: Started SocketListener on 0.0.0.0:50075
>2008-04-05 17:48:19,855 INFO org.mortbay.util.Container: Started org.mortbay.jetty.Server@13f991Does anyone know how to resolve it?
>
>Thanks,
>
>Rui
>
>
>
>      ____________________________________________________________________________________
>You rock. That's why Blockbuster's offering you one month of Blockbuster Total Access, No Cost.  
>http://tc.deals.yahoo.com/tc/blockbuster/text5.com
>  
>