You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Marc Leavitt <le...@gmail.com> on 2011/03/24 19:35:18 UTC

Re-generate datanode storageID?

I am setting up a (very) small Hadoop/CDH3 beta 4 cluster in virtual machines to do some initial feasibility work.  I proceeded by progressing through the Cloudera documentation standalone -> pseudo-cluster -> cluster with a single VM and then, when I had it stable(-ish) I copied the VM to a couple of slave images.

The good news is that all three tasktrackers show up.
The bad news is that only one datanode shows up.

And, after after "some" research, I am pretty sure I know why - each of the datanodes is trying to claim the same storageID (as defined in .../cache/hdfs/dfs/data/current/VERSION.

So, my question is "How do I resolve the collision of the storageIDs?"

Thanks!
-mgl





Re: Re-generate datanode storageID?

Posted by Marc Leavitt <le...@gmail.com>.
Worked perfectly.

Thanks Niels!

-mgl

On Mar 24, 2011, at 12:48 PM, Niels Basjes wrote:

> Hi,
> 
> To solve that simply do the following on the problematic nodes:
> 1) Stop the datanode (probably not running)
> 2) Remove everything inside the .../cache/hdfs/
> 3) Start the datanode again.
> 
> Note: With cloudera always use "service" way to stop/start hadoop software!
> service hadoop-0.20-datanode stop
> 
> 2011/3/24 Marc Leavitt <le...@gmail.com>:
>> I am setting up a (very) small Hadoop/CDH3 beta 4 cluster in virtual machines to do some initial feasibility work.  I proceeded by progressing through the Cloudera documentation standalone -> pseudo-cluster -> cluster with a single VM and then, when I had it stable(-ish) I copied the VM to a couple of slave images.
>> 
>> The good news is that all three tasktrackers show up.
>> The bad news is that only one datanode shows up.
>> 
>> And, after after "some" research, I am pretty sure I know why - each of the datanodes is trying to claim the same storageID (as defined in .../cache/hdfs/dfs/data/current/VERSION.
>> 
>> So, my question is "How do I resolve the collision of the storageIDs?"
>> 
>> Thanks!
>> -mgl
>> 
>> 
>> 
>> 
>> 
> 
> 
> 
> -- 
> Met vriendelijke groeten,
> 
> Niels Basjes





Re: Re-generate datanode storageID?

Posted by Niels Basjes <Ni...@basjes.nl>.
Hi,

To solve that simply do the following on the problematic nodes:
1) Stop the datanode (probably not running)
2) Remove everything inside the .../cache/hdfs/
3) Start the datanode again.

Note: With cloudera always use "service" way to stop/start hadoop software!
service hadoop-0.20-datanode stop

2011/3/24 Marc Leavitt <le...@gmail.com>:
> I am setting up a (very) small Hadoop/CDH3 beta 4 cluster in virtual machines to do some initial feasibility work.  I proceeded by progressing through the Cloudera documentation standalone -> pseudo-cluster -> cluster with a single VM and then, when I had it stable(-ish) I copied the VM to a couple of slave images.
>
> The good news is that all three tasktrackers show up.
> The bad news is that only one datanode shows up.
>
> And, after after "some" research, I am pretty sure I know why - each of the datanodes is trying to claim the same storageID (as defined in .../cache/hdfs/dfs/data/current/VERSION.
>
> So, my question is "How do I resolve the collision of the storageIDs?"
>
> Thanks!
> -mgl
>
>
>
>
>



-- 
Met vriendelijke groeten,

Niels Basjes