You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Marc Leavitt <le...@gmail.com> on 2011/03/24 19:35:18 UTC
Re-generate datanode storageID?
I am setting up a (very) small Hadoop/CDH3 beta 4 cluster in virtual machines to do some initial feasibility work. I proceeded by progressing through the Cloudera documentation standalone -> pseudo-cluster -> cluster with a single VM and then, when I had it stable(-ish) I copied the VM to a couple of slave images.
The good news is that all three tasktrackers show up.
The bad news is that only one datanode shows up.
And, after after "some" research, I am pretty sure I know why - each of the datanodes is trying to claim the same storageID (as defined in .../cache/hdfs/dfs/data/current/VERSION.
So, my question is "How do I resolve the collision of the storageIDs?"
Thanks!
-mgl
Re: Re-generate datanode storageID?
Posted by Marc Leavitt <le...@gmail.com>.
Worked perfectly.
Thanks Niels!
-mgl
On Mar 24, 2011, at 12:48 PM, Niels Basjes wrote:
> Hi,
>
> To solve that simply do the following on the problematic nodes:
> 1) Stop the datanode (probably not running)
> 2) Remove everything inside the .../cache/hdfs/
> 3) Start the datanode again.
>
> Note: With cloudera always use "service" way to stop/start hadoop software!
> service hadoop-0.20-datanode stop
>
> 2011/3/24 Marc Leavitt <le...@gmail.com>:
>> I am setting up a (very) small Hadoop/CDH3 beta 4 cluster in virtual machines to do some initial feasibility work. I proceeded by progressing through the Cloudera documentation standalone -> pseudo-cluster -> cluster with a single VM and then, when I had it stable(-ish) I copied the VM to a couple of slave images.
>>
>> The good news is that all three tasktrackers show up.
>> The bad news is that only one datanode shows up.
>>
>> And, after after "some" research, I am pretty sure I know why - each of the datanodes is trying to claim the same storageID (as defined in .../cache/hdfs/dfs/data/current/VERSION.
>>
>> So, my question is "How do I resolve the collision of the storageIDs?"
>>
>> Thanks!
>> -mgl
>>
>>
>>
>>
>>
>
>
>
> --
> Met vriendelijke groeten,
>
> Niels Basjes
Re: Re-generate datanode storageID?
Posted by Niels Basjes <Ni...@basjes.nl>.
Hi,
To solve that simply do the following on the problematic nodes:
1) Stop the datanode (probably not running)
2) Remove everything inside the .../cache/hdfs/
3) Start the datanode again.
Note: With cloudera always use "service" way to stop/start hadoop software!
service hadoop-0.20-datanode stop
2011/3/24 Marc Leavitt <le...@gmail.com>:
> I am setting up a (very) small Hadoop/CDH3 beta 4 cluster in virtual machines to do some initial feasibility work. I proceeded by progressing through the Cloudera documentation standalone -> pseudo-cluster -> cluster with a single VM and then, when I had it stable(-ish) I copied the VM to a couple of slave images.
>
> The good news is that all three tasktrackers show up.
> The bad news is that only one datanode shows up.
>
> And, after after "some" research, I am pretty sure I know why - each of the datanodes is trying to claim the same storageID (as defined in .../cache/hdfs/dfs/data/current/VERSION.
>
> So, my question is "How do I resolve the collision of the storageIDs?"
>
> Thanks!
> -mgl
>
>
>
>
>
--
Met vriendelijke groeten,
Niels Basjes