You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Adarsh Sharma <ad...@orkash.com> on 2011/04/28 07:32:26 UTC

Namenode error :- FSNamesystem initialization failed

Dear all,

I have a running 4-node Hadoop cluster and some data stored in HDFS.

Today by mistake,I start the hadoop cluster with root user.

root>bin/start-all.sh

After correcting my mistake , when I try to run with the hadoop user, my 
Namenode fails with the below exception\ :


STARTUP_MSG:   build = 
https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 
911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
************************************************************/
2011-04-28 10:53:49,516 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: 
Initializing RPC Metrics with hostName=NameNode, port=54310
2011-04-28 10:53:49,520 INFO 
org.apache.hadoop.hdfs.server.namenode.NameNode: Namenode up at: 
ws-test/192.168.0.207:54310
2011-04-28 10:53:49,523 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: 
Initializing JVM Metrics with processName=NameNode, sessionId=null
2011-04-28 10:53:49,527 INFO 
org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics: 
Initializing NameNodeMeterics using context 
object:org.apache.hadoop.metrics.ganglia.GangliaContext
2011-04-28 10:53:49,573 INFO 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner=hadoop,hadoop
2011-04-28 10:53:49,573 INFO 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup=supergroup
2011-04-28 10:53:49,573 INFO 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: 
isPermissionEnabled=false
2011-04-28 10:53:49,580 INFO 
org.apache.hadoop.hdfs.server.namenode.metrics.FSNamesystemMetrics: 
Initializing FSNamesystemMetrics using context 
object:org.apache.hadoop.metrics.ganglia.GangliaContext
2011-04-28 10:53:49,581 INFO 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered 
FSNamesystemStatusMBean
2011-04-28 10:53:49,608 ERROR 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem 
initialization failed.
java.io.IOException: NameNode is not formatted.
        at 
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:317)
        at 
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:87)
        at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:311)
        at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:292)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:201)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:279)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:956)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:965)
2011-04-28 10:53:49,609 INFO org.apache.hadoop.ipc.Server: Stopping 
server on 54310
2011-04-28 10:53:49,609 ERROR 
org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException: 
NameNode is not formatted.
        at 
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:317)
        at 
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:87)
        at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:311)
        at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:292)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:201)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:279)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:956)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:965)

2011-04-28 10:53:49,610 INFO 
org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:
/************************************************************

I know if i delete the data & mapred directory of all node & issue the 
bin/hadoop namenode -format command , my cluster starts again but 
previous data lost.

Is it possible to start cluster without the data loss. If yes, Please 
let me know



Thanks



Re: Namenode error :- FSNamesystem initialization failed

Posted by Adarsh Sharma <ad...@orkash.com>.
 I start the cluster by formatting the namenode after deleting all 
folders (check,name,data,mapred) & previous data is lost. It's urgent

Now for my knowledge base, Don't we start our namenode from our SNN 
check directory if by mistake we format the namenode.

Some latest data might be missing.

If yes, what are the steps to do this.



Thanks


Harsh J wrote:
> I believe the format must have killed it all away. You might be lucky
> with the SNN location if you were running it, though - could you check
> in the SNN's directory for valid fsimage?
>
> On Thu, Apr 28, 2011 at 12:53 PM, Adarsh Sharma
> <ad...@orkash.com> wrote:
>   
>> I correct my posting that this problem arises due to running a script that
>> internally issue below commands as root user:
>>
>>     
>>> bin/hadoop namenode -format
>>> bin/start-all.sh
>>>       
>> No is it possible to start the previous cluster with the previous data or
>> not.
>>
>> Thanks
>>
>>
>> Adarsh Sharma wrote:
>>     
>>> Thanks harsh,
>>>
>>> My dfs.name.dir is /home/hadoop/project/hadoop-0.20.2/name
>>> & there is only one file fsimage inimage directory & current directory is
>>> empty.
>>>
>>> But there must be four files fsname, edits, image etc in current
>>> directory. How they got deleted even I don't issue format command yet.
>>>
>>> I think there must be a remedy to get the previous data.
>>>
>>>
>>>
>>>
>>> Harsh J wrote:
>>>       
>>>> Hello Adarsh,
>>>>
>>>> On Thu, Apr 28, 2011 at 11:02 AM, Adarsh Sharma
>>>> <ad...@orkash.com> wrote:
>>>>
>>>>         
>>>>> After correcting my mistake , when I try to run with the hadoop user, my
>>>>> Namenode fails with the below exception\ :
>>>>> 2011-04-28 10:53:49,608 ERROR
>>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>>>>> initialization failed.
>>>>> java.io.IOException: NameNode is not formatted.
>>>>>      at
>>>>>
>>>>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:317)
>>>>>
>>>>>           
>>>> Do you still have the valid contents in your good ${dfs.name.dir}? You
>>>> should be able to recover with that.
>>>>
>>>>
>>>>         
>>>       
>>     
>
>
>
>   


Re: Namenode error :- FSNamesystem initialization failed

Posted by Harsh J <ha...@cloudera.com>.
I believe the format must have killed it all away. You might be lucky
with the SNN location if you were running it, though - could you check
in the SNN's directory for valid fsimage?

On Thu, Apr 28, 2011 at 12:53 PM, Adarsh Sharma
<ad...@orkash.com> wrote:
> I correct my posting that this problem arises due to running a script that
> internally issue below commands as root user:
>
>> bin/hadoop namenode -format
>>bin/start-all.sh
>
> No is it possible to start the previous cluster with the previous data or
> not.
>
> Thanks
>
>
> Adarsh Sharma wrote:
>>
>> Thanks harsh,
>>
>> My dfs.name.dir is /home/hadoop/project/hadoop-0.20.2/name
>> & there is only one file fsimage inimage directory & current directory is
>> empty.
>>
>> But there must be four files fsname, edits, image etc in current
>> directory. How they got deleted even I don't issue format command yet.
>>
>> I think there must be a remedy to get the previous data.
>>
>>
>>
>>
>> Harsh J wrote:
>>>
>>> Hello Adarsh,
>>>
>>> On Thu, Apr 28, 2011 at 11:02 AM, Adarsh Sharma
>>> <ad...@orkash.com> wrote:
>>>
>>>>
>>>> After correcting my mistake , when I try to run with the hadoop user, my
>>>> Namenode fails with the below exception\ :
>>>> 2011-04-28 10:53:49,608 ERROR
>>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>>>> initialization failed.
>>>> java.io.IOException: NameNode is not formatted.
>>>>      at
>>>>
>>>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:317)
>>>>
>>>
>>> Do you still have the valid contents in your good ${dfs.name.dir}? You
>>> should be able to recover with that.
>>>
>>>
>>
>>
>
>



-- 
Harsh J

Re: Namenode error :- FSNamesystem initialization failed

Posted by Adarsh Sharma <ad...@orkash.com>.
I correct my posting that this problem arises due to running a script 
that internally issue below commands as root user:

 > bin/hadoop namenode -format
 >bin/start-all.sh

No is it possible to start the previous cluster with the previous data 
or not.

Thanks


Adarsh Sharma wrote:
> Thanks harsh,
>
> My dfs.name.dir is /home/hadoop/project/hadoop-0.20.2/name
> & there is only one file fsimage inimage directory & current directory 
> is empty.
>
> But there must be four files fsname, edits, image etc in current 
> directory. How they got deleted even I don't issue format command yet.
>
> I think there must be a remedy to get the previous data.
>
>
>
>
> Harsh J wrote:
>> Hello Adarsh,
>>
>> On Thu, Apr 28, 2011 at 11:02 AM, Adarsh Sharma
>> <ad...@orkash.com> wrote:
>>  
>>> After correcting my mistake , when I try to run with the hadoop 
>>> user, my
>>> Namenode fails with the below exception\ :
>>> 2011-04-28 10:53:49,608 ERROR
>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>>> initialization failed.
>>> java.io.IOException: NameNode is not formatted.
>>>       at
>>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:317) 
>>>
>>>     
>>
>> Do you still have the valid contents in your good ${dfs.name.dir}? You
>> should be able to recover with that.
>>
>>   
>
>


Re: Namenode error :- FSNamesystem initialization failed

Posted by Adarsh Sharma <ad...@orkash.com>.
Thanks harsh,

My dfs.name.dir is /home/hadoop/project/hadoop-0.20.2/name
& there is only one file fsimage inimage directory & current directory 
is empty.

But there must be four files fsname, edits, image etc in current 
directory. How they got deleted even I don't issue format command yet.

I think there must be a remedy to get the previous data.




Harsh J wrote:
> Hello Adarsh,
>
> On Thu, Apr 28, 2011 at 11:02 AM, Adarsh Sharma
> <ad...@orkash.com> wrote:
>   
>> After correcting my mistake , when I try to run with the hadoop user, my
>> Namenode fails with the below exception\ :
>> 2011-04-28 10:53:49,608 ERROR
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>> initialization failed.
>> java.io.IOException: NameNode is not formatted.
>>       at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:317)
>>     
>
> Do you still have the valid contents in your good ${dfs.name.dir}? You
> should be able to recover with that.
>
>   


Re: Namenode error :- FSNamesystem initialization failed

Posted by Harsh J <ha...@cloudera.com>.
Hello Adarsh,

On Thu, Apr 28, 2011 at 11:02 AM, Adarsh Sharma
<ad...@orkash.com> wrote:
> After correcting my mistake , when I try to run with the hadoop user, my
> Namenode fails with the below exception\ :
> 2011-04-28 10:53:49,608 ERROR
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> initialization failed.
> java.io.IOException: NameNode is not formatted.
>       at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:317)

Do you still have the valid contents in your good ${dfs.name.dir}? You
should be able to recover with that.

-- 
Harsh J