You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Abhay Ratnaparkhi <ab...@gmail.com> on 2012/08/24 09:28:41 UTC
namenode not starting
Hello,
I had a running hadoop cluster.
I restarted it and after that namenode is unable to start. I am getting
error saying that it's not formatted. :(
Is it possible to recover the data on HDFS?
2012-08-24 03:17:55,378 ERROR
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
initialization failed.
java.io.IOException: NameNode is not formatted.
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
2012-08-24 03:17:55,380 ERROR
org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
NameNode is not formatted.
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
Regards,
Abhay
Re: namenode not starting
Posted by Nitin Pawar <ni...@gmail.com>.
did you run the command bin/hadoop namenode -format before starting
the namenode ?
On Fri, Aug 24, 2012 at 12:58 PM, Abhay Ratnaparkhi
<ab...@gmail.com> wrote:
> Hello,
>
> I had a running hadoop cluster.
> I restarted it and after that namenode is unable to start. I am getting
> error saying that it's not formatted. :(
> Is it possible to recover the data on HDFS?
>
> 2012-08-24 03:17:55,378 ERROR
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> initialization failed.
> java.io.IOException: NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
> 2012-08-24 03:17:55,380 ERROR
> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>
> Regards,
> Abhay
>
>
--
Nitin Pawar
Re: namenode not starting
Posted by Nitin Pawar <ni...@gmail.com>.
did you run the command bin/hadoop namenode -format before starting
the namenode ?
On Fri, Aug 24, 2012 at 12:58 PM, Abhay Ratnaparkhi
<ab...@gmail.com> wrote:
> Hello,
>
> I had a running hadoop cluster.
> I restarted it and after that namenode is unable to start. I am getting
> error saying that it's not formatted. :(
> Is it possible to recover the data on HDFS?
>
> 2012-08-24 03:17:55,378 ERROR
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> initialization failed.
> java.io.IOException: NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
> 2012-08-24 03:17:55,380 ERROR
> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>
> Regards,
> Abhay
>
>
--
Nitin Pawar
Re: namenode not starting
Posted by Mohammad Tariq <do...@gmail.com>.
Hello Abhay,
Along with dfs.name.dir, also include dfs.data.dir in hdfs-site.xml.
On Monday, August 27, 2012, Abhay Ratnaparkhi <ab...@gmail.com>
wrote:
> Thank you Harsh,
> I have set "dfs.name.dir" explicitly. Still don't know why the data loss
has happened.
> <property>
> <name>dfs.name.dir</name>
> <value>/wsadfs/${host.name}/name</value>
> <description>Determines where on the local filesystem the DFS name node
> should store the name table. If this is a comma-delimited list
> of directories then the name table is replicated in all of the
> directories, for redundancy. </description>
> </property>
> The secondary namenode was same as namenode. Does this affect anyway
since path of "dfs.name.dir" were same?
> I have now configured another machine as secondary namenode.
> I have now formatted the filesystem since not seen any way of
recovering.
> I have some questions.
> 1. Apart from setting secondary namenode what are the other techniques
used for namenode directory backups?
> 2. Is there any way or tools to recover some of the data even if namenode
crashes.
> Regards,
> Abhay
>
>
>
>
> On Sat, Aug 25, 2012 at 7:45 PM, Harsh J <ha...@cloudera.com> wrote:
>
> Abhay,
>
> I suspect that if you haven't set your dfs.name.dir explicitly, then
> you haven't set fs.checkpoint.dir either, and since both use
> hadoop.tmp.dir paths, you may have lost your data completely and there
> is no recovery possible now.
>
> On Fri, Aug 24, 2012 at 1:10 PM, Abhay Ratnaparkhi
> <ab...@gmail.com> wrote:
>> Hello,
>>
>> I was using cluster for long time and not formatted the namenode.
>> I ran bin/stop-all.sh and bin/start-all.sh scripts only.
>>
>> I am using NFS for dfs.name.dir.
>> hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any way
to
>> recover the data?
>>
>> Thanks,
>> Abhay
>>
>>
>> On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
>>>
>>> Hi Abhay
>>>
>>> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to
>>> /tmp the contents would be deleted on a OS restart. You need to change
this
>>> location before you start your NN.
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ________________________________
>>> From: Abhay Ratnaparkhi <ab...@gmail.com>
>>> Date: Fri, 24 Aug 2012 12:58:41 +0530
>>> To: <us...@hadoop.apache.org>
>>> ReplyTo: user@hadoop.apache.org
>>> Subject: namenode not starting
>>>
>>> Hello,
>>>
>>> I had a running hadoop cluster.
>>> I restarted it and after that namenode is unable to start. I am getting
>>> error saying that it's not formatted. :(
>>> Is it possible to recover the data on HDFS?
>>>
>>> 2012-08-24 03:17:55,378 ERROR
>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>>> initialization failed.
>>> java.io.IOException: NameNode is not formatted.
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>>> at
>>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>>> 2012-08-24 03:17:55,380 ERROR
>>> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
>>> NameNode is not formatted.
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>>> at
>>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(Nam
--
Regards,
Mohammad Tariq
RE: namenode not starting
Posted by Leo Leung <ll...@ddn.com>.
I suggested a while back to Abhay, he should check/track-down the data on his NFS server / dir.
Now the ${host.name} looks odd.
I hope this is something you (Abhay) edited out and not something that "IS" in the xml file.
If it is, please fix that. It needs to be a static name not a variable.
-----Original Message-----
From: Harsh J [mailto:harsh@cloudera.com]
Sent: Monday, August 27, 2012 12:30 AM
To: user@hadoop.apache.org
Subject: Re: namenode not starting
Abhay,
On Mon, Aug 27, 2012 at 11:19 AM, Abhay Ratnaparkhi <ab...@gmail.com> wrote:
> Thank you Harsh,
>
> I have set "dfs.name.dir" explicitly. Still don't know why the data
> loss has happened.
>
> <property>
> <name>dfs.name.dir</name>
> <value>/wsadfs/${host.name}/name</value>
> <description>Determines where on the local filesystem the DFS name node
> should store the name table. If this is a comma-delimited list
> of directories then the name table is replicated in all of the
> directories, for redundancy. </description> </property>
Sorry, I missed you had said NFS above. Is the data not present at all in that directory there?
> The secondary namenode was same as namenode. Does this affect anyway
> since path of "dfs.name.dir" were same?
> I have now configured another machine as secondary namenode.
> I have now formatted the filesystem since not seen any way of recovering.
>
> I have some questions.
>
> 1. Apart from setting secondary namenode what are the other techniques
> used for namenode directory backups?
Duplicate dfs.name.dir directories are what we use in production. That is, at least two paths, one local FS and another NFS mounted:
dfs.name.dir = /path/to/local/dfs/name,/path/to/nfs/dfs/name
This will give you two copies of good metadata, and loss of one can still be handled.
> 2. Is there any way or tools to recover some of the data even if
> namenode crashes.
If there's any form of fsimage/edits left, a manual/automated recovery can be made via tools such as oiv/oev and the NN's "-recover" flag, if your version has it, or even with a hexdump and some time.
If there's no trace of fsimage files, its backups from any date, any SNN checkpoints from past, then the metadata is all gone and there's no recovery.
--
Harsh J
RE: namenode not starting
Posted by Leo Leung <ll...@ddn.com>.
I suggested a while back to Abhay, he should check/track-down the data on his NFS server / dir.
Now the ${host.name} looks odd.
I hope this is something you (Abhay) edited out and not something that "IS" in the xml file.
If it is, please fix that. It needs to be a static name not a variable.
-----Original Message-----
From: Harsh J [mailto:harsh@cloudera.com]
Sent: Monday, August 27, 2012 12:30 AM
To: user@hadoop.apache.org
Subject: Re: namenode not starting
Abhay,
On Mon, Aug 27, 2012 at 11:19 AM, Abhay Ratnaparkhi <ab...@gmail.com> wrote:
> Thank you Harsh,
>
> I have set "dfs.name.dir" explicitly. Still don't know why the data
> loss has happened.
>
> <property>
> <name>dfs.name.dir</name>
> <value>/wsadfs/${host.name}/name</value>
> <description>Determines where on the local filesystem the DFS name node
> should store the name table. If this is a comma-delimited list
> of directories then the name table is replicated in all of the
> directories, for redundancy. </description> </property>
Sorry, I missed you had said NFS above. Is the data not present at all in that directory there?
> The secondary namenode was same as namenode. Does this affect anyway
> since path of "dfs.name.dir" were same?
> I have now configured another machine as secondary namenode.
> I have now formatted the filesystem since not seen any way of recovering.
>
> I have some questions.
>
> 1. Apart from setting secondary namenode what are the other techniques
> used for namenode directory backups?
Duplicate dfs.name.dir directories are what we use in production. That is, at least two paths, one local FS and another NFS mounted:
dfs.name.dir = /path/to/local/dfs/name,/path/to/nfs/dfs/name
This will give you two copies of good metadata, and loss of one can still be handled.
> 2. Is there any way or tools to recover some of the data even if
> namenode crashes.
If there's any form of fsimage/edits left, a manual/automated recovery can be made via tools such as oiv/oev and the NN's "-recover" flag, if your version has it, or even with a hexdump and some time.
If there's no trace of fsimage files, its backups from any date, any SNN checkpoints from past, then the metadata is all gone and there's no recovery.
--
Harsh J
RE: namenode not starting
Posted by Leo Leung <ll...@ddn.com>.
I suggested a while back to Abhay, he should check/track-down the data on his NFS server / dir.
Now the ${host.name} looks odd.
I hope this is something you (Abhay) edited out and not something that "IS" in the xml file.
If it is, please fix that. It needs to be a static name not a variable.
-----Original Message-----
From: Harsh J [mailto:harsh@cloudera.com]
Sent: Monday, August 27, 2012 12:30 AM
To: user@hadoop.apache.org
Subject: Re: namenode not starting
Abhay,
On Mon, Aug 27, 2012 at 11:19 AM, Abhay Ratnaparkhi <ab...@gmail.com> wrote:
> Thank you Harsh,
>
> I have set "dfs.name.dir" explicitly. Still don't know why the data
> loss has happened.
>
> <property>
> <name>dfs.name.dir</name>
> <value>/wsadfs/${host.name}/name</value>
> <description>Determines where on the local filesystem the DFS name node
> should store the name table. If this is a comma-delimited list
> of directories then the name table is replicated in all of the
> directories, for redundancy. </description> </property>
Sorry, I missed you had said NFS above. Is the data not present at all in that directory there?
> The secondary namenode was same as namenode. Does this affect anyway
> since path of "dfs.name.dir" were same?
> I have now configured another machine as secondary namenode.
> I have now formatted the filesystem since not seen any way of recovering.
>
> I have some questions.
>
> 1. Apart from setting secondary namenode what are the other techniques
> used for namenode directory backups?
Duplicate dfs.name.dir directories are what we use in production. That is, at least two paths, one local FS and another NFS mounted:
dfs.name.dir = /path/to/local/dfs/name,/path/to/nfs/dfs/name
This will give you two copies of good metadata, and loss of one can still be handled.
> 2. Is there any way or tools to recover some of the data even if
> namenode crashes.
If there's any form of fsimage/edits left, a manual/automated recovery can be made via tools such as oiv/oev and the NN's "-recover" flag, if your version has it, or even with a hexdump and some time.
If there's no trace of fsimage files, its backups from any date, any SNN checkpoints from past, then the metadata is all gone and there's no recovery.
--
Harsh J
RE: namenode not starting
Posted by Leo Leung <ll...@ddn.com>.
I suggested a while back to Abhay, he should check/track-down the data on his NFS server / dir.
Now the ${host.name} looks odd.
I hope this is something you (Abhay) edited out and not something that "IS" in the xml file.
If it is, please fix that. It needs to be a static name not a variable.
-----Original Message-----
From: Harsh J [mailto:harsh@cloudera.com]
Sent: Monday, August 27, 2012 12:30 AM
To: user@hadoop.apache.org
Subject: Re: namenode not starting
Abhay,
On Mon, Aug 27, 2012 at 11:19 AM, Abhay Ratnaparkhi <ab...@gmail.com> wrote:
> Thank you Harsh,
>
> I have set "dfs.name.dir" explicitly. Still don't know why the data
> loss has happened.
>
> <property>
> <name>dfs.name.dir</name>
> <value>/wsadfs/${host.name}/name</value>
> <description>Determines where on the local filesystem the DFS name node
> should store the name table. If this is a comma-delimited list
> of directories then the name table is replicated in all of the
> directories, for redundancy. </description> </property>
Sorry, I missed you had said NFS above. Is the data not present at all in that directory there?
> The secondary namenode was same as namenode. Does this affect anyway
> since path of "dfs.name.dir" were same?
> I have now configured another machine as secondary namenode.
> I have now formatted the filesystem since not seen any way of recovering.
>
> I have some questions.
>
> 1. Apart from setting secondary namenode what are the other techniques
> used for namenode directory backups?
Duplicate dfs.name.dir directories are what we use in production. That is, at least two paths, one local FS and another NFS mounted:
dfs.name.dir = /path/to/local/dfs/name,/path/to/nfs/dfs/name
This will give you two copies of good metadata, and loss of one can still be handled.
> 2. Is there any way or tools to recover some of the data even if
> namenode crashes.
If there's any form of fsimage/edits left, a manual/automated recovery can be made via tools such as oiv/oev and the NN's "-recover" flag, if your version has it, or even with a hexdump and some time.
If there's no trace of fsimage files, its backups from any date, any SNN checkpoints from past, then the metadata is all gone and there's no recovery.
--
Harsh J
Re: namenode not starting
Posted by Harsh J <ha...@cloudera.com>.
Abhay,
On Mon, Aug 27, 2012 at 11:19 AM, Abhay Ratnaparkhi
<ab...@gmail.com> wrote:
> Thank you Harsh,
>
> I have set "dfs.name.dir" explicitly. Still don't know why the data loss has
> happened.
>
> <property>
> <name>dfs.name.dir</name>
> <value>/wsadfs/${host.name}/name</value>
> <description>Determines where on the local filesystem the DFS name node
> should store the name table. If this is a comma-delimited list
> of directories then the name table is replicated in all of the
> directories, for redundancy. </description>
> </property>
Sorry, I missed you had said NFS above. Is the data not present at all
in that directory there?
> The secondary namenode was same as namenode. Does this affect anyway since
> path of "dfs.name.dir" were same?
> I have now configured another machine as secondary namenode.
> I have now formatted the filesystem since not seen any way of recovering.
>
> I have some questions.
>
> 1. Apart from setting secondary namenode what are the other techniques used
> for namenode directory backups?
Duplicate dfs.name.dir directories are what we use in production. That
is, at least two paths, one local FS and another NFS mounted:
dfs.name.dir = /path/to/local/dfs/name,/path/to/nfs/dfs/name
This will give you two copies of good metadata, and loss of one can
still be handled.
> 2. Is there any way or tools to recover some of the data even if namenode
> crashes.
If there's any form of fsimage/edits left, a manual/automated recovery
can be made via tools such as oiv/oev and the NN's "-recover" flag, if
your version has it, or even with a hexdump and some time.
If there's no trace of fsimage files, its backups from any date, any
SNN checkpoints from past, then the metadata is all gone and there's
no recovery.
--
Harsh J
Re: namenode not starting
Posted by Harsh J <ha...@cloudera.com>.
Abhay,
On Mon, Aug 27, 2012 at 11:19 AM, Abhay Ratnaparkhi
<ab...@gmail.com> wrote:
> Thank you Harsh,
>
> I have set "dfs.name.dir" explicitly. Still don't know why the data loss has
> happened.
>
> <property>
> <name>dfs.name.dir</name>
> <value>/wsadfs/${host.name}/name</value>
> <description>Determines where on the local filesystem the DFS name node
> should store the name table. If this is a comma-delimited list
> of directories then the name table is replicated in all of the
> directories, for redundancy. </description>
> </property>
Sorry, I missed you had said NFS above. Is the data not present at all
in that directory there?
> The secondary namenode was same as namenode. Does this affect anyway since
> path of "dfs.name.dir" were same?
> I have now configured another machine as secondary namenode.
> I have now formatted the filesystem since not seen any way of recovering.
>
> I have some questions.
>
> 1. Apart from setting secondary namenode what are the other techniques used
> for namenode directory backups?
Duplicate dfs.name.dir directories are what we use in production. That
is, at least two paths, one local FS and another NFS mounted:
dfs.name.dir = /path/to/local/dfs/name,/path/to/nfs/dfs/name
This will give you two copies of good metadata, and loss of one can
still be handled.
> 2. Is there any way or tools to recover some of the data even if namenode
> crashes.
If there's any form of fsimage/edits left, a manual/automated recovery
can be made via tools such as oiv/oev and the NN's "-recover" flag, if
your version has it, or even with a hexdump and some time.
If there's no trace of fsimage files, its backups from any date, any
SNN checkpoints from past, then the metadata is all gone and there's
no recovery.
--
Harsh J
Re: namenode not starting
Posted by Harsh J <ha...@cloudera.com>.
Abhay,
On Mon, Aug 27, 2012 at 11:19 AM, Abhay Ratnaparkhi
<ab...@gmail.com> wrote:
> Thank you Harsh,
>
> I have set "dfs.name.dir" explicitly. Still don't know why the data loss has
> happened.
>
> <property>
> <name>dfs.name.dir</name>
> <value>/wsadfs/${host.name}/name</value>
> <description>Determines where on the local filesystem the DFS name node
> should store the name table. If this is a comma-delimited list
> of directories then the name table is replicated in all of the
> directories, for redundancy. </description>
> </property>
Sorry, I missed you had said NFS above. Is the data not present at all
in that directory there?
> The secondary namenode was same as namenode. Does this affect anyway since
> path of "dfs.name.dir" were same?
> I have now configured another machine as secondary namenode.
> I have now formatted the filesystem since not seen any way of recovering.
>
> I have some questions.
>
> 1. Apart from setting secondary namenode what are the other techniques used
> for namenode directory backups?
Duplicate dfs.name.dir directories are what we use in production. That
is, at least two paths, one local FS and another NFS mounted:
dfs.name.dir = /path/to/local/dfs/name,/path/to/nfs/dfs/name
This will give you two copies of good metadata, and loss of one can
still be handled.
> 2. Is there any way or tools to recover some of the data even if namenode
> crashes.
If there's any form of fsimage/edits left, a manual/automated recovery
can be made via tools such as oiv/oev and the NN's "-recover" flag, if
your version has it, or even with a hexdump and some time.
If there's no trace of fsimage files, its backups from any date, any
SNN checkpoints from past, then the metadata is all gone and there's
no recovery.
--
Harsh J
Re: namenode not starting
Posted by Mohammad Tariq <do...@gmail.com>.
Hello Abhay,
Along with dfs.name.dir, also include dfs.data.dir in hdfs-site.xml.
On Monday, August 27, 2012, Abhay Ratnaparkhi <ab...@gmail.com>
wrote:
> Thank you Harsh,
> I have set "dfs.name.dir" explicitly. Still don't know why the data loss
has happened.
> <property>
> <name>dfs.name.dir</name>
> <value>/wsadfs/${host.name}/name</value>
> <description>Determines where on the local filesystem the DFS name node
> should store the name table. If this is a comma-delimited list
> of directories then the name table is replicated in all of the
> directories, for redundancy. </description>
> </property>
> The secondary namenode was same as namenode. Does this affect anyway
since path of "dfs.name.dir" were same?
> I have now configured another machine as secondary namenode.
> I have now formatted the filesystem since not seen any way of
recovering.
> I have some questions.
> 1. Apart from setting secondary namenode what are the other techniques
used for namenode directory backups?
> 2. Is there any way or tools to recover some of the data even if namenode
crashes.
> Regards,
> Abhay
>
>
>
>
> On Sat, Aug 25, 2012 at 7:45 PM, Harsh J <ha...@cloudera.com> wrote:
>
> Abhay,
>
> I suspect that if you haven't set your dfs.name.dir explicitly, then
> you haven't set fs.checkpoint.dir either, and since both use
> hadoop.tmp.dir paths, you may have lost your data completely and there
> is no recovery possible now.
>
> On Fri, Aug 24, 2012 at 1:10 PM, Abhay Ratnaparkhi
> <ab...@gmail.com> wrote:
>> Hello,
>>
>> I was using cluster for long time and not formatted the namenode.
>> I ran bin/stop-all.sh and bin/start-all.sh scripts only.
>>
>> I am using NFS for dfs.name.dir.
>> hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any way
to
>> recover the data?
>>
>> Thanks,
>> Abhay
>>
>>
>> On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
>>>
>>> Hi Abhay
>>>
>>> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to
>>> /tmp the contents would be deleted on a OS restart. You need to change
this
>>> location before you start your NN.
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ________________________________
>>> From: Abhay Ratnaparkhi <ab...@gmail.com>
>>> Date: Fri, 24 Aug 2012 12:58:41 +0530
>>> To: <us...@hadoop.apache.org>
>>> ReplyTo: user@hadoop.apache.org
>>> Subject: namenode not starting
>>>
>>> Hello,
>>>
>>> I had a running hadoop cluster.
>>> I restarted it and after that namenode is unable to start. I am getting
>>> error saying that it's not formatted. :(
>>> Is it possible to recover the data on HDFS?
>>>
>>> 2012-08-24 03:17:55,378 ERROR
>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>>> initialization failed.
>>> java.io.IOException: NameNode is not formatted.
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>>> at
>>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>>> 2012-08-24 03:17:55,380 ERROR
>>> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
>>> NameNode is not formatted.
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>>> at
>>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(Nam
--
Regards,
Mohammad Tariq
Re: namenode not starting
Posted by Harsh J <ha...@cloudera.com>.
Abhay,
On Mon, Aug 27, 2012 at 11:19 AM, Abhay Ratnaparkhi
<ab...@gmail.com> wrote:
> Thank you Harsh,
>
> I have set "dfs.name.dir" explicitly. Still don't know why the data loss has
> happened.
>
> <property>
> <name>dfs.name.dir</name>
> <value>/wsadfs/${host.name}/name</value>
> <description>Determines where on the local filesystem the DFS name node
> should store the name table. If this is a comma-delimited list
> of directories then the name table is replicated in all of the
> directories, for redundancy. </description>
> </property>
Sorry, I missed you had said NFS above. Is the data not present at all
in that directory there?
> The secondary namenode was same as namenode. Does this affect anyway since
> path of "dfs.name.dir" were same?
> I have now configured another machine as secondary namenode.
> I have now formatted the filesystem since not seen any way of recovering.
>
> I have some questions.
>
> 1. Apart from setting secondary namenode what are the other techniques used
> for namenode directory backups?
Duplicate dfs.name.dir directories are what we use in production. That
is, at least two paths, one local FS and another NFS mounted:
dfs.name.dir = /path/to/local/dfs/name,/path/to/nfs/dfs/name
This will give you two copies of good metadata, and loss of one can
still be handled.
> 2. Is there any way or tools to recover some of the data even if namenode
> crashes.
If there's any form of fsimage/edits left, a manual/automated recovery
can be made via tools such as oiv/oev and the NN's "-recover" flag, if
your version has it, or even with a hexdump and some time.
If there's no trace of fsimage files, its backups from any date, any
SNN checkpoints from past, then the metadata is all gone and there's
no recovery.
--
Harsh J
Re: namenode not starting
Posted by Mohammad Tariq <do...@gmail.com>.
Hello Abhay,
Along with dfs.name.dir, also include dfs.data.dir in hdfs-site.xml.
On Monday, August 27, 2012, Abhay Ratnaparkhi <ab...@gmail.com>
wrote:
> Thank you Harsh,
> I have set "dfs.name.dir" explicitly. Still don't know why the data loss
has happened.
> <property>
> <name>dfs.name.dir</name>
> <value>/wsadfs/${host.name}/name</value>
> <description>Determines where on the local filesystem the DFS name node
> should store the name table. If this is a comma-delimited list
> of directories then the name table is replicated in all of the
> directories, for redundancy. </description>
> </property>
> The secondary namenode was same as namenode. Does this affect anyway
since path of "dfs.name.dir" were same?
> I have now configured another machine as secondary namenode.
> I have now formatted the filesystem since not seen any way of
recovering.
> I have some questions.
> 1. Apart from setting secondary namenode what are the other techniques
used for namenode directory backups?
> 2. Is there any way or tools to recover some of the data even if namenode
crashes.
> Regards,
> Abhay
>
>
>
>
> On Sat, Aug 25, 2012 at 7:45 PM, Harsh J <ha...@cloudera.com> wrote:
>
> Abhay,
>
> I suspect that if you haven't set your dfs.name.dir explicitly, then
> you haven't set fs.checkpoint.dir either, and since both use
> hadoop.tmp.dir paths, you may have lost your data completely and there
> is no recovery possible now.
>
> On Fri, Aug 24, 2012 at 1:10 PM, Abhay Ratnaparkhi
> <ab...@gmail.com> wrote:
>> Hello,
>>
>> I was using cluster for long time and not formatted the namenode.
>> I ran bin/stop-all.sh and bin/start-all.sh scripts only.
>>
>> I am using NFS for dfs.name.dir.
>> hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any way
to
>> recover the data?
>>
>> Thanks,
>> Abhay
>>
>>
>> On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
>>>
>>> Hi Abhay
>>>
>>> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to
>>> /tmp the contents would be deleted on a OS restart. You need to change
this
>>> location before you start your NN.
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ________________________________
>>> From: Abhay Ratnaparkhi <ab...@gmail.com>
>>> Date: Fri, 24 Aug 2012 12:58:41 +0530
>>> To: <us...@hadoop.apache.org>
>>> ReplyTo: user@hadoop.apache.org
>>> Subject: namenode not starting
>>>
>>> Hello,
>>>
>>> I had a running hadoop cluster.
>>> I restarted it and after that namenode is unable to start. I am getting
>>> error saying that it's not formatted. :(
>>> Is it possible to recover the data on HDFS?
>>>
>>> 2012-08-24 03:17:55,378 ERROR
>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>>> initialization failed.
>>> java.io.IOException: NameNode is not formatted.
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>>> at
>>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>>> 2012-08-24 03:17:55,380 ERROR
>>> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
>>> NameNode is not formatted.
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>>> at
>>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(Nam
--
Regards,
Mohammad Tariq
Re: namenode not starting
Posted by Mohammad Tariq <do...@gmail.com>.
Hello Abhay,
Along with dfs.name.dir, also include dfs.data.dir in hdfs-site.xml.
On Monday, August 27, 2012, Abhay Ratnaparkhi <ab...@gmail.com>
wrote:
> Thank you Harsh,
> I have set "dfs.name.dir" explicitly. Still don't know why the data loss
has happened.
> <property>
> <name>dfs.name.dir</name>
> <value>/wsadfs/${host.name}/name</value>
> <description>Determines where on the local filesystem the DFS name node
> should store the name table. If this is a comma-delimited list
> of directories then the name table is replicated in all of the
> directories, for redundancy. </description>
> </property>
> The secondary namenode was same as namenode. Does this affect anyway
since path of "dfs.name.dir" were same?
> I have now configured another machine as secondary namenode.
> I have now formatted the filesystem since not seen any way of
recovering.
> I have some questions.
> 1. Apart from setting secondary namenode what are the other techniques
used for namenode directory backups?
> 2. Is there any way or tools to recover some of the data even if namenode
crashes.
> Regards,
> Abhay
>
>
>
>
> On Sat, Aug 25, 2012 at 7:45 PM, Harsh J <ha...@cloudera.com> wrote:
>
> Abhay,
>
> I suspect that if you haven't set your dfs.name.dir explicitly, then
> you haven't set fs.checkpoint.dir either, and since both use
> hadoop.tmp.dir paths, you may have lost your data completely and there
> is no recovery possible now.
>
> On Fri, Aug 24, 2012 at 1:10 PM, Abhay Ratnaparkhi
> <ab...@gmail.com> wrote:
>> Hello,
>>
>> I was using cluster for long time and not formatted the namenode.
>> I ran bin/stop-all.sh and bin/start-all.sh scripts only.
>>
>> I am using NFS for dfs.name.dir.
>> hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any way
to
>> recover the data?
>>
>> Thanks,
>> Abhay
>>
>>
>> On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
>>>
>>> Hi Abhay
>>>
>>> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to
>>> /tmp the contents would be deleted on a OS restart. You need to change
this
>>> location before you start your NN.
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ________________________________
>>> From: Abhay Ratnaparkhi <ab...@gmail.com>
>>> Date: Fri, 24 Aug 2012 12:58:41 +0530
>>> To: <us...@hadoop.apache.org>
>>> ReplyTo: user@hadoop.apache.org
>>> Subject: namenode not starting
>>>
>>> Hello,
>>>
>>> I had a running hadoop cluster.
>>> I restarted it and after that namenode is unable to start. I am getting
>>> error saying that it's not formatted. :(
>>> Is it possible to recover the data on HDFS?
>>>
>>> 2012-08-24 03:17:55,378 ERROR
>>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>>> initialization failed.
>>> java.io.IOException: NameNode is not formatted.
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>>> at
>>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>>> 2012-08-24 03:17:55,380 ERROR
>>> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
>>> NameNode is not formatted.
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>>> at
>>>
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>>> at
>>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(Nam
--
Regards,
Mohammad Tariq
Re: namenode not starting
Posted by Abhay Ratnaparkhi <ab...@gmail.com>.
Thank you Harsh,
I have set "dfs.name.dir" explicitly. Still don't know why the data loss
has happened.
<property>
<name>dfs.name.dir</name>
<value>/wsadfs/${host.name}/name</value>
<description>Determines where on the local filesystem the DFS name node
should store the name table. If this is a comma-delimited list
of directories then the name table is replicated in all of the
directories, for redundancy. </description>
</property>
The secondary namenode was same as namenode. Does this affect anyway since
path of "dfs.name.dir" were same?
I have now configured another machine as secondary namenode.
I have now formatted the filesystem since not seen any way of recovering.
I have some questions.
1. Apart from setting secondary namenode what are the other techniques used
for namenode directory backups?
2. Is there any way or tools to recover some of the data even if namenode
crashes.
Regards,
Abhay
On Sat, Aug 25, 2012 at 7:45 PM, Harsh J <ha...@cloudera.com> wrote:
> Abhay,
>
> I suspect that if you haven't set your dfs.name.dir explicitly, then
> you haven't set fs.checkpoint.dir either, and since both use
> hadoop.tmp.dir paths, you may have lost your data completely and there
> is no recovery possible now.
>
> On Fri, Aug 24, 2012 at 1:10 PM, Abhay Ratnaparkhi
> <ab...@gmail.com> wrote:
> > Hello,
> >
> > I was using cluster for long time and not formatted the namenode.
> > I ran bin/stop-all.sh and bin/start-all.sh scripts only.
> >
> > I am using NFS for dfs.name.dir.
> > hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any way
> to
> > recover the data?
> >
> > Thanks,
> > Abhay
> >
> >
> > On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com>
> wrote:
> >>
> >> Hi Abhay
> >>
> >> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to
> >> /tmp the contents would be deleted on a OS restart. You need to change
> this
> >> location before you start your NN.
> >> Regards
> >> Bejoy KS
> >>
> >> Sent from handheld, please excuse typos.
> >> ________________________________
> >> From: Abhay Ratnaparkhi <ab...@gmail.com>
> >> Date: Fri, 24 Aug 2012 12:58:41 +0530
> >> To: <us...@hadoop.apache.org>
> >> ReplyTo: user@hadoop.apache.org
> >> Subject: namenode not starting
> >>
> >> Hello,
> >>
> >> I had a running hadoop cluster.
> >> I restarted it and after that namenode is unable to start. I am getting
> >> error saying that it's not formatted. :(
> >> Is it possible to recover the data on HDFS?
> >>
> >> 2012-08-24 03:17:55,378 ERROR
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> >> initialization failed.
> >> java.io.IOException: NameNode is not formatted.
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
> >> 2012-08-24 03:17:55,380 ERROR
> >> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> >> NameNode is not formatted.
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
> >>
> >> Regards,
> >> Abhay
> >>
> >>
> >
>
>
>
> --
> Harsh J
>
Re: namenode not starting
Posted by Abhay Ratnaparkhi <ab...@gmail.com>.
Thank you Harsh,
I have set "dfs.name.dir" explicitly. Still don't know why the data loss
has happened.
<property>
<name>dfs.name.dir</name>
<value>/wsadfs/${host.name}/name</value>
<description>Determines where on the local filesystem the DFS name node
should store the name table. If this is a comma-delimited list
of directories then the name table is replicated in all of the
directories, for redundancy. </description>
</property>
The secondary namenode was same as namenode. Does this affect anyway since
path of "dfs.name.dir" were same?
I have now configured another machine as secondary namenode.
I have now formatted the filesystem since not seen any way of recovering.
I have some questions.
1. Apart from setting secondary namenode what are the other techniques used
for namenode directory backups?
2. Is there any way or tools to recover some of the data even if namenode
crashes.
Regards,
Abhay
On Sat, Aug 25, 2012 at 7:45 PM, Harsh J <ha...@cloudera.com> wrote:
> Abhay,
>
> I suspect that if you haven't set your dfs.name.dir explicitly, then
> you haven't set fs.checkpoint.dir either, and since both use
> hadoop.tmp.dir paths, you may have lost your data completely and there
> is no recovery possible now.
>
> On Fri, Aug 24, 2012 at 1:10 PM, Abhay Ratnaparkhi
> <ab...@gmail.com> wrote:
> > Hello,
> >
> > I was using cluster for long time and not formatted the namenode.
> > I ran bin/stop-all.sh and bin/start-all.sh scripts only.
> >
> > I am using NFS for dfs.name.dir.
> > hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any way
> to
> > recover the data?
> >
> > Thanks,
> > Abhay
> >
> >
> > On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com>
> wrote:
> >>
> >> Hi Abhay
> >>
> >> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to
> >> /tmp the contents would be deleted on a OS restart. You need to change
> this
> >> location before you start your NN.
> >> Regards
> >> Bejoy KS
> >>
> >> Sent from handheld, please excuse typos.
> >> ________________________________
> >> From: Abhay Ratnaparkhi <ab...@gmail.com>
> >> Date: Fri, 24 Aug 2012 12:58:41 +0530
> >> To: <us...@hadoop.apache.org>
> >> ReplyTo: user@hadoop.apache.org
> >> Subject: namenode not starting
> >>
> >> Hello,
> >>
> >> I had a running hadoop cluster.
> >> I restarted it and after that namenode is unable to start. I am getting
> >> error saying that it's not formatted. :(
> >> Is it possible to recover the data on HDFS?
> >>
> >> 2012-08-24 03:17:55,378 ERROR
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> >> initialization failed.
> >> java.io.IOException: NameNode is not formatted.
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
> >> 2012-08-24 03:17:55,380 ERROR
> >> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> >> NameNode is not formatted.
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
> >>
> >> Regards,
> >> Abhay
> >>
> >>
> >
>
>
>
> --
> Harsh J
>
Re: namenode not starting
Posted by Abhay Ratnaparkhi <ab...@gmail.com>.
Thank you Harsh,
I have set "dfs.name.dir" explicitly. Still don't know why the data loss
has happened.
<property>
<name>dfs.name.dir</name>
<value>/wsadfs/${host.name}/name</value>
<description>Determines where on the local filesystem the DFS name node
should store the name table. If this is a comma-delimited list
of directories then the name table is replicated in all of the
directories, for redundancy. </description>
</property>
The secondary namenode was same as namenode. Does this affect anyway since
path of "dfs.name.dir" were same?
I have now configured another machine as secondary namenode.
I have now formatted the filesystem since not seen any way of recovering.
I have some questions.
1. Apart from setting secondary namenode what are the other techniques used
for namenode directory backups?
2. Is there any way or tools to recover some of the data even if namenode
crashes.
Regards,
Abhay
On Sat, Aug 25, 2012 at 7:45 PM, Harsh J <ha...@cloudera.com> wrote:
> Abhay,
>
> I suspect that if you haven't set your dfs.name.dir explicitly, then
> you haven't set fs.checkpoint.dir either, and since both use
> hadoop.tmp.dir paths, you may have lost your data completely and there
> is no recovery possible now.
>
> On Fri, Aug 24, 2012 at 1:10 PM, Abhay Ratnaparkhi
> <ab...@gmail.com> wrote:
> > Hello,
> >
> > I was using cluster for long time and not formatted the namenode.
> > I ran bin/stop-all.sh and bin/start-all.sh scripts only.
> >
> > I am using NFS for dfs.name.dir.
> > hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any way
> to
> > recover the data?
> >
> > Thanks,
> > Abhay
> >
> >
> > On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com>
> wrote:
> >>
> >> Hi Abhay
> >>
> >> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to
> >> /tmp the contents would be deleted on a OS restart. You need to change
> this
> >> location before you start your NN.
> >> Regards
> >> Bejoy KS
> >>
> >> Sent from handheld, please excuse typos.
> >> ________________________________
> >> From: Abhay Ratnaparkhi <ab...@gmail.com>
> >> Date: Fri, 24 Aug 2012 12:58:41 +0530
> >> To: <us...@hadoop.apache.org>
> >> ReplyTo: user@hadoop.apache.org
> >> Subject: namenode not starting
> >>
> >> Hello,
> >>
> >> I had a running hadoop cluster.
> >> I restarted it and after that namenode is unable to start. I am getting
> >> error saying that it's not formatted. :(
> >> Is it possible to recover the data on HDFS?
> >>
> >> 2012-08-24 03:17:55,378 ERROR
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> >> initialization failed.
> >> java.io.IOException: NameNode is not formatted.
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
> >> 2012-08-24 03:17:55,380 ERROR
> >> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> >> NameNode is not formatted.
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
> >>
> >> Regards,
> >> Abhay
> >>
> >>
> >
>
>
>
> --
> Harsh J
>
Re: namenode not starting
Posted by Abhay Ratnaparkhi <ab...@gmail.com>.
Thank you Harsh,
I have set "dfs.name.dir" explicitly. Still don't know why the data loss
has happened.
<property>
<name>dfs.name.dir</name>
<value>/wsadfs/${host.name}/name</value>
<description>Determines where on the local filesystem the DFS name node
should store the name table. If this is a comma-delimited list
of directories then the name table is replicated in all of the
directories, for redundancy. </description>
</property>
The secondary namenode was same as namenode. Does this affect anyway since
path of "dfs.name.dir" were same?
I have now configured another machine as secondary namenode.
I have now formatted the filesystem since not seen any way of recovering.
I have some questions.
1. Apart from setting secondary namenode what are the other techniques used
for namenode directory backups?
2. Is there any way or tools to recover some of the data even if namenode
crashes.
Regards,
Abhay
On Sat, Aug 25, 2012 at 7:45 PM, Harsh J <ha...@cloudera.com> wrote:
> Abhay,
>
> I suspect that if you haven't set your dfs.name.dir explicitly, then
> you haven't set fs.checkpoint.dir either, and since both use
> hadoop.tmp.dir paths, you may have lost your data completely and there
> is no recovery possible now.
>
> On Fri, Aug 24, 2012 at 1:10 PM, Abhay Ratnaparkhi
> <ab...@gmail.com> wrote:
> > Hello,
> >
> > I was using cluster for long time and not formatted the namenode.
> > I ran bin/stop-all.sh and bin/start-all.sh scripts only.
> >
> > I am using NFS for dfs.name.dir.
> > hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any way
> to
> > recover the data?
> >
> > Thanks,
> > Abhay
> >
> >
> > On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com>
> wrote:
> >>
> >> Hi Abhay
> >>
> >> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to
> >> /tmp the contents would be deleted on a OS restart. You need to change
> this
> >> location before you start your NN.
> >> Regards
> >> Bejoy KS
> >>
> >> Sent from handheld, please excuse typos.
> >> ________________________________
> >> From: Abhay Ratnaparkhi <ab...@gmail.com>
> >> Date: Fri, 24 Aug 2012 12:58:41 +0530
> >> To: <us...@hadoop.apache.org>
> >> ReplyTo: user@hadoop.apache.org
> >> Subject: namenode not starting
> >>
> >> Hello,
> >>
> >> I had a running hadoop cluster.
> >> I restarted it and after that namenode is unable to start. I am getting
> >> error saying that it's not formatted. :(
> >> Is it possible to recover the data on HDFS?
> >>
> >> 2012-08-24 03:17:55,378 ERROR
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> >> initialization failed.
> >> java.io.IOException: NameNode is not formatted.
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
> >> 2012-08-24 03:17:55,380 ERROR
> >> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> >> NameNode is not formatted.
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> >> at
> >>
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
> >>
> >> Regards,
> >> Abhay
> >>
> >>
> >
>
>
>
> --
> Harsh J
>
Re: namenode not starting
Posted by Harsh J <ha...@cloudera.com>.
Abhay,
I suspect that if you haven't set your dfs.name.dir explicitly, then
you haven't set fs.checkpoint.dir either, and since both use
hadoop.tmp.dir paths, you may have lost your data completely and there
is no recovery possible now.
On Fri, Aug 24, 2012 at 1:10 PM, Abhay Ratnaparkhi
<ab...@gmail.com> wrote:
> Hello,
>
> I was using cluster for long time and not formatted the namenode.
> I ran bin/stop-all.sh and bin/start-all.sh scripts only.
>
> I am using NFS for dfs.name.dir.
> hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any way to
> recover the data?
>
> Thanks,
> Abhay
>
>
> On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
>>
>> Hi Abhay
>>
>> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to
>> /tmp the contents would be deleted on a OS restart. You need to change this
>> location before you start your NN.
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ________________________________
>> From: Abhay Ratnaparkhi <ab...@gmail.com>
>> Date: Fri, 24 Aug 2012 12:58:41 +0530
>> To: <us...@hadoop.apache.org>
>> ReplyTo: user@hadoop.apache.org
>> Subject: namenode not starting
>>
>> Hello,
>>
>> I had a running hadoop cluster.
>> I restarted it and after that namenode is unable to start. I am getting
>> error saying that it's not formatted. :(
>> Is it possible to recover the data on HDFS?
>>
>> 2012-08-24 03:17:55,378 ERROR
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>> initialization failed.
>> java.io.IOException: NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>> 2012-08-24 03:17:55,380 ERROR
>> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
>> NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>>
>> Regards,
>> Abhay
>>
>>
>
--
Harsh J
RE: namenode not starting
Posted by Siddharth Tiwari <si...@live.com>.
Hi Abhay,
I totaly conform with Bejoy. Can you paste your mapred-site.xml and hdfs-site.xml content here ?
*------------------------*
Cheers !!!
Siddharth Tiwari
Have a refreshing day !!!
"Every duty is holy, and devotion to duty is the highest form of worship of God.”
"Maybe other people will try to limit me but I don't limit myself"
> From: lleung@ddn.com
> To: user@hadoop.apache.org
> Subject: RE: namenode not starting
> Date: Fri, 24 Aug 2012 16:38:01 +0000
>
> Abhay,
> Sounds like your namenode cannot find the metadata information it needs to start (the <path>/current | image | *checppints etc)
>
> Basically, if you cannot locate that data locally or on your NFS Server, your cluster is busted.
>
> But, let's us be optimistic about this.
>
> There is a chance that your NFS Server is down or the path mounted is lost.
>
> If it is NFS mounted (as you suggested) check that your host still have that path mounted. (from the proper NFS Server)
> ( [shell] mount ) can tell.
> * obviously if you originally mounted from foo:/mydata and now do bar:/mydata / you'll need to do some digging to find which NFS server it was writing to before.
>
> Failing to locate your namenode metadata (locally or on any of your NFS Server) either because the NFS Server decided to become a blackhole, or some<one|thing> removed it.
>
> And you don't have a backup of your namenode (tape or Secondary Namenode),
> I think you are in a world of hurt there.
>
> In theory you can read the blocks on the DN and try to recover some of your data (assume not in CODEC / compressed) .
> Humm.. anyone knows about recovery services? (^^)
>
>
>
> -----Original Message-----
> From: Håvard Wahl Kongsgård [mailto:haavard.kongsgaard@gmail.com]
> Sent: Friday, August 24, 2012 5:38 AM
> To: user@hadoop.apache.org
> Subject: Re: namenode not starting
>
> You should start with a reboot of the system.
>
> A lesson to everyone, this is exactly why you should have a secondary name node (http://wiki.apache.org/hadoop/FAQ#What_is_the_purpose_of_the_secondary_name-node.3F)
> and run the namenode a mirrored RAID-5/10 disk.
>
>
> -Håvard
>
>
>
> On Fri, Aug 24, 2012 at 9:40 AM, Abhay Ratnaparkhi <ab...@gmail.com> wrote:
> > Hello,
> >
> > I was using cluster for long time and not formatted the namenode.
> > I ran bin/stop-all.sh and bin/start-all.sh scripts only.
> >
> > I am using NFS for dfs.name.dir.
> > hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any
> > way to recover the data?
> >
> > Thanks,
> > Abhay
> >
> >
> > On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
> >>
> >> Hi Abhay
> >>
> >> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set
> >> to /tmp the contents would be deleted on a OS restart. You need to
> >> change this location before you start your NN.
> >> Regards
> >> Bejoy KS
> >>
> >> Sent from handheld, please excuse typos.
> >> ________________________________
> >> From: Abhay Ratnaparkhi <ab...@gmail.com>
> >> Date: Fri, 24 Aug 2012 12:58:41 +0530
> >> To: <us...@hadoop.apache.org>
> >> ReplyTo: user@hadoop.apache.org
> >> Subject: namenode not starting
> >>
> >> Hello,
> >>
> >> I had a running hadoop cluster.
> >> I restarted it and after that namenode is unable to start. I am
> >> getting error saying that it's not formatted. :( Is it possible to
> >> recover the data on HDFS?
> >>
> >> 2012-08-24 03:17:55,378 ERROR
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> >> initialization failed.
> >> java.io.IOException: NameNode is not formatted.
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:13
> >> 68)
> >> 2012-08-24 03:17:55,380 ERROR
> >> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> >> NameNode is not formatted.
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:13
> >> 68)
> >>
> >> Regards,
> >> Abhay
> >>
> >>
> >
>
>
>
> --
> Håvard Wahl Kongsgård
> Faculty of Medicine &
> Department of Mathematical Sciences
> NTNU
>
> http://havard.security-review.net/
RE: namenode not starting
Posted by Siddharth Tiwari <si...@live.com>.
Hi Abhay,
I totaly conform with Bejoy. Can you paste your mapred-site.xml and hdfs-site.xml content here ?
*------------------------*
Cheers !!!
Siddharth Tiwari
Have a refreshing day !!!
"Every duty is holy, and devotion to duty is the highest form of worship of God.”
"Maybe other people will try to limit me but I don't limit myself"
> From: lleung@ddn.com
> To: user@hadoop.apache.org
> Subject: RE: namenode not starting
> Date: Fri, 24 Aug 2012 16:38:01 +0000
>
> Abhay,
> Sounds like your namenode cannot find the metadata information it needs to start (the <path>/current | image | *checppints etc)
>
> Basically, if you cannot locate that data locally or on your NFS Server, your cluster is busted.
>
> But, let's us be optimistic about this.
>
> There is a chance that your NFS Server is down or the path mounted is lost.
>
> If it is NFS mounted (as you suggested) check that your host still have that path mounted. (from the proper NFS Server)
> ( [shell] mount ) can tell.
> * obviously if you originally mounted from foo:/mydata and now do bar:/mydata / you'll need to do some digging to find which NFS server it was writing to before.
>
> Failing to locate your namenode metadata (locally or on any of your NFS Server) either because the NFS Server decided to become a blackhole, or some<one|thing> removed it.
>
> And you don't have a backup of your namenode (tape or Secondary Namenode),
> I think you are in a world of hurt there.
>
> In theory you can read the blocks on the DN and try to recover some of your data (assume not in CODEC / compressed) .
> Humm.. anyone knows about recovery services? (^^)
>
>
>
> -----Original Message-----
> From: Håvard Wahl Kongsgård [mailto:haavard.kongsgaard@gmail.com]
> Sent: Friday, August 24, 2012 5:38 AM
> To: user@hadoop.apache.org
> Subject: Re: namenode not starting
>
> You should start with a reboot of the system.
>
> A lesson to everyone, this is exactly why you should have a secondary name node (http://wiki.apache.org/hadoop/FAQ#What_is_the_purpose_of_the_secondary_name-node.3F)
> and run the namenode a mirrored RAID-5/10 disk.
>
>
> -Håvard
>
>
>
> On Fri, Aug 24, 2012 at 9:40 AM, Abhay Ratnaparkhi <ab...@gmail.com> wrote:
> > Hello,
> >
> > I was using cluster for long time and not formatted the namenode.
> > I ran bin/stop-all.sh and bin/start-all.sh scripts only.
> >
> > I am using NFS for dfs.name.dir.
> > hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any
> > way to recover the data?
> >
> > Thanks,
> > Abhay
> >
> >
> > On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
> >>
> >> Hi Abhay
> >>
> >> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set
> >> to /tmp the contents would be deleted on a OS restart. You need to
> >> change this location before you start your NN.
> >> Regards
> >> Bejoy KS
> >>
> >> Sent from handheld, please excuse typos.
> >> ________________________________
> >> From: Abhay Ratnaparkhi <ab...@gmail.com>
> >> Date: Fri, 24 Aug 2012 12:58:41 +0530
> >> To: <us...@hadoop.apache.org>
> >> ReplyTo: user@hadoop.apache.org
> >> Subject: namenode not starting
> >>
> >> Hello,
> >>
> >> I had a running hadoop cluster.
> >> I restarted it and after that namenode is unable to start. I am
> >> getting error saying that it's not formatted. :( Is it possible to
> >> recover the data on HDFS?
> >>
> >> 2012-08-24 03:17:55,378 ERROR
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> >> initialization failed.
> >> java.io.IOException: NameNode is not formatted.
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:13
> >> 68)
> >> 2012-08-24 03:17:55,380 ERROR
> >> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> >> NameNode is not formatted.
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:13
> >> 68)
> >>
> >> Regards,
> >> Abhay
> >>
> >>
> >
>
>
>
> --
> Håvard Wahl Kongsgård
> Faculty of Medicine &
> Department of Mathematical Sciences
> NTNU
>
> http://havard.security-review.net/
RE: namenode not starting
Posted by Siddharth Tiwari <si...@live.com>.
Hi Abhay,
I totaly conform with Bejoy. Can you paste your mapred-site.xml and hdfs-site.xml content here ?
*------------------------*
Cheers !!!
Siddharth Tiwari
Have a refreshing day !!!
"Every duty is holy, and devotion to duty is the highest form of worship of God.”
"Maybe other people will try to limit me but I don't limit myself"
> From: lleung@ddn.com
> To: user@hadoop.apache.org
> Subject: RE: namenode not starting
> Date: Fri, 24 Aug 2012 16:38:01 +0000
>
> Abhay,
> Sounds like your namenode cannot find the metadata information it needs to start (the <path>/current | image | *checppints etc)
>
> Basically, if you cannot locate that data locally or on your NFS Server, your cluster is busted.
>
> But, let's us be optimistic about this.
>
> There is a chance that your NFS Server is down or the path mounted is lost.
>
> If it is NFS mounted (as you suggested) check that your host still have that path mounted. (from the proper NFS Server)
> ( [shell] mount ) can tell.
> * obviously if you originally mounted from foo:/mydata and now do bar:/mydata / you'll need to do some digging to find which NFS server it was writing to before.
>
> Failing to locate your namenode metadata (locally or on any of your NFS Server) either because the NFS Server decided to become a blackhole, or some<one|thing> removed it.
>
> And you don't have a backup of your namenode (tape or Secondary Namenode),
> I think you are in a world of hurt there.
>
> In theory you can read the blocks on the DN and try to recover some of your data (assume not in CODEC / compressed) .
> Humm.. anyone knows about recovery services? (^^)
>
>
>
> -----Original Message-----
> From: Håvard Wahl Kongsgård [mailto:haavard.kongsgaard@gmail.com]
> Sent: Friday, August 24, 2012 5:38 AM
> To: user@hadoop.apache.org
> Subject: Re: namenode not starting
>
> You should start with a reboot of the system.
>
> A lesson to everyone, this is exactly why you should have a secondary name node (http://wiki.apache.org/hadoop/FAQ#What_is_the_purpose_of_the_secondary_name-node.3F)
> and run the namenode a mirrored RAID-5/10 disk.
>
>
> -Håvard
>
>
>
> On Fri, Aug 24, 2012 at 9:40 AM, Abhay Ratnaparkhi <ab...@gmail.com> wrote:
> > Hello,
> >
> > I was using cluster for long time and not formatted the namenode.
> > I ran bin/stop-all.sh and bin/start-all.sh scripts only.
> >
> > I am using NFS for dfs.name.dir.
> > hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any
> > way to recover the data?
> >
> > Thanks,
> > Abhay
> >
> >
> > On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
> >>
> >> Hi Abhay
> >>
> >> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set
> >> to /tmp the contents would be deleted on a OS restart. You need to
> >> change this location before you start your NN.
> >> Regards
> >> Bejoy KS
> >>
> >> Sent from handheld, please excuse typos.
> >> ________________________________
> >> From: Abhay Ratnaparkhi <ab...@gmail.com>
> >> Date: Fri, 24 Aug 2012 12:58:41 +0530
> >> To: <us...@hadoop.apache.org>
> >> ReplyTo: user@hadoop.apache.org
> >> Subject: namenode not starting
> >>
> >> Hello,
> >>
> >> I had a running hadoop cluster.
> >> I restarted it and after that namenode is unable to start. I am
> >> getting error saying that it's not formatted. :( Is it possible to
> >> recover the data on HDFS?
> >>
> >> 2012-08-24 03:17:55,378 ERROR
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> >> initialization failed.
> >> java.io.IOException: NameNode is not formatted.
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:13
> >> 68)
> >> 2012-08-24 03:17:55,380 ERROR
> >> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> >> NameNode is not formatted.
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:13
> >> 68)
> >>
> >> Regards,
> >> Abhay
> >>
> >>
> >
>
>
>
> --
> Håvard Wahl Kongsgård
> Faculty of Medicine &
> Department of Mathematical Sciences
> NTNU
>
> http://havard.security-review.net/
RE: namenode not starting
Posted by Siddharth Tiwari <si...@live.com>.
Hi Abhay,
I totaly conform with Bejoy. Can you paste your mapred-site.xml and hdfs-site.xml content here ?
*------------------------*
Cheers !!!
Siddharth Tiwari
Have a refreshing day !!!
"Every duty is holy, and devotion to duty is the highest form of worship of God.”
"Maybe other people will try to limit me but I don't limit myself"
> From: lleung@ddn.com
> To: user@hadoop.apache.org
> Subject: RE: namenode not starting
> Date: Fri, 24 Aug 2012 16:38:01 +0000
>
> Abhay,
> Sounds like your namenode cannot find the metadata information it needs to start (the <path>/current | image | *checppints etc)
>
> Basically, if you cannot locate that data locally or on your NFS Server, your cluster is busted.
>
> But, let's us be optimistic about this.
>
> There is a chance that your NFS Server is down or the path mounted is lost.
>
> If it is NFS mounted (as you suggested) check that your host still have that path mounted. (from the proper NFS Server)
> ( [shell] mount ) can tell.
> * obviously if you originally mounted from foo:/mydata and now do bar:/mydata / you'll need to do some digging to find which NFS server it was writing to before.
>
> Failing to locate your namenode metadata (locally or on any of your NFS Server) either because the NFS Server decided to become a blackhole, or some<one|thing> removed it.
>
> And you don't have a backup of your namenode (tape or Secondary Namenode),
> I think you are in a world of hurt there.
>
> In theory you can read the blocks on the DN and try to recover some of your data (assume not in CODEC / compressed) .
> Humm.. anyone knows about recovery services? (^^)
>
>
>
> -----Original Message-----
> From: Håvard Wahl Kongsgård [mailto:haavard.kongsgaard@gmail.com]
> Sent: Friday, August 24, 2012 5:38 AM
> To: user@hadoop.apache.org
> Subject: Re: namenode not starting
>
> You should start with a reboot of the system.
>
> A lesson to everyone, this is exactly why you should have a secondary name node (http://wiki.apache.org/hadoop/FAQ#What_is_the_purpose_of_the_secondary_name-node.3F)
> and run the namenode a mirrored RAID-5/10 disk.
>
>
> -Håvard
>
>
>
> On Fri, Aug 24, 2012 at 9:40 AM, Abhay Ratnaparkhi <ab...@gmail.com> wrote:
> > Hello,
> >
> > I was using cluster for long time and not formatted the namenode.
> > I ran bin/stop-all.sh and bin/start-all.sh scripts only.
> >
> > I am using NFS for dfs.name.dir.
> > hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any
> > way to recover the data?
> >
> > Thanks,
> > Abhay
> >
> >
> > On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
> >>
> >> Hi Abhay
> >>
> >> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set
> >> to /tmp the contents would be deleted on a OS restart. You need to
> >> change this location before you start your NN.
> >> Regards
> >> Bejoy KS
> >>
> >> Sent from handheld, please excuse typos.
> >> ________________________________
> >> From: Abhay Ratnaparkhi <ab...@gmail.com>
> >> Date: Fri, 24 Aug 2012 12:58:41 +0530
> >> To: <us...@hadoop.apache.org>
> >> ReplyTo: user@hadoop.apache.org
> >> Subject: namenode not starting
> >>
> >> Hello,
> >>
> >> I had a running hadoop cluster.
> >> I restarted it and after that namenode is unable to start. I am
> >> getting error saying that it's not formatted. :( Is it possible to
> >> recover the data on HDFS?
> >>
> >> 2012-08-24 03:17:55,378 ERROR
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> >> initialization failed.
> >> java.io.IOException: NameNode is not formatted.
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:13
> >> 68)
> >> 2012-08-24 03:17:55,380 ERROR
> >> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> >> NameNode is not formatted.
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:13
> >> 68)
> >>
> >> Regards,
> >> Abhay
> >>
> >>
> >
>
>
>
> --
> Håvard Wahl Kongsgård
> Faculty of Medicine &
> Department of Mathematical Sciences
> NTNU
>
> http://havard.security-review.net/
RE: namenode not starting
Posted by Leo Leung <ll...@ddn.com>.
Abhay,
Sounds like your namenode cannot find the metadata information it needs to start (the <path>/current | image | *checppints etc)
Basically, if you cannot locate that data locally or on your NFS Server, your cluster is busted.
But, let's us be optimistic about this.
There is a chance that your NFS Server is down or the path mounted is lost.
If it is NFS mounted (as you suggested) check that your host still have that path mounted. (from the proper NFS Server)
( [shell] mount ) can tell.
* obviously if you originally mounted from foo:/mydata and now do bar:/mydata / you'll need to do some digging to find which NFS server it was writing to before.
Failing to locate your namenode metadata (locally or on any of your NFS Server) either because the NFS Server decided to become a blackhole, or some<one|thing> removed it.
And you don't have a backup of your namenode (tape or Secondary Namenode),
I think you are in a world of hurt there.
In theory you can read the blocks on the DN and try to recover some of your data (assume not in CODEC / compressed) .
Humm.. anyone knows about recovery services? (^^)
-----Original Message-----
From: Håvard Wahl Kongsgård [mailto:haavard.kongsgaard@gmail.com]
Sent: Friday, August 24, 2012 5:38 AM
To: user@hadoop.apache.org
Subject: Re: namenode not starting
You should start with a reboot of the system.
A lesson to everyone, this is exactly why you should have a secondary name node (http://wiki.apache.org/hadoop/FAQ#What_is_the_purpose_of_the_secondary_name-node.3F)
and run the namenode a mirrored RAID-5/10 disk.
-Håvard
On Fri, Aug 24, 2012 at 9:40 AM, Abhay Ratnaparkhi <ab...@gmail.com> wrote:
> Hello,
>
> I was using cluster for long time and not formatted the namenode.
> I ran bin/stop-all.sh and bin/start-all.sh scripts only.
>
> I am using NFS for dfs.name.dir.
> hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any
> way to recover the data?
>
> Thanks,
> Abhay
>
>
> On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
>>
>> Hi Abhay
>>
>> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set
>> to /tmp the contents would be deleted on a OS restart. You need to
>> change this location before you start your NN.
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ________________________________
>> From: Abhay Ratnaparkhi <ab...@gmail.com>
>> Date: Fri, 24 Aug 2012 12:58:41 +0530
>> To: <us...@hadoop.apache.org>
>> ReplyTo: user@hadoop.apache.org
>> Subject: namenode not starting
>>
>> Hello,
>>
>> I had a running hadoop cluster.
>> I restarted it and after that namenode is unable to start. I am
>> getting error saying that it's not formatted. :( Is it possible to
>> recover the data on HDFS?
>>
>> 2012-08-24 03:17:55,378 ERROR
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>> initialization failed.
>> java.io.IOException: NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:13
>> 68)
>> 2012-08-24 03:17:55,380 ERROR
>> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
>> NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:13
>> 68)
>>
>> Regards,
>> Abhay
>>
>>
>
--
Håvard Wahl Kongsgård
Faculty of Medicine &
Department of Mathematical Sciences
NTNU
http://havard.security-review.net/
RE: namenode not starting
Posted by Leo Leung <ll...@ddn.com>.
Abhay,
Sounds like your namenode cannot find the metadata information it needs to start (the <path>/current | image | *checppints etc)
Basically, if you cannot locate that data locally or on your NFS Server, your cluster is busted.
But, let's us be optimistic about this.
There is a chance that your NFS Server is down or the path mounted is lost.
If it is NFS mounted (as you suggested) check that your host still have that path mounted. (from the proper NFS Server)
( [shell] mount ) can tell.
* obviously if you originally mounted from foo:/mydata and now do bar:/mydata / you'll need to do some digging to find which NFS server it was writing to before.
Failing to locate your namenode metadata (locally or on any of your NFS Server) either because the NFS Server decided to become a blackhole, or some<one|thing> removed it.
And you don't have a backup of your namenode (tape or Secondary Namenode),
I think you are in a world of hurt there.
In theory you can read the blocks on the DN and try to recover some of your data (assume not in CODEC / compressed) .
Humm.. anyone knows about recovery services? (^^)
-----Original Message-----
From: Håvard Wahl Kongsgård [mailto:haavard.kongsgaard@gmail.com]
Sent: Friday, August 24, 2012 5:38 AM
To: user@hadoop.apache.org
Subject: Re: namenode not starting
You should start with a reboot of the system.
A lesson to everyone, this is exactly why you should have a secondary name node (http://wiki.apache.org/hadoop/FAQ#What_is_the_purpose_of_the_secondary_name-node.3F)
and run the namenode a mirrored RAID-5/10 disk.
-Håvard
On Fri, Aug 24, 2012 at 9:40 AM, Abhay Ratnaparkhi <ab...@gmail.com> wrote:
> Hello,
>
> I was using cluster for long time and not formatted the namenode.
> I ran bin/stop-all.sh and bin/start-all.sh scripts only.
>
> I am using NFS for dfs.name.dir.
> hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any
> way to recover the data?
>
> Thanks,
> Abhay
>
>
> On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
>>
>> Hi Abhay
>>
>> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set
>> to /tmp the contents would be deleted on a OS restart. You need to
>> change this location before you start your NN.
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ________________________________
>> From: Abhay Ratnaparkhi <ab...@gmail.com>
>> Date: Fri, 24 Aug 2012 12:58:41 +0530
>> To: <us...@hadoop.apache.org>
>> ReplyTo: user@hadoop.apache.org
>> Subject: namenode not starting
>>
>> Hello,
>>
>> I had a running hadoop cluster.
>> I restarted it and after that namenode is unable to start. I am
>> getting error saying that it's not formatted. :( Is it possible to
>> recover the data on HDFS?
>>
>> 2012-08-24 03:17:55,378 ERROR
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>> initialization failed.
>> java.io.IOException: NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:13
>> 68)
>> 2012-08-24 03:17:55,380 ERROR
>> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
>> NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:13
>> 68)
>>
>> Regards,
>> Abhay
>>
>>
>
--
Håvard Wahl Kongsgård
Faculty of Medicine &
Department of Mathematical Sciences
NTNU
http://havard.security-review.net/
RE: namenode not starting
Posted by Leo Leung <ll...@ddn.com>.
Abhay,
Sounds like your namenode cannot find the metadata information it needs to start (the <path>/current | image | *checppints etc)
Basically, if you cannot locate that data locally or on your NFS Server, your cluster is busted.
But, let's us be optimistic about this.
There is a chance that your NFS Server is down or the path mounted is lost.
If it is NFS mounted (as you suggested) check that your host still have that path mounted. (from the proper NFS Server)
( [shell] mount ) can tell.
* obviously if you originally mounted from foo:/mydata and now do bar:/mydata / you'll need to do some digging to find which NFS server it was writing to before.
Failing to locate your namenode metadata (locally or on any of your NFS Server) either because the NFS Server decided to become a blackhole, or some<one|thing> removed it.
And you don't have a backup of your namenode (tape or Secondary Namenode),
I think you are in a world of hurt there.
In theory you can read the blocks on the DN and try to recover some of your data (assume not in CODEC / compressed) .
Humm.. anyone knows about recovery services? (^^)
-----Original Message-----
From: Håvard Wahl Kongsgård [mailto:haavard.kongsgaard@gmail.com]
Sent: Friday, August 24, 2012 5:38 AM
To: user@hadoop.apache.org
Subject: Re: namenode not starting
You should start with a reboot of the system.
A lesson to everyone, this is exactly why you should have a secondary name node (http://wiki.apache.org/hadoop/FAQ#What_is_the_purpose_of_the_secondary_name-node.3F)
and run the namenode a mirrored RAID-5/10 disk.
-Håvard
On Fri, Aug 24, 2012 at 9:40 AM, Abhay Ratnaparkhi <ab...@gmail.com> wrote:
> Hello,
>
> I was using cluster for long time and not formatted the namenode.
> I ran bin/stop-all.sh and bin/start-all.sh scripts only.
>
> I am using NFS for dfs.name.dir.
> hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any
> way to recover the data?
>
> Thanks,
> Abhay
>
>
> On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
>>
>> Hi Abhay
>>
>> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set
>> to /tmp the contents would be deleted on a OS restart. You need to
>> change this location before you start your NN.
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ________________________________
>> From: Abhay Ratnaparkhi <ab...@gmail.com>
>> Date: Fri, 24 Aug 2012 12:58:41 +0530
>> To: <us...@hadoop.apache.org>
>> ReplyTo: user@hadoop.apache.org
>> Subject: namenode not starting
>>
>> Hello,
>>
>> I had a running hadoop cluster.
>> I restarted it and after that namenode is unable to start. I am
>> getting error saying that it's not formatted. :( Is it possible to
>> recover the data on HDFS?
>>
>> 2012-08-24 03:17:55,378 ERROR
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>> initialization failed.
>> java.io.IOException: NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:13
>> 68)
>> 2012-08-24 03:17:55,380 ERROR
>> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
>> NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:13
>> 68)
>>
>> Regards,
>> Abhay
>>
>>
>
--
Håvard Wahl Kongsgård
Faculty of Medicine &
Department of Mathematical Sciences
NTNU
http://havard.security-review.net/
RE: namenode not starting
Posted by Leo Leung <ll...@ddn.com>.
Abhay,
Sounds like your namenode cannot find the metadata information it needs to start (the <path>/current | image | *checppints etc)
Basically, if you cannot locate that data locally or on your NFS Server, your cluster is busted.
But, let's us be optimistic about this.
There is a chance that your NFS Server is down or the path mounted is lost.
If it is NFS mounted (as you suggested) check that your host still have that path mounted. (from the proper NFS Server)
( [shell] mount ) can tell.
* obviously if you originally mounted from foo:/mydata and now do bar:/mydata / you'll need to do some digging to find which NFS server it was writing to before.
Failing to locate your namenode metadata (locally or on any of your NFS Server) either because the NFS Server decided to become a blackhole, or some<one|thing> removed it.
And you don't have a backup of your namenode (tape or Secondary Namenode),
I think you are in a world of hurt there.
In theory you can read the blocks on the DN and try to recover some of your data (assume not in CODEC / compressed) .
Humm.. anyone knows about recovery services? (^^)
-----Original Message-----
From: Håvard Wahl Kongsgård [mailto:haavard.kongsgaard@gmail.com]
Sent: Friday, August 24, 2012 5:38 AM
To: user@hadoop.apache.org
Subject: Re: namenode not starting
You should start with a reboot of the system.
A lesson to everyone, this is exactly why you should have a secondary name node (http://wiki.apache.org/hadoop/FAQ#What_is_the_purpose_of_the_secondary_name-node.3F)
and run the namenode a mirrored RAID-5/10 disk.
-Håvard
On Fri, Aug 24, 2012 at 9:40 AM, Abhay Ratnaparkhi <ab...@gmail.com> wrote:
> Hello,
>
> I was using cluster for long time and not formatted the namenode.
> I ran bin/stop-all.sh and bin/start-all.sh scripts only.
>
> I am using NFS for dfs.name.dir.
> hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any
> way to recover the data?
>
> Thanks,
> Abhay
>
>
> On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
>>
>> Hi Abhay
>>
>> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set
>> to /tmp the contents would be deleted on a OS restart. You need to
>> change this location before you start your NN.
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ________________________________
>> From: Abhay Ratnaparkhi <ab...@gmail.com>
>> Date: Fri, 24 Aug 2012 12:58:41 +0530
>> To: <us...@hadoop.apache.org>
>> ReplyTo: user@hadoop.apache.org
>> Subject: namenode not starting
>>
>> Hello,
>>
>> I had a running hadoop cluster.
>> I restarted it and after that namenode is unable to start. I am
>> getting error saying that it's not formatted. :( Is it possible to
>> recover the data on HDFS?
>>
>> 2012-08-24 03:17:55,378 ERROR
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>> initialization failed.
>> java.io.IOException: NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:13
>> 68)
>> 2012-08-24 03:17:55,380 ERROR
>> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
>> NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:13
>> 68)
>>
>> Regards,
>> Abhay
>>
>>
>
--
Håvard Wahl Kongsgård
Faculty of Medicine &
Department of Mathematical Sciences
NTNU
http://havard.security-review.net/
Re: namenode not starting
Posted by Håvard Wahl Kongsgård <ha...@gmail.com>.
You should start with a reboot of the system.
A lesson to everyone, this is exactly why you should have a secondary
name node (http://wiki.apache.org/hadoop/FAQ#What_is_the_purpose_of_the_secondary_name-node.3F)
and run the namenode a mirrored RAID-5/10 disk.
-Håvard
On Fri, Aug 24, 2012 at 9:40 AM, Abhay Ratnaparkhi
<ab...@gmail.com> wrote:
> Hello,
>
> I was using cluster for long time and not formatted the namenode.
> I ran bin/stop-all.sh and bin/start-all.sh scripts only.
>
> I am using NFS for dfs.name.dir.
> hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any way to
> recover the data?
>
> Thanks,
> Abhay
>
>
> On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
>>
>> Hi Abhay
>>
>> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to
>> /tmp the contents would be deleted on a OS restart. You need to change this
>> location before you start your NN.
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ________________________________
>> From: Abhay Ratnaparkhi <ab...@gmail.com>
>> Date: Fri, 24 Aug 2012 12:58:41 +0530
>> To: <us...@hadoop.apache.org>
>> ReplyTo: user@hadoop.apache.org
>> Subject: namenode not starting
>>
>> Hello,
>>
>> I had a running hadoop cluster.
>> I restarted it and after that namenode is unable to start. I am getting
>> error saying that it's not formatted. :(
>> Is it possible to recover the data on HDFS?
>>
>> 2012-08-24 03:17:55,378 ERROR
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>> initialization failed.
>> java.io.IOException: NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>> 2012-08-24 03:17:55,380 ERROR
>> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
>> NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>>
>> Regards,
>> Abhay
>>
>>
>
--
Håvard Wahl Kongsgård
Faculty of Medicine &
Department of Mathematical Sciences
NTNU
http://havard.security-review.net/
Re: namenode not starting
Posted by Håvard Wahl Kongsgård <ha...@gmail.com>.
You should start with a reboot of the system.
A lesson to everyone, this is exactly why you should have a secondary
name node (http://wiki.apache.org/hadoop/FAQ#What_is_the_purpose_of_the_secondary_name-node.3F)
and run the namenode a mirrored RAID-5/10 disk.
-Håvard
On Fri, Aug 24, 2012 at 9:40 AM, Abhay Ratnaparkhi
<ab...@gmail.com> wrote:
> Hello,
>
> I was using cluster for long time and not formatted the namenode.
> I ran bin/stop-all.sh and bin/start-all.sh scripts only.
>
> I am using NFS for dfs.name.dir.
> hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any way to
> recover the data?
>
> Thanks,
> Abhay
>
>
> On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
>>
>> Hi Abhay
>>
>> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to
>> /tmp the contents would be deleted on a OS restart. You need to change this
>> location before you start your NN.
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ________________________________
>> From: Abhay Ratnaparkhi <ab...@gmail.com>
>> Date: Fri, 24 Aug 2012 12:58:41 +0530
>> To: <us...@hadoop.apache.org>
>> ReplyTo: user@hadoop.apache.org
>> Subject: namenode not starting
>>
>> Hello,
>>
>> I had a running hadoop cluster.
>> I restarted it and after that namenode is unable to start. I am getting
>> error saying that it's not formatted. :(
>> Is it possible to recover the data on HDFS?
>>
>> 2012-08-24 03:17:55,378 ERROR
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>> initialization failed.
>> java.io.IOException: NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>> 2012-08-24 03:17:55,380 ERROR
>> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
>> NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>>
>> Regards,
>> Abhay
>>
>>
>
--
Håvard Wahl Kongsgård
Faculty of Medicine &
Department of Mathematical Sciences
NTNU
http://havard.security-review.net/
Re: namenode not starting
Posted by Harsh J <ha...@cloudera.com>.
Abhay,
I suspect that if you haven't set your dfs.name.dir explicitly, then
you haven't set fs.checkpoint.dir either, and since both use
hadoop.tmp.dir paths, you may have lost your data completely and there
is no recovery possible now.
On Fri, Aug 24, 2012 at 1:10 PM, Abhay Ratnaparkhi
<ab...@gmail.com> wrote:
> Hello,
>
> I was using cluster for long time and not formatted the namenode.
> I ran bin/stop-all.sh and bin/start-all.sh scripts only.
>
> I am using NFS for dfs.name.dir.
> hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any way to
> recover the data?
>
> Thanks,
> Abhay
>
>
> On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
>>
>> Hi Abhay
>>
>> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to
>> /tmp the contents would be deleted on a OS restart. You need to change this
>> location before you start your NN.
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ________________________________
>> From: Abhay Ratnaparkhi <ab...@gmail.com>
>> Date: Fri, 24 Aug 2012 12:58:41 +0530
>> To: <us...@hadoop.apache.org>
>> ReplyTo: user@hadoop.apache.org
>> Subject: namenode not starting
>>
>> Hello,
>>
>> I had a running hadoop cluster.
>> I restarted it and after that namenode is unable to start. I am getting
>> error saying that it's not formatted. :(
>> Is it possible to recover the data on HDFS?
>>
>> 2012-08-24 03:17:55,378 ERROR
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>> initialization failed.
>> java.io.IOException: NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>> 2012-08-24 03:17:55,380 ERROR
>> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
>> NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>>
>> Regards,
>> Abhay
>>
>>
>
--
Harsh J
Re: namenode not starting
Posted by Harsh J <ha...@cloudera.com>.
Abhay,
I suspect that if you haven't set your dfs.name.dir explicitly, then
you haven't set fs.checkpoint.dir either, and since both use
hadoop.tmp.dir paths, you may have lost your data completely and there
is no recovery possible now.
On Fri, Aug 24, 2012 at 1:10 PM, Abhay Ratnaparkhi
<ab...@gmail.com> wrote:
> Hello,
>
> I was using cluster for long time and not formatted the namenode.
> I ran bin/stop-all.sh and bin/start-all.sh scripts only.
>
> I am using NFS for dfs.name.dir.
> hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any way to
> recover the data?
>
> Thanks,
> Abhay
>
>
> On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
>>
>> Hi Abhay
>>
>> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to
>> /tmp the contents would be deleted on a OS restart. You need to change this
>> location before you start your NN.
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ________________________________
>> From: Abhay Ratnaparkhi <ab...@gmail.com>
>> Date: Fri, 24 Aug 2012 12:58:41 +0530
>> To: <us...@hadoop.apache.org>
>> ReplyTo: user@hadoop.apache.org
>> Subject: namenode not starting
>>
>> Hello,
>>
>> I had a running hadoop cluster.
>> I restarted it and after that namenode is unable to start. I am getting
>> error saying that it's not formatted. :(
>> Is it possible to recover the data on HDFS?
>>
>> 2012-08-24 03:17:55,378 ERROR
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>> initialization failed.
>> java.io.IOException: NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>> 2012-08-24 03:17:55,380 ERROR
>> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
>> NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>>
>> Regards,
>> Abhay
>>
>>
>
--
Harsh J
Re: namenode not starting
Posted by Håvard Wahl Kongsgård <ha...@gmail.com>.
You should start with a reboot of the system.
A lesson to everyone, this is exactly why you should have a secondary
name node (http://wiki.apache.org/hadoop/FAQ#What_is_the_purpose_of_the_secondary_name-node.3F)
and run the namenode a mirrored RAID-5/10 disk.
-Håvard
On Fri, Aug 24, 2012 at 9:40 AM, Abhay Ratnaparkhi
<ab...@gmail.com> wrote:
> Hello,
>
> I was using cluster for long time and not formatted the namenode.
> I ran bin/stop-all.sh and bin/start-all.sh scripts only.
>
> I am using NFS for dfs.name.dir.
> hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any way to
> recover the data?
>
> Thanks,
> Abhay
>
>
> On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
>>
>> Hi Abhay
>>
>> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to
>> /tmp the contents would be deleted on a OS restart. You need to change this
>> location before you start your NN.
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ________________________________
>> From: Abhay Ratnaparkhi <ab...@gmail.com>
>> Date: Fri, 24 Aug 2012 12:58:41 +0530
>> To: <us...@hadoop.apache.org>
>> ReplyTo: user@hadoop.apache.org
>> Subject: namenode not starting
>>
>> Hello,
>>
>> I had a running hadoop cluster.
>> I restarted it and after that namenode is unable to start. I am getting
>> error saying that it's not formatted. :(
>> Is it possible to recover the data on HDFS?
>>
>> 2012-08-24 03:17:55,378 ERROR
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>> initialization failed.
>> java.io.IOException: NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>> 2012-08-24 03:17:55,380 ERROR
>> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
>> NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>>
>> Regards,
>> Abhay
>>
>>
>
--
Håvard Wahl Kongsgård
Faculty of Medicine &
Department of Mathematical Sciences
NTNU
http://havard.security-review.net/
Re: namenode not starting
Posted by Håvard Wahl Kongsgård <ha...@gmail.com>.
You should start with a reboot of the system.
A lesson to everyone, this is exactly why you should have a secondary
name node (http://wiki.apache.org/hadoop/FAQ#What_is_the_purpose_of_the_secondary_name-node.3F)
and run the namenode a mirrored RAID-5/10 disk.
-Håvard
On Fri, Aug 24, 2012 at 9:40 AM, Abhay Ratnaparkhi
<ab...@gmail.com> wrote:
> Hello,
>
> I was using cluster for long time and not formatted the namenode.
> I ran bin/stop-all.sh and bin/start-all.sh scripts only.
>
> I am using NFS for dfs.name.dir.
> hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any way to
> recover the data?
>
> Thanks,
> Abhay
>
>
> On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
>>
>> Hi Abhay
>>
>> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to
>> /tmp the contents would be deleted on a OS restart. You need to change this
>> location before you start your NN.
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ________________________________
>> From: Abhay Ratnaparkhi <ab...@gmail.com>
>> Date: Fri, 24 Aug 2012 12:58:41 +0530
>> To: <us...@hadoop.apache.org>
>> ReplyTo: user@hadoop.apache.org
>> Subject: namenode not starting
>>
>> Hello,
>>
>> I had a running hadoop cluster.
>> I restarted it and after that namenode is unable to start. I am getting
>> error saying that it's not formatted. :(
>> Is it possible to recover the data on HDFS?
>>
>> 2012-08-24 03:17:55,378 ERROR
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>> initialization failed.
>> java.io.IOException: NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>> 2012-08-24 03:17:55,380 ERROR
>> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
>> NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>>
>> Regards,
>> Abhay
>>
>>
>
--
Håvard Wahl Kongsgård
Faculty of Medicine &
Department of Mathematical Sciences
NTNU
http://havard.security-review.net/
Re: namenode not starting
Posted by Harsh J <ha...@cloudera.com>.
Abhay,
I suspect that if you haven't set your dfs.name.dir explicitly, then
you haven't set fs.checkpoint.dir either, and since both use
hadoop.tmp.dir paths, you may have lost your data completely and there
is no recovery possible now.
On Fri, Aug 24, 2012 at 1:10 PM, Abhay Ratnaparkhi
<ab...@gmail.com> wrote:
> Hello,
>
> I was using cluster for long time and not formatted the namenode.
> I ran bin/stop-all.sh and bin/start-all.sh scripts only.
>
> I am using NFS for dfs.name.dir.
> hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any way to
> recover the data?
>
> Thanks,
> Abhay
>
>
> On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
>>
>> Hi Abhay
>>
>> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to
>> /tmp the contents would be deleted on a OS restart. You need to change this
>> location before you start your NN.
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ________________________________
>> From: Abhay Ratnaparkhi <ab...@gmail.com>
>> Date: Fri, 24 Aug 2012 12:58:41 +0530
>> To: <us...@hadoop.apache.org>
>> ReplyTo: user@hadoop.apache.org
>> Subject: namenode not starting
>>
>> Hello,
>>
>> I had a running hadoop cluster.
>> I restarted it and after that namenode is unable to start. I am getting
>> error saying that it's not formatted. :(
>> Is it possible to recover the data on HDFS?
>>
>> 2012-08-24 03:17:55,378 ERROR
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
>> initialization failed.
>> java.io.IOException: NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>> 2012-08-24 03:17:55,380 ERROR
>> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
>> NameNode is not formatted.
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>>
>> Regards,
>> Abhay
>>
>>
>
--
Harsh J
Re: namenode not starting
Posted by Abhay Ratnaparkhi <ab...@gmail.com>.
Hello,
I was using cluster for long time and not formatted the namenode.
I ran bin/stop-all.sh and bin/start-all.sh scripts only.
I am using NFS for dfs.name.dir.
hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any way to
recover the data?
Thanks,
Abhay
On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
> **
> Hi Abhay
>
> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to
> /tmp the contents would be deleted on a OS restart. You need to change this
> location before you start your NN.
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Abhay Ratnaparkhi <ab...@gmail.com>
> *Date: *Fri, 24 Aug 2012 12:58:41 +0530
> *To: *<us...@hadoop.apache.org>
> *ReplyTo: * user@hadoop.apache.org
> *Subject: *namenode not starting
>
> Hello,
>
> I had a running hadoop cluster.
> I restarted it and after that namenode is unable to start. I am getting
> error saying that it's not formatted. :(
> Is it possible to recover the data on HDFS?
>
> 2012-08-24 03:17:55,378 ERROR
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> initialization failed.
> java.io.IOException: NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
> 2012-08-24 03:17:55,380 ERROR
> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>
> Regards,
> Abhay
>
>
>
Re: namenode not starting
Posted by Abhay Ratnaparkhi <ab...@gmail.com>.
Hello,
I was using cluster for long time and not formatted the namenode.
I ran bin/stop-all.sh and bin/start-all.sh scripts only.
I am using NFS for dfs.name.dir.
hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any way to
recover the data?
Thanks,
Abhay
On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
> **
> Hi Abhay
>
> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to
> /tmp the contents would be deleted on a OS restart. You need to change this
> location before you start your NN.
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Abhay Ratnaparkhi <ab...@gmail.com>
> *Date: *Fri, 24 Aug 2012 12:58:41 +0530
> *To: *<us...@hadoop.apache.org>
> *ReplyTo: * user@hadoop.apache.org
> *Subject: *namenode not starting
>
> Hello,
>
> I had a running hadoop cluster.
> I restarted it and after that namenode is unable to start. I am getting
> error saying that it's not formatted. :(
> Is it possible to recover the data on HDFS?
>
> 2012-08-24 03:17:55,378 ERROR
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> initialization failed.
> java.io.IOException: NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
> 2012-08-24 03:17:55,380 ERROR
> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>
> Regards,
> Abhay
>
>
>
Re: namenode not starting
Posted by Abhay Ratnaparkhi <ab...@gmail.com>.
Hello,
I was using cluster for long time and not formatted the namenode.
I ran bin/stop-all.sh and bin/start-all.sh scripts only.
I am using NFS for dfs.name.dir.
hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any way to
recover the data?
Thanks,
Abhay
On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
> **
> Hi Abhay
>
> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to
> /tmp the contents would be deleted on a OS restart. You need to change this
> location before you start your NN.
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Abhay Ratnaparkhi <ab...@gmail.com>
> *Date: *Fri, 24 Aug 2012 12:58:41 +0530
> *To: *<us...@hadoop.apache.org>
> *ReplyTo: * user@hadoop.apache.org
> *Subject: *namenode not starting
>
> Hello,
>
> I had a running hadoop cluster.
> I restarted it and after that namenode is unable to start. I am getting
> error saying that it's not formatted. :(
> Is it possible to recover the data on HDFS?
>
> 2012-08-24 03:17:55,378 ERROR
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> initialization failed.
> java.io.IOException: NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
> 2012-08-24 03:17:55,380 ERROR
> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>
> Regards,
> Abhay
>
>
>
Re: namenode not starting
Posted by Abhay Ratnaparkhi <ab...@gmail.com>.
Hello,
I was using cluster for long time and not formatted the namenode.
I ran bin/stop-all.sh and bin/start-all.sh scripts only.
I am using NFS for dfs.name.dir.
hadoop.tmp.dir is a /tmp directory. I've not restarted the OS. Any way to
recover the data?
Thanks,
Abhay
On Fri, Aug 24, 2012 at 1:01 PM, Bejoy KS <be...@gmail.com> wrote:
> **
> Hi Abhay
>
> What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to
> /tmp the contents would be deleted on a OS restart. You need to change this
> location before you start your NN.
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Abhay Ratnaparkhi <ab...@gmail.com>
> *Date: *Fri, 24 Aug 2012 12:58:41 +0530
> *To: *<us...@hadoop.apache.org>
> *ReplyTo: * user@hadoop.apache.org
> *Subject: *namenode not starting
>
> Hello,
>
> I had a running hadoop cluster.
> I restarted it and after that namenode is unable to start. I am getting
> error saying that it's not formatted. :(
> Is it possible to recover the data on HDFS?
>
> 2012-08-24 03:17:55,378 ERROR
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> initialization failed.
> java.io.IOException: NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
> 2012-08-24 03:17:55,380 ERROR
> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>
> Regards,
> Abhay
>
>
>
Re: namenode not starting
Posted by Bejoy KS <be...@gmail.com>.
Hi Abhay
What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to /tmp the contents would be deleted on a OS restart. You need to change this location before you start your NN.
Regards
Bejoy KS
Sent from handheld, please excuse typos.
-----Original Message-----
From: Abhay Ratnaparkhi <ab...@gmail.com>
Date: Fri, 24 Aug 2012 12:58:41
To: <us...@hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Subject: namenode not starting
Hello,
I had a running hadoop cluster.
I restarted it and after that namenode is unable to start. I am getting
error saying that it's not formatted. :(
Is it possible to recover the data on HDFS?
2012-08-24 03:17:55,378 ERROR
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
initialization failed.
java.io.IOException: NameNode is not formatted.
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
2012-08-24 03:17:55,380 ERROR
org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
NameNode is not formatted.
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
Regards,
Abhay
Re: namenode not starting
Posted by Bejoy KS <be...@gmail.com>.
Hi Abhay
What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to /tmp the contents would be deleted on a OS restart. You need to change this location before you start your NN.
Regards
Bejoy KS
Sent from handheld, please excuse typos.
-----Original Message-----
From: Abhay Ratnaparkhi <ab...@gmail.com>
Date: Fri, 24 Aug 2012 12:58:41
To: <us...@hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Subject: namenode not starting
Hello,
I had a running hadoop cluster.
I restarted it and after that namenode is unable to start. I am getting
error saying that it's not formatted. :(
Is it possible to recover the data on HDFS?
2012-08-24 03:17:55,378 ERROR
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
initialization failed.
java.io.IOException: NameNode is not formatted.
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
2012-08-24 03:17:55,380 ERROR
org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
NameNode is not formatted.
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
Regards,
Abhay
Re: namenode not starting
Posted by Bejoy KS <be...@gmail.com>.
Hi Abhay
What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to /tmp the contents would be deleted on a OS restart. You need to change this location before you start your NN.
Regards
Bejoy KS
Sent from handheld, please excuse typos.
-----Original Message-----
From: Abhay Ratnaparkhi <ab...@gmail.com>
Date: Fri, 24 Aug 2012 12:58:41
To: <us...@hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Subject: namenode not starting
Hello,
I had a running hadoop cluster.
I restarted it and after that namenode is unable to start. I am getting
error saying that it's not formatted. :(
Is it possible to recover the data on HDFS?
2012-08-24 03:17:55,378 ERROR
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
initialization failed.
java.io.IOException: NameNode is not formatted.
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
2012-08-24 03:17:55,380 ERROR
org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
NameNode is not formatted.
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
Regards,
Abhay
Re: namenode not starting
Posted by Nitin Pawar <ni...@gmail.com>.
did you run the command bin/hadoop namenode -format before starting
the namenode ?
On Fri, Aug 24, 2012 at 12:58 PM, Abhay Ratnaparkhi
<ab...@gmail.com> wrote:
> Hello,
>
> I had a running hadoop cluster.
> I restarted it and after that namenode is unable to start. I am getting
> error saying that it's not formatted. :(
> Is it possible to recover the data on HDFS?
>
> 2012-08-24 03:17:55,378 ERROR
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> initialization failed.
> java.io.IOException: NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
> 2012-08-24 03:17:55,380 ERROR
> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>
> Regards,
> Abhay
>
>
--
Nitin Pawar
Re: namenode not starting
Posted by vivek <vi...@gmail.com>.
hi,
Have u rubn the command namenode -format???
Thanks & regards ,
Vivek
On Fri, Aug 24, 2012 at 12:58 PM, Abhay Ratnaparkhi <
abhay.ratnaparkhi@gmail.com> wrote:
> Hello,
>
> I had a running hadoop cluster.
> I restarted it and after that namenode is unable to start. I am getting
> error saying that it's not formatted. :(
> Is it possible to recover the data on HDFS?
>
> 2012-08-24 03:17:55,378 ERROR
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> initialization failed.
> java.io.IOException: NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
> 2012-08-24 03:17:55,380 ERROR
> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>
> Regards,
> Abhay
>
>
>
--
Thanks and Regards,
VIVEK KOUL
Re: namenode not starting
Posted by Nitin Pawar <ni...@gmail.com>.
did you run the command bin/hadoop namenode -format before starting
the namenode ?
On Fri, Aug 24, 2012 at 12:58 PM, Abhay Ratnaparkhi
<ab...@gmail.com> wrote:
> Hello,
>
> I had a running hadoop cluster.
> I restarted it and after that namenode is unable to start. I am getting
> error saying that it's not formatted. :(
> Is it possible to recover the data on HDFS?
>
> 2012-08-24 03:17:55,378 ERROR
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> initialization failed.
> java.io.IOException: NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
> 2012-08-24 03:17:55,380 ERROR
> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>
> Regards,
> Abhay
>
>
--
Nitin Pawar
Re: namenode not starting
Posted by vivek <vi...@gmail.com>.
hi,
Have u rubn the command namenode -format???
Thanks & regards ,
Vivek
On Fri, Aug 24, 2012 at 12:58 PM, Abhay Ratnaparkhi <
abhay.ratnaparkhi@gmail.com> wrote:
> Hello,
>
> I had a running hadoop cluster.
> I restarted it and after that namenode is unable to start. I am getting
> error saying that it's not formatted. :(
> Is it possible to recover the data on HDFS?
>
> 2012-08-24 03:17:55,378 ERROR
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> initialization failed.
> java.io.IOException: NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
> 2012-08-24 03:17:55,380 ERROR
> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>
> Regards,
> Abhay
>
>
>
--
Thanks and Regards,
VIVEK KOUL
Re: namenode not starting
Posted by vivek <vi...@gmail.com>.
hi,
Have u rubn the command namenode -format???
Thanks & regards ,
Vivek
On Fri, Aug 24, 2012 at 12:58 PM, Abhay Ratnaparkhi <
abhay.ratnaparkhi@gmail.com> wrote:
> Hello,
>
> I had a running hadoop cluster.
> I restarted it and after that namenode is unable to start. I am getting
> error saying that it's not formatted. :(
> Is it possible to recover the data on HDFS?
>
> 2012-08-24 03:17:55,378 ERROR
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> initialization failed.
> java.io.IOException: NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
> 2012-08-24 03:17:55,380 ERROR
> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>
> Regards,
> Abhay
>
>
>
--
Thanks and Regards,
VIVEK KOUL
Re: namenode not starting
Posted by vivek <vi...@gmail.com>.
hi,
Have u rubn the command namenode -format???
Thanks & regards ,
Vivek
On Fri, Aug 24, 2012 at 12:58 PM, Abhay Ratnaparkhi <
abhay.ratnaparkhi@gmail.com> wrote:
> Hello,
>
> I had a running hadoop cluster.
> I restarted it and after that namenode is unable to start. I am getting
> error saying that it's not formatted. :(
> Is it possible to recover the data on HDFS?
>
> 2012-08-24 03:17:55,378 ERROR
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
> initialization failed.
> java.io.IOException: NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
> 2012-08-24 03:17:55,380 ERROR
> org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
> NameNode is not formatted.
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
>
> Regards,
> Abhay
>
>
>
--
Thanks and Regards,
VIVEK KOUL
Re: namenode not starting
Posted by Bejoy KS <be...@gmail.com>.
Hi Abhay
What is the value for hadoop.tmp.dir or dfs.name.dir . If it was set to /tmp the contents would be deleted on a OS restart. You need to change this location before you start your NN.
Regards
Bejoy KS
Sent from handheld, please excuse typos.
-----Original Message-----
From: Abhay Ratnaparkhi <ab...@gmail.com>
Date: Fri, 24 Aug 2012 12:58:41
To: <us...@hadoop.apache.org>
Reply-To: user@hadoop.apache.org
Subject: namenode not starting
Hello,
I had a running hadoop cluster.
I restarted it and after that namenode is unable to start. I am getting
error saying that it's not formatted. :(
Is it possible to recover the data on HDFS?
2012-08-24 03:17:55,378 ERROR
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem
initialization failed.
java.io.IOException: NameNode is not formatted.
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
2012-08-24 03:17:55,380 ERROR
org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException:
NameNode is not formatted.
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:434)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:110)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:291)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:270)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:271)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:303)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:433)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:421)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1359)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1368)
Regards,
Abhay