You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Brendan cheng <cc...@hotmail.com> on 2012/05/22 05:21:44 UTC
namenode directory disappear after machines restart
Hi,
I'm not sure if there is a setting to avoid the Namenode removed after hosting machine of Namenode restart.I found that after successfully installed single node pseudo distributed hadoop following from your website, the name node dir /tmp/hadoop-brendan/dfs/name are removed if machine reboot.
What do I miss?
Brendan
2012-05-22 11:14:05,678 INFO org.apache.hadoop.hdfs.server.common.Storage: Storage directory /tmp/hadoop-brendan/dfs/name does not exist.2012-05-22 11:14:05,680 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed.org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /tmp/hadoop-brendan/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible. at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:303) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)2012-05-22 11:14:05,685 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /tmp/hadoop-brendan/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible. at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:303) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)
Re: namenode directory disappear after machines restart
Posted by Mohammad Tariq <do...@gmail.com>.
That's great..The best way to get these kind of info is to ask
questions on the mailing list whenever we face any problem.There are
certain things that are not documented anywhere. I had faced a lot of
problems initially, but the community and the people are really
great.There are so many experienced people that we get the answers for
almost all of our problems.(I am also planning to write a blog this
weekend that addresses these kind of problems so that people like us
do not face the problems that we have already faced).
Regards,
Mohammad Tariq
On Tue, May 22, 2012 at 3:04 PM, Brendan cheng <cc...@hotmail.com> wrote:
>
> Thanks and it works!
> I wonder where can we find all the settings. I check the code for hdfs-default.xml but it doesn't have the settings you mentioned.
> Brendan
>
> ----------------------------------------
>> From: dontariq@gmail.com
>> Date: Tue, 22 May 2012 13:03:17 +0530
>> Subject: Re: namenode directory disappear after machines restart
>> To: hdfs-user@hadoop.apache.org
>>
>> Hello Brendan,
>>
>> Do as suggested by Marcos..If you do not set these properties,
>> Hadoop uses "tmp" directory by default..Apart from setting these
>> properties in your hdfs-site.xml file add the following property in
>> your core-site.xml file -
>>
>> <property>
>> <name>hadoop.tmp.dir</name>
>> <value>some_permanent_path</value>
>> </property>
>>
>> Regards,
>> Mohammad Tariq
>>
>>
>> On Tue, May 22, 2012 at 10:54 AM, Marcos Ortiz <ml...@uci.cu> wrote:
>> > This is an usual behavior on Unix/Linux systems. When you restart the
>> > system, the content of the /tmp directory is cleaned, because precisely, the
>> > purpose of this directory is to keep files temporally.
>> > For that reason, the data directory for the HDFS filesystem should be
>> > another directory, /var/hadoop/data for example, of course, a directory
>> > durable in time.
>> > So, you should change your dfs.name.dir and your dfs.data.dir variable in
>> > your hdfs-site.xml.
>> >
>> > Regards
>> >
>> >
>> > On 05/21/2012 11:21 PM, Brendan cheng wrote:
>> >>
>> >> Hi,
>> >> I'm not sure if there is a setting to avoid the Namenode removed after
>> >> hosting machine of Namenode restart.I found that after successfully
>> >> installed single node pseudo distributed hadoop following from your website,
>> >> the name node dir /tmp/hadoop-brendan/dfs/name are removed if machine
>> >> reboot.
>> >> What do I miss?
>> >> Brendan
>> >> 2012-05-22 11:14:05,678 INFO org.apache.hadoop.hdfs.server.common.Storage:
>> >> Storage directory /tmp/hadoop-brendan/dfs/name does not exist.2012-05-22
>> >> 11:14:05,680 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
>> >> FSNamesystem initialization
>> >> failed.org.apache.hadoop.hdfs.server.common.InconsistentFSStateException:
>> >> Directory /tmp/hadoop-brendan/dfs/name is in an inconsistent state: storage
>> >> directory does not exist or is not accessible. at
>> >> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:303)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)2012-05-22
>> >> 11:14:05,685 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
>> >> org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory
>> >> /tmp/hadoop-brendan/dfs/name is in an inconsistent state: storage directory
>> >> does not exist or is not accessible. at
>> >> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:303)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)
>> >>
>> >> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS
>> >> INFORMATICAS...
>> >> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>> >>
>> >> http://www.uci.cu
>> >> http://www.facebook.com/universidad.uci
>> >> http://www.flickr.com/photos/universidad_uci
>> >
>> >
>> > --
>> > Marcos Luis Ortíz Valmaseda
>> > Data Engineer&& Sr. System Administrator at UCI
>> > http://marcosluis2186.posterous.com
>> > http://www.linkedin.com/in/marcosluis2186
>> > Twitter: @marcosluis2186
>> >
>> >
>> >
>> > 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS
>> > INFORMATICAS...
>> > CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>> >
>> > http://www.uci.cu
>> > http://www.facebook.com/universidad.uci
>> > http://www.flickr.com/photos/universidad_uci
>
Re: namenode directory disappear after machines restart
Posted by Harsh J <ha...@cloudera.com>.
Brendan,
The hdfs-default.xml does have dfs.name.dir listed:
http://hadoop.apache.org/common/docs/current/hdfs-default.html. The
configuration is also mentioned on the official tutorial:
http://hadoop.apache.org/common/docs/current/cluster_setup.html#Configuration+Files
On Tue, May 22, 2012 at 3:04 PM, Brendan cheng <cc...@hotmail.com> wrote:
>
> Thanks and it works!
> I wonder where can we find all the settings. I check the code for hdfs-default.xml but it doesn't have the settings you mentioned.
> Brendan
>
> ----------------------------------------
>> From: dontariq@gmail.com
>> Date: Tue, 22 May 2012 13:03:17 +0530
>> Subject: Re: namenode directory disappear after machines restart
>> To: hdfs-user@hadoop.apache.org
>>
>> Hello Brendan,
>>
>> Do as suggested by Marcos..If you do not set these properties,
>> Hadoop uses "tmp" directory by default..Apart from setting these
>> properties in your hdfs-site.xml file add the following property in
>> your core-site.xml file -
>>
>> <property>
>> <name>hadoop.tmp.dir</name>
>> <value>some_permanent_path</value>
>> </property>
>>
>> Regards,
>> Mohammad Tariq
>>
>>
>> On Tue, May 22, 2012 at 10:54 AM, Marcos Ortiz <ml...@uci.cu> wrote:
>> > This is an usual behavior on Unix/Linux systems. When you restart the
>> > system, the content of the /tmp directory is cleaned, because precisely, the
>> > purpose of this directory is to keep files temporally.
>> > For that reason, the data directory for the HDFS filesystem should be
>> > another directory, /var/hadoop/data for example, of course, a directory
>> > durable in time.
>> > So, you should change your dfs.name.dir and your dfs.data.dir variable in
>> > your hdfs-site.xml.
>> >
>> > Regards
>> >
>> >
>> > On 05/21/2012 11:21 PM, Brendan cheng wrote:
>> >>
>> >> Hi,
>> >> I'm not sure if there is a setting to avoid the Namenode removed after
>> >> hosting machine of Namenode restart.I found that after successfully
>> >> installed single node pseudo distributed hadoop following from your website,
>> >> the name node dir /tmp/hadoop-brendan/dfs/name are removed if machine
>> >> reboot.
>> >> What do I miss?
>> >> Brendan
>> >> 2012-05-22 11:14:05,678 INFO org.apache.hadoop.hdfs.server.common.Storage:
>> >> Storage directory /tmp/hadoop-brendan/dfs/name does not exist.2012-05-22
>> >> 11:14:05,680 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
>> >> FSNamesystem initialization
>> >> failed.org.apache.hadoop.hdfs.server.common.InconsistentFSStateException:
>> >> Directory /tmp/hadoop-brendan/dfs/name is in an inconsistent state: storage
>> >> directory does not exist or is not accessible. at
>> >> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:303)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)2012-05-22
>> >> 11:14:05,685 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
>> >> org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory
>> >> /tmp/hadoop-brendan/dfs/name is in an inconsistent state: storage directory
>> >> does not exist or is not accessible. at
>> >> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:303)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279)
>> >> at
>> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)
>> >>
>> >> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS
>> >> INFORMATICAS...
>> >> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>> >>
>> >> http://www.uci.cu
>> >> http://www.facebook.com/universidad.uci
>> >> http://www.flickr.com/photos/universidad_uci
>> >
>> >
>> > --
>> > Marcos Luis Ortíz Valmaseda
>> > Data Engineer&& Sr. System Administrator at UCI
>> > http://marcosluis2186.posterous.com
>> > http://www.linkedin.com/in/marcosluis2186
>> > Twitter: @marcosluis2186
>> >
>> >
>> >
>> > 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS
>> > INFORMATICAS...
>> > CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>> >
>> > http://www.uci.cu
>> > http://www.facebook.com/universidad.uci
>> > http://www.flickr.com/photos/universidad_uci
>
--
Harsh J
RE: namenode directory disappear after machines restart
Posted by Brendan cheng <cc...@hotmail.com>.
Thanks and it works!
I wonder where can we find all the settings. I check the code for hdfs-default.xml but it doesn't have the settings you mentioned.
Brendan
----------------------------------------
> From: dontariq@gmail.com
> Date: Tue, 22 May 2012 13:03:17 +0530
> Subject: Re: namenode directory disappear after machines restart
> To: hdfs-user@hadoop.apache.org
>
> Hello Brendan,
>
> Do as suggested by Marcos..If you do not set these properties,
> Hadoop uses "tmp" directory by default..Apart from setting these
> properties in your hdfs-site.xml file add the following property in
> your core-site.xml file -
>
> <property>
> <name>hadoop.tmp.dir</name>
> <value>some_permanent_path</value>
> </property>
>
> Regards,
> Mohammad Tariq
>
>
> On Tue, May 22, 2012 at 10:54 AM, Marcos Ortiz <ml...@uci.cu> wrote:
> > This is an usual behavior on Unix/Linux systems. When you restart the
> > system, the content of the /tmp directory is cleaned, because precisely, the
> > purpose of this directory is to keep files temporally.
> > For that reason, the data directory for the HDFS filesystem should be
> > another directory, /var/hadoop/data for example, of course, a directory
> > durable in time.
> > So, you should change your dfs.name.dir and your dfs.data.dir variable in
> > your hdfs-site.xml.
> >
> > Regards
> >
> >
> > On 05/21/2012 11:21 PM, Brendan cheng wrote:
> >>
> >> Hi,
> >> I'm not sure if there is a setting to avoid the Namenode removed after
> >> hosting machine of Namenode restart.I found that after successfully
> >> installed single node pseudo distributed hadoop following from your website,
> >> the name node dir /tmp/hadoop-brendan/dfs/name are removed if machine
> >> reboot.
> >> What do I miss?
> >> Brendan
> >> 2012-05-22 11:14:05,678 INFO org.apache.hadoop.hdfs.server.common.Storage:
> >> Storage directory /tmp/hadoop-brendan/dfs/name does not exist.2012-05-22
> >> 11:14:05,680 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
> >> FSNamesystem initialization
> >> failed.org.apache.hadoop.hdfs.server.common.InconsistentFSStateException:
> >> Directory /tmp/hadoop-brendan/dfs/name is in an inconsistent state: storage
> >> directory does not exist or is not accessible. at
> >> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:303)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)2012-05-22
> >> 11:14:05,685 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
> >> org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory
> >> /tmp/hadoop-brendan/dfs/name is in an inconsistent state: storage directory
> >> does not exist or is not accessible. at
> >> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:303)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279)
> >> at
> >> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)
> >>
> >> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS
> >> INFORMATICAS...
> >> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
> >>
> >> http://www.uci.cu
> >> http://www.facebook.com/universidad.uci
> >> http://www.flickr.com/photos/universidad_uci
> >
> >
> > --
> > Marcos Luis Ortíz Valmaseda
> > Data Engineer&& Sr. System Administrator at UCI
> > http://marcosluis2186.posterous.com
> > http://www.linkedin.com/in/marcosluis2186
> > Twitter: @marcosluis2186
> >
> >
> >
> > 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS
> > INFORMATICAS...
> > CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
> >
> > http://www.uci.cu
> > http://www.facebook.com/universidad.uci
> > http://www.flickr.com/photos/universidad_uci
Re: namenode directory disappear after machines restart
Posted by Mohammad Tariq <do...@gmail.com>.
Hello Brendan,
Do as suggested by Marcos..If you do not set these properties,
Hadoop uses "tmp" directory by default..Apart from setting these
properties in your hdfs-site.xml file add the following property in
your core-site.xml file -
<property>
<name>hadoop.tmp.dir</name>
<value>some_permanent_path</value>
</property>
Regards,
Mohammad Tariq
On Tue, May 22, 2012 at 10:54 AM, Marcos Ortiz <ml...@uci.cu> wrote:
> This is an usual behavior on Unix/Linux systems. When you restart the
> system, the content of the /tmp directory is cleaned, because precisely, the
> purpose of this directory is to keep files temporally.
> For that reason, the data directory for the HDFS filesystem should be
> another directory, /var/hadoop/data for example, of course, a directory
> durable in time.
> So, you should change your dfs.name.dir and your dfs.data.dir variable in
> your hdfs-site.xml.
>
> Regards
>
>
> On 05/21/2012 11:21 PM, Brendan cheng wrote:
>>
>> Hi,
>> I'm not sure if there is a setting to avoid the Namenode removed after
>> hosting machine of Namenode restart.I found that after successfully
>> installed single node pseudo distributed hadoop following from your website,
>> the name node dir /tmp/hadoop-brendan/dfs/name are removed if machine
>> reboot.
>> What do I miss?
>> Brendan
>> 2012-05-22 11:14:05,678 INFO org.apache.hadoop.hdfs.server.common.Storage:
>> Storage directory /tmp/hadoop-brendan/dfs/name does not exist.2012-05-22
>> 11:14:05,680 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem:
>> FSNamesystem initialization
>> failed.org.apache.hadoop.hdfs.server.common.InconsistentFSStateException:
>> Directory /tmp/hadoop-brendan/dfs/name is in an inconsistent state: storage
>> directory does not exist or is not accessible. at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)2012-05-22
>> 11:14:05,685 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
>> org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory
>> /tmp/hadoop-brendan/dfs/name is in an inconsistent state: storage directory
>> does not exist or is not accessible. at
>> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:303)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388)
>> at
>> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279)
>> at
>> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)
>>
>> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS
>> INFORMATICAS...
>> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>>
>> http://www.uci.cu
>> http://www.facebook.com/universidad.uci
>> http://www.flickr.com/photos/universidad_uci
>
>
> --
> Marcos Luis Ortíz Valmaseda
> Data Engineer&& Sr. System Administrator at UCI
> http://marcosluis2186.posterous.com
> http://www.linkedin.com/in/marcosluis2186
> Twitter: @marcosluis2186
>
>
>
> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS
> INFORMATICAS...
> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>
> http://www.uci.cu
> http://www.facebook.com/universidad.uci
> http://www.flickr.com/photos/universidad_uci
Re: namenode directory disappear after machines restart
Posted by Marcos Ortiz <ml...@uci.cu>.
This is an usual behavior on Unix/Linux systems. When you restart the
system, the content of the /tmp directory is cleaned, because precisely,
the purpose of this directory is to keep files temporally.
For that reason, the data directory for the HDFS filesystem should be
another directory, /var/hadoop/data for example, of course, a directory
durable in time.
So, you should change your dfs.name.dir and your dfs.data.dir variable
in your hdfs-site.xml.
Regards
On 05/21/2012 11:21 PM, Brendan cheng wrote:
> Hi,
> I'm not sure if there is a setting to avoid the Namenode removed after hosting machine of Namenode restart.I found that after successfully installed single node pseudo distributed hadoop following from your website, the name node dir /tmp/hadoop-brendan/dfs/name are removed if machine reboot.
> What do I miss?
> Brendan
> 2012-05-22 11:14:05,678 INFO org.apache.hadoop.hdfs.server.common.Storage: Storage directory /tmp/hadoop-brendan/dfs/name does not exist.2012-05-22 11:14:05,680 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed.org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /tmp/hadoop-brendan/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible. at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:303) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)2012-05-22 11:14:05,685 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /tmp/hadoop-brendan/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible. at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:303) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:388) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:362) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:276) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:496) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1279) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1288)
> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>
> http://www.uci.cu
> http://www.facebook.com/universidad.uci
> http://www.flickr.com/photos/universidad_uci
--
Marcos Luis Ortíz Valmaseda
Data Engineer&& Sr. System Administrator at UCI
http://marcosluis2186.posterous.com
http://www.linkedin.com/in/marcosluis2186
Twitter: @marcosluis2186
10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci