You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by mike anderson <sa...@gmail.com> on 2011/02/10 17:57:59 UTC
multiple namenode directories
This should be a straightforward question, but better safe than sorry.
I wanted to add a second name node directory (on an NFS as a backup), so now
my hdfs-site.xml contains:
<property>
<name>dfs.name.dir</name>
<value>/mnt/hadoop/name</value>
</property>
<property>
<name>dfs.name.dir</name>
<value>/public/hadoop/name</value>
</property>
When I go to start DFS i'm getting the exception:
org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory
/public/hadoop/name is in an inconsistent state: storage directory does not
exist or is not accessible.
After googling a bit, it seems like I want to do "bin/hadoop namenode
-format"
Is this right? As long as I shut down DFS before issuing the command I
shouldn't lose any data?
Thanks in advance,
Mike
Re: multiple namenode directories
Posted by Harsh J <qw...@gmail.com>.
The links appeared outdated, I've updated those to reflect the current
release 0.21's configurations. The configuration descriptions describe
properly, the way to set them 'right'.
For 0.20 releases, only the configuration name changes:
dfs.name.dir instead of dfs.namenode.name.dir, and
dfs.data.dir instead of dfs.datanode.data.dir
The value formatting remains the same.
On Thu, Feb 10, 2011 at 11:18 PM, mike anderson <sa...@gmail.com> wrote:
> Whew, glad I asked.
>
> It might be useful for someone to update the wiki:
> http://wiki.apache.org/hadoop/FAQ#How_do_I_set_up_a_hadoop_node_to_use_multiple_volumes.3F
>
> -Mike
>
> On Thu, Feb 10, 2011 at 12:43 PM, Harsh J <qw...@gmail.com> wrote:
>
>> DO NOT format your NameNode. Formatting a NameNode is equivalent to
>> formatting a FS -- you're bound lose it all.
>>
>> And while messing with NameNode, after bringing it down safely, ALWAYS
>> take a backup of the existing dfs.name.dir contents and preferably the
>> SNN checkpoint directory contents too (if you're running it).
>>
>> The RIGHT way to add new directories to the NameNode's dfs.name.dir is
>> by comma-separating them in the same value and NOT by adding two
>> properties - that is not how Hadoop's configuration operates. In your
>> case, bring NN down and edit conf as:
>>
>> > <property>
>> > <name>dfs.name.dir</name>
>> > <value>/mnt/hadoop/name,/public/hadoop/name</value>
>> > </property>
>>
>> Create the new directory by copying the existing one. Both must have
>> the SAME file and structure in them, like mirror copies of one
>> another. Ensure that this new location, apart from being symmetric in
>> content, is also symmetric in permissions. NameNode will require WRITE
>> permissions via its user on all locations configured.
>>
>> Having configured properly and ensured that both storage directories
>> mirror one another, launch your NameNode back up again (feel a little
>> paranoid and do check namenode logs for any issues -- in which case
>> your backup would be very essential as a requirement for recovery!).
>>
>> P.s. Hold on for a bit for a possible comment from another user before
>> getting into action. I've added extra directories this way, but I do
>> not know if this is "the" genuine way to do so - although it feels
>> right to me.
>>
>> On Thu, Feb 10, 2011 at 10:27 PM, mike anderson <sa...@gmail.com>
>> wrote:
>> > This should be a straightforward question, but better safe than sorry.
>> >
>> > I wanted to add a second name node directory (on an NFS as a backup), so
>> now
>> > my hdfs-site.xml contains:
>> >
>> > <property>
>> > <name>dfs.name.dir</name>
>> > <value>/mnt/hadoop/name</value>
>> > </property>
>> > <property>
>> > <name>dfs.name.dir</name>
>> > <value>/public/hadoop/name</value>
>> > </property>
>> >
>> >
>> > When I go to start DFS i'm getting the exception:
>> >
>> > org.apache.hadoop.hdfs.server.common.InconsistentFSStateException:
>> Directory
>> > /public/hadoop/name is in an inconsistent state: storage directory does
>> not
>> > exist or is not accessible.
>> >
>> >
>> > After googling a bit, it seems like I want to do "bin/hadoop namenode
>> > -format"
>> >
>> > Is this right? As long as I shut down DFS before issuing the command I
>> > shouldn't lose any data?
>> >
>> > Thanks in advance,
>> > Mike
>> >
>>
>>
>>
>> --
>> Harsh J
>> www.harshj.com
>>
>
--
Harsh J
www.harshj.com
Re: multiple namenode directories
Posted by mike anderson <sa...@gmail.com>.
Whew, glad I asked.
It might be useful for someone to update the wiki:
http://wiki.apache.org/hadoop/FAQ#How_do_I_set_up_a_hadoop_node_to_use_multiple_volumes.3F
-Mike
On Thu, Feb 10, 2011 at 12:43 PM, Harsh J <qw...@gmail.com> wrote:
> DO NOT format your NameNode. Formatting a NameNode is equivalent to
> formatting a FS -- you're bound lose it all.
>
> And while messing with NameNode, after bringing it down safely, ALWAYS
> take a backup of the existing dfs.name.dir contents and preferably the
> SNN checkpoint directory contents too (if you're running it).
>
> The RIGHT way to add new directories to the NameNode's dfs.name.dir is
> by comma-separating them in the same value and NOT by adding two
> properties - that is not how Hadoop's configuration operates. In your
> case, bring NN down and edit conf as:
>
> > <property>
> > <name>dfs.name.dir</name>
> > <value>/mnt/hadoop/name,/public/hadoop/name</value>
> > </property>
>
> Create the new directory by copying the existing one. Both must have
> the SAME file and structure in them, like mirror copies of one
> another. Ensure that this new location, apart from being symmetric in
> content, is also symmetric in permissions. NameNode will require WRITE
> permissions via its user on all locations configured.
>
> Having configured properly and ensured that both storage directories
> mirror one another, launch your NameNode back up again (feel a little
> paranoid and do check namenode logs for any issues -- in which case
> your backup would be very essential as a requirement for recovery!).
>
> P.s. Hold on for a bit for a possible comment from another user before
> getting into action. I've added extra directories this way, but I do
> not know if this is "the" genuine way to do so - although it feels
> right to me.
>
> On Thu, Feb 10, 2011 at 10:27 PM, mike anderson <sa...@gmail.com>
> wrote:
> > This should be a straightforward question, but better safe than sorry.
> >
> > I wanted to add a second name node directory (on an NFS as a backup), so
> now
> > my hdfs-site.xml contains:
> >
> > <property>
> > <name>dfs.name.dir</name>
> > <value>/mnt/hadoop/name</value>
> > </property>
> > <property>
> > <name>dfs.name.dir</name>
> > <value>/public/hadoop/name</value>
> > </property>
> >
> >
> > When I go to start DFS i'm getting the exception:
> >
> > org.apache.hadoop.hdfs.server.common.InconsistentFSStateException:
> Directory
> > /public/hadoop/name is in an inconsistent state: storage directory does
> not
> > exist or is not accessible.
> >
> >
> > After googling a bit, it seems like I want to do "bin/hadoop namenode
> > -format"
> >
> > Is this right? As long as I shut down DFS before issuing the command I
> > shouldn't lose any data?
> >
> > Thanks in advance,
> > Mike
> >
>
>
>
> --
> Harsh J
> www.harshj.com
>
Re: multiple namenode directories
Posted by Harsh J <qw...@gmail.com>.
DO NOT format your NameNode. Formatting a NameNode is equivalent to
formatting a FS -- you're bound lose it all.
And while messing with NameNode, after bringing it down safely, ALWAYS
take a backup of the existing dfs.name.dir contents and preferably the
SNN checkpoint directory contents too (if you're running it).
The RIGHT way to add new directories to the NameNode's dfs.name.dir is
by comma-separating them in the same value and NOT by adding two
properties - that is not how Hadoop's configuration operates. In your
case, bring NN down and edit conf as:
> <property>
> <name>dfs.name.dir</name>
> <value>/mnt/hadoop/name,/public/hadoop/name</value>
> </property>
Create the new directory by copying the existing one. Both must have
the SAME file and structure in them, like mirror copies of one
another. Ensure that this new location, apart from being symmetric in
content, is also symmetric in permissions. NameNode will require WRITE
permissions via its user on all locations configured.
Having configured properly and ensured that both storage directories
mirror one another, launch your NameNode back up again (feel a little
paranoid and do check namenode logs for any issues -- in which case
your backup would be very essential as a requirement for recovery!).
P.s. Hold on for a bit for a possible comment from another user before
getting into action. I've added extra directories this way, but I do
not know if this is "the" genuine way to do so - although it feels
right to me.
On Thu, Feb 10, 2011 at 10:27 PM, mike anderson <sa...@gmail.com> wrote:
> This should be a straightforward question, but better safe than sorry.
>
> I wanted to add a second name node directory (on an NFS as a backup), so now
> my hdfs-site.xml contains:
>
> <property>
> <name>dfs.name.dir</name>
> <value>/mnt/hadoop/name</value>
> </property>
> <property>
> <name>dfs.name.dir</name>
> <value>/public/hadoop/name</value>
> </property>
>
>
> When I go to start DFS i'm getting the exception:
>
> org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory
> /public/hadoop/name is in an inconsistent state: storage directory does not
> exist or is not accessible.
>
>
> After googling a bit, it seems like I want to do "bin/hadoop namenode
> -format"
>
> Is this right? As long as I shut down DFS before issuing the command I
> shouldn't lose any data?
>
> Thanks in advance,
> Mike
>
--
Harsh J
www.harshj.com
Fwd: multiple namenode directories
Posted by mike anderson <sa...@gmail.com>.
---------- Forwarded message ----------
From: mike anderson <sa...@gmail.com>
Date: Thu, Feb 10, 2011 at 11:57 AM
Subject: multiple namenode directories
To: core-user@hadoop.apache.org
This should be a straightforward question, but better safe than sorry.
I wanted to add a second name node directory (on an NFS as a backup), so now
my hdfs-site.xml contains:
<property>
<name>dfs.name.dir</name>
<value>/mnt/hadoop/name</value>
</property>
<property>
<name>dfs.name.dir</name>
<value>/public/hadoop/name</value>
</property>
When I go to start DFS i'm getting the exception:
org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory
/public/hadoop/name is in an inconsistent state: storage directory does not
exist or is not accessible.
After googling a bit, it seems like I want to do "bin/hadoop namenode
-format"
Is this right? As long as I shut down DFS before issuing the command I
shouldn't lose any data?
Thanks in advance,
Mike