You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Alex Loddengaard <al...@google.com> on 2008/08/26 11:46:52 UTC

NameNode formatting issues in 1.16.4 and higher

First post to the list; excited to be joining the community!

I've been following the getting started guide (<
http://wiki.apache.org/hadoop/GettingStartedWithHadoop>) in hopes of getting
just a single-node Hadoop instance up and running, and I've run into an
issue that I can't seem to find anywhere.

While attempting to format my NameNode for the first time, I'm getting the
following exception:

ERROR namenode.NameNode: java.io.IOException:
> javax.security.auth.login.LoginException: Login failed: id: cannot find name
> for group ID 5001
> id: cannot find name for group ID 5221
>
>         at
> org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:250)
>         at
> org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:275)
>         at
> org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:257)
>         at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setConfigurationParameters(FSNamesystem.java:405)
>         at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:394)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:759)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:841)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:858)
>
>         at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setConfigurationParameters(FSNamesystem.java:407)
>         at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:394)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:759)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:841)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:858)
>

Firstly, I'm not familiar with LoginException at all, but after Googling for
a while, most of the issues came about by errors with /etc/groups and
permissions; I couldn't find a solution, though.  What's most curious is
that this issue doesn't come up with 0.15.2, but it does come up with every
later version, including 0.18.0 and trunk.

Anyone have any ideas?  In the meantime I'm going to try to dig in to
org.apache.hadoop.security.UnixUserGroupInformation along with best
practices for using LoginException.  I'm also going to try and create a
hadoop user and group and run the NameNode format process through that user.

Thanks ahead of time for pulling me out of the trenches.

Alex

Re: NameNode formatting issues in 1.16.4 and higher

Posted by Alex Loddengaard <al...@google.com>.
Thanks for your help, Arijit.  After an entire day, I'm up and running!
Yippee!

I had some strangeness going on with LDAP that made Hadoop not able to find
some groups that I'm in.  Solution: got rid of LDAP :).

Thanks again!

Alex

On Tue, Aug 26, 2008 at 5:55 PM, Arijit Mukherjee <
armukherjee@connectivasystems.com> wrote:

> Hi
>
> Most likely, it's due to login permissions. Have you set up ssh for
> accessing the nodes? This page might be helpful -
> http://tinyurl.com/6lz6o3 - contains detailed explanation of the steps
> you should follow.
>
> Hope this helps
>
> Cheers
> Arijit
>
> Dr. Arijit Mukherjee
> Principal Member of Technical Staff, Level-II
> Connectiva Systems (I) Pvt. Ltd.
> J-2, Block GP, Sector V, Salt Lake
> Kolkata 700 091, India
> Phone: +91 (0)33 23577531/32 x 107
> http://www.connectivasystems.com
>
>
> -----Original Message-----
> From: Alex Loddengaard [mailto:alexlod@google.com]
> Sent: Tuesday, August 26, 2008 3:17 PM
> To: core-user@hadoop.apache.org
> Subject: NameNode formatting issues in 1.16.4 and higher
>
>
> First post to the list; excited to be joining the community!
>
> I've been following the getting started guide (<
> http://wiki.apache.org/hadoop/GettingStartedWithHadoop>) in hopes of
> getting just a single-node Hadoop instance up and running, and I've run
> into an issue that I can't seem to find anywhere.
>
> While attempting to format my NameNode for the first time, I'm getting
> the following exception:
>
> ERROR namenode.NameNode: java.io.IOException:
> > javax.security.auth.login.LoginException: Login failed: id: cannot
> > find name for group ID 5001
> > id: cannot find name for group ID 5221
> >
> >         at
> >
> org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupI
> nformation.java:250)
> >         at
> >
> org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupI
> nformation.java:275)
> >         at
> >
> org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupI
> nformation.java:257)
> >         at
> >
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setConfigurationPara
> meters(FSNamesystem.java:405)
> >         at
> >
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.
> java:394)
> >         at
> >
> org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:759
> )
> >         at
> >
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.
> java:841)
> >         at
> > org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:858
> > )
> >
> >         at
> >
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setConfigurationPara
> meters(FSNamesystem.java:407)
> >         at
> >
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.
> java:394)
> >         at
> >
> org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:759
> )
> >         at
> >
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.
> java:841)
> >         at
> > org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:858
> > )
> >
>
> Firstly, I'm not familiar with LoginException at all, but after Googling
> for a while, most of the issues came about by errors with /etc/groups
> and permissions; I couldn't find a solution, though.  What's most
> curious is that this issue doesn't come up with 0.15.2, but it does come
> up with every later version, including 0.18.0 and trunk.
>
> Anyone have any ideas?  In the meantime I'm going to try to dig in to
> org.apache.hadoop.security.UnixUserGroupInformation along with best
> practices for using LoginException.  I'm also going to try and create a
> hadoop user and group and run the NameNode format process through that
> user.
>
> Thanks ahead of time for pulling me out of the trenches.
>
> Alex
> No virus found in this incoming message.
> Checked by AVG.
> Version: 8.0.100 / Virus Database: 270.6.9/1634 - Release Date:
> 8/25/2008 8:48 PM
>
>
>

RE: NameNode formatting issues in 1.16.4 and higher

Posted by Arijit Mukherjee <ar...@connectivasystems.com>.
Hi

Most likely, it's due to login permissions. Have you set up ssh for
accessing the nodes? This page might be helpful -
http://tinyurl.com/6lz6o3 - contains detailed explanation of the steps
you should follow.

Hope this helps

Cheers
Arijit

Dr. Arijit Mukherjee
Principal Member of Technical Staff, Level-II
Connectiva Systems (I) Pvt. Ltd.
J-2, Block GP, Sector V, Salt Lake
Kolkata 700 091, India
Phone: +91 (0)33 23577531/32 x 107
http://www.connectivasystems.com


-----Original Message-----
From: Alex Loddengaard [mailto:alexlod@google.com] 
Sent: Tuesday, August 26, 2008 3:17 PM
To: core-user@hadoop.apache.org
Subject: NameNode formatting issues in 1.16.4 and higher


First post to the list; excited to be joining the community!

I've been following the getting started guide (<
http://wiki.apache.org/hadoop/GettingStartedWithHadoop>) in hopes of
getting just a single-node Hadoop instance up and running, and I've run
into an issue that I can't seem to find anywhere.

While attempting to format my NameNode for the first time, I'm getting
the following exception:

ERROR namenode.NameNode: java.io.IOException:
> javax.security.auth.login.LoginException: Login failed: id: cannot 
> find name for group ID 5001
> id: cannot find name for group ID 5221
>
>         at
>
org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupI
nformation.java:250)
>         at
>
org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupI
nformation.java:275)
>         at
>
org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupI
nformation.java:257)
>         at
>
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setConfigurationPara
meters(FSNamesystem.java:405)
>         at
>
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.
java:394)
>         at
>
org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:759
)
>         at
>
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.
java:841)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:858
> )
>
>         at
>
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.setConfigurationPara
meters(FSNamesystem.java:407)
>         at
>
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.
java:394)
>         at
>
org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:759
)
>         at
>
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.
java:841)
>         at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:858
> )
>

Firstly, I'm not familiar with LoginException at all, but after Googling
for a while, most of the issues came about by errors with /etc/groups
and permissions; I couldn't find a solution, though.  What's most
curious is that this issue doesn't come up with 0.15.2, but it does come
up with every later version, including 0.18.0 and trunk.

Anyone have any ideas?  In the meantime I'm going to try to dig in to
org.apache.hadoop.security.UnixUserGroupInformation along with best
practices for using LoginException.  I'm also going to try and create a
hadoop user and group and run the NameNode format process through that
user.

Thanks ahead of time for pulling me out of the trenches.

Alex
No virus found in this incoming message.
Checked by AVG. 
Version: 8.0.100 / Virus Database: 270.6.9/1634 - Release Date:
8/25/2008 8:48 PM