You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Eric Schwartz <sc...@csail.mit.edu> on 2012/03/24 00:04:37 UTC
cannot start secure cluster without privileged resources
Howdy,
I'm working on getting a kerberized hadoop cluster up and running. I'm
using version 1.0.1, as packaged for 64-bit Debian Squeeze. I have the
namenode up and have successfully formatted HDFS. But when I tried to
bring up the first datanode, I ran into an error that I'm having
difficulty with, namely:
2012-03-23 17:49:55,291 INFO
org.apache.hadoop.security.UserGroupInformation: Login successful for
user hdfs/datanode0@REALM using keytab file /etc/hadoop/hdfs.keytab
2012-03-23 17:49:55,292 ERROR
org.apache.hadoop.hdfs.server.datanode.DataNode:
java.lang.RuntimeException: Cannot start secure cluster without
privileged resources.
at
org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:324)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:299)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1582)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1521)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1539)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1665)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1682)
Checking the mailing lists, I found that previously folks encountered
trouble with the privileged ports
(http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201108.mbox/%3CCAMs9kVi70RL=6yxuLsCovdD5gYsULfPWJonT2U2pd35duCoH_w@mail.gmail.com%3E)
and with a bug in the /usr/bin/hadoop script.
(https://issues.apache.org/jira/browse/HADOOP-8132) Neither of these
seem to be the culprit here, as I'm root when I try to start the
datanode and there's nothing already using ports 1004 and 1006, and the
patch already seemed to be applied to my executable. (I ran it through
patch just for good measure and confirmed that the trouble remained.)
I've traced the exception stack through the source to little avail (as
my Java skills are limited) and don't particularly relish the prospect
of stracing here. That error message is awfully vague; are there some
debugging knobs I can turn up somewhere?
Thanks in advance for any help. This is my first post here -- please let
me know if I've made a mistake and should have posted this elsewhere.
Cheers,
Eric Schwartz
Re: cannot start secure cluster without privileged resources
Posted by Eric Schwartz <sc...@csail.mit.edu>.
Yes, there's nothing filtering packets between systems in the cluster.
I've verified with tcpdump what network traffic is transpiring: just the
kerberos login, and nothing else.
Thanks,
Eric Schwartz
On Fri, 23 Mar 2012, Mapred Learn wrote:
> Do you have these ports open amongst the datanodes and name node ?
>
> Sent from my iPhone
>
> On Mar 23, 2012, at 4:04 PM, Eric Schwartz <sc...@csail.mit.edu> wrote:
>
>> Howdy,
>>
>> I'm working on getting a kerberized hadoop cluster up and running. I'm using version 1.0.1, as packaged for 64-bit Debian Squeeze. I have the namenode up and have successfully formatted HDFS. But when I tried to bring up the first datanode, I ran into an error that I'm having difficulty with, namely:
>>
>> 2012-03-23 17:49:55,291 INFO org.apache.hadoop.security.UserGroupInformation: Login successful for user hdfs/datanode0@REALM using keytab file /etc/hadoop/hdfs.keytab
>> 2012-03-23 17:49:55,292 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.lang.RuntimeException: Cannot start secure cluster without privileged resources.
>> at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:324)
>> at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:299)
>> at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1582)
>> at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1521)
>> at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1539)
>> at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1665)
>> at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1682)
>>
>> Checking the mailing lists, I found that previously folks encountered trouble with the privileged ports (http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201108.mbox/%3CCAMs9kVi70RL=6yxuLsCovdD5gYsULfPWJonT2U2pd35duCoH_w@mail.gmail.com%3E) and with a bug in the /usr/bin/hadoop script. (https://issues.apache.org/jira/browse/HADOOP-8132) Neither of these seem to be the culprit here, as I'm root when I try to start the datanode and there's nothing already using ports 1004 and 1006, and the patch already seemed to be applied to my executable. (I ran it through patch just for good measure and confirmed that the trouble remained.)
>>
>> I've traced the exception stack through the source to little avail (as my Java skills are limited) and don't particularly relish the prospect of stracing here. That error message is awfully vague; are there some debugging knobs I can turn up somewhere?
>>
>> Thanks in advance for any help. This is my first post here -- please let me know if I've made a mistake and should have posted this elsewhere.
>>
>> Cheers,
>> Eric Schwartz
>
Re: cannot start secure cluster without privileged resources
Posted by Mapred Learn <ma...@gmail.com>.
Do you have these ports open amongst the datanodes and name node ?
Sent from my iPhone
On Mar 23, 2012, at 4:04 PM, Eric Schwartz <sc...@csail.mit.edu> wrote:
> Howdy,
>
> I'm working on getting a kerberized hadoop cluster up and running. I'm using version 1.0.1, as packaged for 64-bit Debian Squeeze. I have the namenode up and have successfully formatted HDFS. But when I tried to bring up the first datanode, I ran into an error that I'm having difficulty with, namely:
>
> 2012-03-23 17:49:55,291 INFO org.apache.hadoop.security.UserGroupInformation: Login successful for user hdfs/datanode0@REALM using keytab file /etc/hadoop/hdfs.keytab
> 2012-03-23 17:49:55,292 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.lang.RuntimeException: Cannot start secure cluster without privileged resources.
> at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:324)
> at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:299)
> at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1582)
> at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1521)
> at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1539)
> at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1665)
> at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1682)
>
> Checking the mailing lists, I found that previously folks encountered trouble with the privileged ports (http://mail-archives.apache.org/mod_mbox/hadoop-common-user/201108.mbox/%3CCAMs9kVi70RL=6yxuLsCovdD5gYsULfPWJonT2U2pd35duCoH_w@mail.gmail.com%3E) and with a bug in the /usr/bin/hadoop script. (https://issues.apache.org/jira/browse/HADOOP-8132) Neither of these seem to be the culprit here, as I'm root when I try to start the datanode and there's nothing already using ports 1004 and 1006, and the patch already seemed to be applied to my executable. (I ran it through patch just for good measure and confirmed that the trouble remained.)
>
> I've traced the exception stack through the source to little avail (as my Java skills are limited) and don't particularly relish the prospect of stracing here. That error message is awfully vague; are there some debugging knobs I can turn up somewhere?
>
> Thanks in advance for any help. This is my first post here -- please let me know if I've made a mistake and should have posted this elsewhere.
>
> Cheers,
> Eric Schwartz