[jira] [Comment Edited] (BIGTOP-1336) Puppet recipes failed to deploy kerberos enabled hadoop cluster

["Evans Ye (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BIGTOP-1336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14040038#comment-14040038 ] 

Evans Ye edited comment on BIGTOP-1336 at 6/22/14 5:43 AM:
-----------------------------------------------------------

[~jayunit100], Yes you're right for the namenode part.
And there's another part regarding to datanode in this patch, if we do not setup {{/etc/default/hadoop-hdfs-datanode}} before datanode started, following FATAL error will show in datanode's log:
{noformat}
2014-06-16 15:10:10,711 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain
java.lang.RuntimeException: Cannot start secure cluster without privileged resources.
{noformat}

Overall, this patch is mainly addressing issues to bring a kerberos hadoop cluster up.


was (Author: evans_ye):
[~jayunit100], Yes you're right for the namenode part.
And there's another part regarding to datanode in this patch, if we do not setup {{/etc/default/hadoop-hdfs-datanode}} before datanode started, following FATAL error will show:
{noformat}
2014-06-16 15:10:10,711 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain
java.lang.RuntimeException: Cannot start secure cluster without privileged resources.
{noformat}

Overall, this patch is mainly addressing issues to bring a kerberos hadoop cluster up.

> Puppet recipes failed to deploy kerberos enabled hadoop cluster
> ---------------------------------------------------------------
>
>                 Key: BIGTOP-1336
>                 URL: https://issues.apache.org/jira/browse/BIGTOP-1336
>             Project: Bigtop
>          Issue Type: Bug
>          Components: Deployment
>    Affects Versions: 0.7.0
>            Reporter: Evans Ye
>             Fix For: 0.8.0
>
>         Attachments: BIGTOP-1336.1.patch
>
>
> Here are some missing dependency setting in our puppet recipes in order to get kerberos enabled on the hadoop cluster.
> The first one is that kerberos principal for hdfs user hasn't been created before formatting namenode, which cause the namenode formatting process failed.
> The second one is that {{/etc/default/hadoop-hdfs-datanode}} doesn't get ready before starting up datanodes and results in datanodes failed to startup.
> The datanode error log:
> {noformat}
> 2014-06-16 15:10:10,711 FATAL org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain
> java.lang.RuntimeException: Cannot start secure cluster without privileged resources.
> {noformat}
> Here's the reproduce steps using [vagrant-puppet|https://github.com/apache/bigtop/tree/master/bigtop-deploy/vm/vagrant-puppet]:
> 1.) Enable kerberos on the hadoop cluster.
> {noformat}
> $ vim bigtop-deploy/vm/vagrant-puppet/provision.sh
> {noformat}
> Add kerberos definitions.
> {noformat}
> cat > /bigtop-puppet/config/site.csv << EOF
> hadoop_head_node,$1
> hadoop_storage_dirs,/data/1,/data/2
> bigtop_yumrepo_uri,http://bigtop.s3.amazonaws.com/releases/0.7.0/redhat/6/x86_64
> jdk_package_name,java-1.7.0-openjdk-devel.x86_64
> components,hadoop,hbase,yarn,mapred-app
> hadoop_security,kerberos
> hadoop_kerberos_domain,vagrant
> hadoop_kerberos_realm,BIGTOP.ORG
> hadoop_kerberos_kdc_server,bigtop1.vagrant
> EOF
> {noformat}
> 2.) Spin up the cluster.
> {code}
> $ ./startup.sh --cluster
> {code}
> 3-1.) Get an error while formating namenode.
> {noformat}
> err: /Stage[main]/Hadoop_head_node/Hadoop::Namenode[namenode]/Exec[namenode format]/returns: change from notrun to 0 failed: /bin/bash -c 'yes Y | hdfs namenode -format >> /var/lib/hadoop-hdfs/nn.format.log 2>&1' returned 1 instead of one of [0] at /tmp/vagrant-puppet-2/modules-0/hadoop/manifests/init.pp:361
> {noformat}
> 3-2.) Get an error while starting up datanodes.
> {noformat}
> err: /Stage[main]/Hadoop_worker_node/Hadoop::Datanode[datanode]/Service[hadoop-hdfs-datanode]/ensure: change from stopped to running failed: Could not start Service[hadoop-hdfs-datanode]: Execution of '/sbin/service hadoop-hdfs-datanode start' returned 1:  at /tmp/vagrant-puppet-2/modules-0/hadoop/manifests/init.pp:158
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)