You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bigtop.apache.org by "Evans Ye (JIRA)" <ji...@apache.org> on 2016/09/29 18:04:21 UTC
[jira] [Commented] (BIGTOP-2544)
provisioner/docker/docker-hadoop.sh fails with "Could not find class
node_with_components"
[ https://issues.apache.org/jira/browse/BIGTOP-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15533539#comment-15533539 ]
Evans Ye commented on BIGTOP-2544:
----------------------------------
Yup I can recall I've encountered same issue as well.
And I remember when I switch the OS to centos 7, everything is fine, perhaps?
> provisioner/docker/docker-hadoop.sh fails with "Could not find class node_with_components"
> ------------------------------------------------------------------------------------------
>
> Key: BIGTOP-2544
> URL: https://issues.apache.org/jira/browse/BIGTOP-2544
> Project: Bigtop
> Issue Type: Bug
> Components: provisioner
> Reporter: Kengo Seki
>
> I'm not sure this is a docker-hadoop.sh's problem or my environmental/procedural one, since I'm not so familiar with docker and puppet. Sorry if I misunderstood something.
> I tried provisioner/docker/docker-hadoop.sh, but encountered the following error.
> {code}
> sekikn@ubuntu:~/bigtop/provisioner/docker$ ./docker-hadoop.sh -c 1
> Environment check...
> Check docker:
> Docker version 1.10.3, build 20f81dd
> Check docker-compose:
> docker-compose version 1.5.2, build unknown
> Check ruby:
> ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
> Creating and starting 1 ... done
> Bigtop Puppet configurations are shared between instances, and can be modified under config/hieradata
> '../../bigtop-deploy/puppet/hieradata/site.yaml' -> './config/hieradata/site.yaml'
> '../../bigtop-deploy/puppet/hieradata/bigtop' -> './config/hieradata/bigtop'
> '../../bigtop-deploy/puppet/hieradata/bigtop/cluster.yaml' -> './config/hieradata/bigtop/cluster.yaml'
> '../../bigtop-deploy/puppet/hieradata/bigtop/ha.yaml' -> './config/hieradata/bigtop/ha.yaml'
> '../../bigtop-deploy/puppet/hieradata/bigtop/noha.yaml' -> './config/hieradata/bigtop/noha.yaml'
> (snip)
> local yum = false ; NOT Enabling local yum. Packages will be pulled from remote...
> Error: Could not find class node_with_components for localhost on node localhost
> Error: Could not find class node_with_components for localhost on node localhost
> {code}
> The following command in docker-hadoop.sh didn't seem work. Removing the trailing "/site.pp" made it work in my environment.
> {code}
> 133 bigtop-puppet() {
> 134 docker exec $1 bash -c 'puppet apply --modulepath=/bigtop-home/bigtop-deploy/puppet/modules:/etc/puppet/modules /bigtop-home/bigtop-deploy/puppet/manifests/site.pp'
> 135 }
> {code}
> In addition, it didn't seem to set docker container's hostname properly, so {{docker-hadoop.sh -p}} failed as follows. Setting hostname manually fixed this.
> {code}
> sekikn@ubuntu:~/bigtop/provisioner/docker$ ./docker-hadoop.sh -p
> WARNING: The DOCKER_IMAGE variable is not set. Defaulting to a blank string.
> Notice: Compiled catalog for localhost in environment production in 1.85 seconds
> Notice: /Stage[main]/Hadoop_cluster_node/User[testuser]/ensure: created
> Notice: /Stage[main]/Hadoop_cluster_node/User[jenkins]/ensure: created
> Notice: /Stage[main]/Hadoop::Mapred_app/Hadoop::Create_storage_dir[/data/2/mapred]/Exec[mkdir /data/2/mapred]/returns: executed successfully
> Notice: /Stage[main]/Hadoop::Datanode/Hadoop::Create_storage_dir[/data/2/hdfs]/Exec[mkdir /data/2/hdfs]/returns: executed successfully
> Notice: /Stage[main]/Hadoop::Mapred_app/Hadoop::Create_storage_dir[/data/1/mapred]/Exec[mkdir /data/1/mapred]/returns: executed successfully
> Notice: /Stage[main]/Hadoop::Nodemanager/Hadoop::Create_storage_dir[/data/1/yarn]/Exec[mkdir /data/1/yarn]/returns: executed successfully
> Notice: /Stage[main]/Hadoop_cluster_node/User[hudson]/ensure: created
> Notice: /Stage[main]/Hadoop::Datanode/Hadoop::Create_storage_dir[/data/1/hdfs]/Exec[mkdir /data/1/hdfs]/returns: executed successfully
> Notice: /Stage[main]/Hadoop::Nodemanager/Hadoop::Create_storage_dir[/data/2/yarn]/Exec[mkdir /data/2/yarn]/returns: executed successfully
> Notice: /Stage[main]/Main/Yumrepo[Bigtop]/ensure: created
> Notice: /Stage[main]/Main/Package[java-1.7.0-openjdk-devel.x86_64]/ensure: created
> Notice: /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]/ensure: created
> Notice: /Stage[main]/Hadoop::Nodemanager/Package[hadoop-yarn-nodemanager]/ensure: created
> Notice: /Stage[main]/Hadoop::Common_mapred_app/Package[hadoop-mapreduce]/ensure: created
> Notice: /Stage[main]/Hadoop::Nodemanager/File[/data/2/yarn]/owner: owner changed 'root' to 'yarn'
> Notice: /Stage[main]/Hadoop::Nodemanager/File[/data/2/yarn]/group: group changed 'root' to 'yarn'
> Notice: /Stage[main]/Hadoop::Common/File[/etc/hadoop/conf/hadoop-env.sh]/content: content changed '{md5}829beb2d3c16fa22040afdc0fa277ba1' to '{md5}b4e97a45010e60de2c7d4ca3abecfdde'
> Notice: /Stage[main]/Hadoop::Mapred_app/File[/data/2/mapred]/owner: owner changed 'root' to 'yarn'
> Notice: /Stage[main]/Hadoop::Mapred_app/File[/data/2/mapred]/group: group changed 'root' to 'yarn'
> Notice: /Stage[main]/Hadoop::Common_hdfs/File[/etc/hadoop/conf/core-site.xml]/content: content changed '{md5}315d2770c60bc9a0b3c7dc04e41fda76' to '{md5}ed179cad06cca9faad64e6fff87201cb'
> Notice: /Stage[main]/Hadoop::Common_mapred_app/File[/etc/hadoop/conf/taskcontroller.cfg]/ensure: defined content as '{md5}fa9d342175e04ccf0fa1aba8291bedba'
> Notice: /Stage[main]/Hadoop::Datanode/File[/data/1/hdfs]/owner: owner changed 'root' to 'hdfs'
> Notice: /Stage[main]/Hadoop::Datanode/File[/data/1/hdfs]/group: group changed 'root' to 'hdfs'
> Notice: /Stage[main]/Hadoop::Common_yarn/File[/etc/hadoop/conf/container-executor.cfg]/content: content changed '{md5}8afd041c79a90945ebfdd10ccbc43d9d' to '{md5}6aa9a3efa8965dfee06966b8fcb22e19'
> Notice: /Stage[main]/Hadoop::Common_hdfs/File[/etc/hadoop/conf/hdfs-site.xml]/content: content changed '{md5}06f0cc5cad0018c16ef3dcb404efb74e' to '{md5}d6cd9cdc2e82f58ce240bfd43d89c8c6'
> Notice: /Stage[main]/Hadoop::Datanode/File[/data/2/hdfs]/owner: owner changed 'root' to 'hdfs'
> Notice: /Stage[main]/Hadoop::Datanode/File[/data/2/hdfs]/group: group changed 'root' to 'hdfs'
> Notice: /Stage[main]/Hadoop::Common_mapred_app/File[/etc/hadoop/conf/mapred-site.xml]/content: content changed '{md5}1b058ae210b2e2dafd3407b6bf7dad43' to '{md5}e1555af50524c355a1e375d476057f01'
> Notice: /Stage[main]/Hadoop::Nodemanager/File[/data/1/yarn]/owner: owner changed 'root' to 'yarn'
> Notice: /Stage[main]/Hadoop::Nodemanager/File[/data/1/yarn]/group: group changed 'root' to 'yarn'
> Notice: /Stage[main]/Hadoop::Common_yarn/File[/etc/hadoop/conf/yarn-site.xml]/content: content changed '{md5}f679a7750b76612f7468f61a7c6b75e7' to '{md5}67b5aa29694448b22408de7e9fe178fa'
> Error: Could not start Service[hadoop-yarn-nodemanager]: Execution of '/sbin/service hadoop-yarn-nodemanager start' returned 1: starting nodemanager, logging to /var/log/hadoop-yarn/yarn-yarn-nodemanager-localhost.localdomain.out
> Failed to start Hadoop nodemanager. Return value: 1[FAILED]
> Error: /Stage[main]/Hadoop::Nodemanager/Service[hadoop-yarn-nodemanager]/ensure: change from stopped to running failed: Could not start Service[hadoop-yarn-nodemanager]: Execution of '/sbin/service hadoop-yarn-nodemanager start' returned 1: starting nodemanager, logging to /var/log/hadoop-yarn/yarn-yarn-nodemanager-localhost.localdomain.out
> Failed to start Hadoop nodemanager. Return value: 1[FAILED]
> Notice: /Stage[main]/Hadoop::Nodemanager/Service[hadoop-yarn-nodemanager]: Triggered 'refresh' from 4 events
> Notice: /Stage[main]/Hadoop::Datanode/File[/etc/default/hadoop-hdfs-datanode]/content: content changed '{md5}0c3a4aff527c8fb16d826bf59c9bb985' to '{md5}8aaf7169ae9f4cf6863f2e8c96e8ecb8'
> Error: Could not start Service[hadoop-hdfs-datanode]: Execution of '/sbin/service hadoop-hdfs-datanode start' returned 1: starting datanode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-datanode-localhost.localdomain.out
> Failed to start Hadoop datanode. Return value: 1[FAILED]
> Error: /Stage[main]/Hadoop::Datanode/Service[hadoop-hdfs-datanode]/ensure: change from stopped to running failed: Could not start Service[hadoop-hdfs-datanode]: Execution of '/sbin/service hadoop-hdfs-datanode start' returned 1: starting datanode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-datanode-localhost.localdomain.out
> Failed to start Hadoop datanode. Return value: 1[FAILED]
> {code}
> My environment is as follows:
> {code}
> sekikn@ubuntu:~/bigtop/provisioner/docker$ docker version
> Client:
> Version: 1.10.3
> API version: 1.22
> Go version: go1.6.1
> Git commit: 20f81dd
> Built: Wed, 20 Apr 2016 14:19:16 -0700
> OS/Arch: linux/amd64
> Server:
> Version: 1.10.3
> API version: 1.22
> Go version: go1.6.1
> Git commit: 20f81dd
> Built: Wed, 20 Apr 2016 14:19:16 -0700
> OS/Arch: linux/amd64
> sekikn@ubuntu:~/bigtop/provisioner/docker$ docker-compose version
> docker-compose version 1.5.2, build unknown
> docker-py version: 1.8.0
> CPython version: 2.7.12
> OpenSSL version: OpenSSL 1.0.2g 1 Mar 2016
> sekikn@ubuntu:~/bigtop/provisioner/docker$ puppet -V
> 3.8.5
> sekikn@ubuntu:~/bigtop/provisioner/docker$ cat /etc/issue
> Ubuntu 16.04.1 LTS \n \l
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)