You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by "Kengo Seki (JIRA)" <ji...@apache.org> on 2016/09/29 17:30:20 UTC
[jira] [Created] (BIGTOP-2544) provisioner/docker/docker-hadoop.sh
fails with "Could not find class node_with_components"
Kengo Seki created BIGTOP-2544:
----------------------------------
Summary: provisioner/docker/docker-hadoop.sh fails with "Could not find class node_with_components"
Key: BIGTOP-2544
URL: https://issues.apache.org/jira/browse/BIGTOP-2544
Project: Bigtop
Issue Type: Bug
Components: provisioner
Reporter: Kengo Seki
I'm not sure this is a docker-hadoop.sh's problem or my environmental/procedural one, since I'm not so familiar with docker and puppet. Sorry if I misunderstood something.
I tried provisioner/docker/docker-hadoop.sh, but encountered the following error.
{code}
sekikn@ubuntu:~/bigtop/provisioner/docker$ ./docker-hadoop.sh -c 1
Environment check...
Check docker:
Docker version 1.10.3, build 20f81dd
Check docker-compose:
docker-compose version 1.5.2, build unknown
Check ruby:
ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
Creating and starting 1 ... done
Bigtop Puppet configurations are shared between instances, and can be modified under config/hieradata
'../../bigtop-deploy/puppet/hieradata/site.yaml' -> './config/hieradata/site.yaml'
'../../bigtop-deploy/puppet/hieradata/bigtop' -> './config/hieradata/bigtop'
'../../bigtop-deploy/puppet/hieradata/bigtop/cluster.yaml' -> './config/hieradata/bigtop/cluster.yaml'
'../../bigtop-deploy/puppet/hieradata/bigtop/ha.yaml' -> './config/hieradata/bigtop/ha.yaml'
'../../bigtop-deploy/puppet/hieradata/bigtop/noha.yaml' -> './config/hieradata/bigtop/noha.yaml'
(snip)
local yum = false ; NOT Enabling local yum. Packages will be pulled from remote...
Error: Could not find class node_with_components for localhost on node localhost
Error: Could not find class node_with_components for localhost on node localhost
{code}
The following command in docker-hadoop.sh didn't seem work. Removing the trailing "/site.pp" made it work in my environment.
{code}
133 bigtop-puppet() {
134 docker exec $1 bash -c 'puppet apply --modulepath=/bigtop-home/bigtop-deploy/puppet/modules:/etc/puppet/modules /bigtop-home/bigtop-deploy/puppet/manifests/site.pp'
135 }
{code}
In addition, it didn't seem to set docker container's hostname properly, so {{docker-hadoop.sh -p}} failed as follows. Setting hostname manually fixed this.
{code}
sekikn@ubuntu:~/bigtop/provisioner/docker$ ./docker-hadoop.sh -p
WARNING: The DOCKER_IMAGE variable is not set. Defaulting to a blank string.
Notice: Compiled catalog for localhost in environment production in 1.85 seconds
Notice: /Stage[main]/Hadoop_cluster_node/User[testuser]/ensure: created
Notice: /Stage[main]/Hadoop_cluster_node/User[jenkins]/ensure: created
Notice: /Stage[main]/Hadoop::Mapred_app/Hadoop::Create_storage_dir[/data/2/mapred]/Exec[mkdir /data/2/mapred]/returns: executed successfully
Notice: /Stage[main]/Hadoop::Datanode/Hadoop::Create_storage_dir[/data/2/hdfs]/Exec[mkdir /data/2/hdfs]/returns: executed successfully
Notice: /Stage[main]/Hadoop::Mapred_app/Hadoop::Create_storage_dir[/data/1/mapred]/Exec[mkdir /data/1/mapred]/returns: executed successfully
Notice: /Stage[main]/Hadoop::Nodemanager/Hadoop::Create_storage_dir[/data/1/yarn]/Exec[mkdir /data/1/yarn]/returns: executed successfully
Notice: /Stage[main]/Hadoop_cluster_node/User[hudson]/ensure: created
Notice: /Stage[main]/Hadoop::Datanode/Hadoop::Create_storage_dir[/data/1/hdfs]/Exec[mkdir /data/1/hdfs]/returns: executed successfully
Notice: /Stage[main]/Hadoop::Nodemanager/Hadoop::Create_storage_dir[/data/2/yarn]/Exec[mkdir /data/2/yarn]/returns: executed successfully
Notice: /Stage[main]/Main/Yumrepo[Bigtop]/ensure: created
Notice: /Stage[main]/Main/Package[java-1.7.0-openjdk-devel.x86_64]/ensure: created
Notice: /Stage[main]/Hadoop::Datanode/Package[hadoop-hdfs-datanode]/ensure: created
Notice: /Stage[main]/Hadoop::Nodemanager/Package[hadoop-yarn-nodemanager]/ensure: created
Notice: /Stage[main]/Hadoop::Common_mapred_app/Package[hadoop-mapreduce]/ensure: created
Notice: /Stage[main]/Hadoop::Nodemanager/File[/data/2/yarn]/owner: owner changed 'root' to 'yarn'
Notice: /Stage[main]/Hadoop::Nodemanager/File[/data/2/yarn]/group: group changed 'root' to 'yarn'
Notice: /Stage[main]/Hadoop::Common/File[/etc/hadoop/conf/hadoop-env.sh]/content: content changed '{md5}829beb2d3c16fa22040afdc0fa277ba1' to '{md5}b4e97a45010e60de2c7d4ca3abecfdde'
Notice: /Stage[main]/Hadoop::Mapred_app/File[/data/2/mapred]/owner: owner changed 'root' to 'yarn'
Notice: /Stage[main]/Hadoop::Mapred_app/File[/data/2/mapred]/group: group changed 'root' to 'yarn'
Notice: /Stage[main]/Hadoop::Common_hdfs/File[/etc/hadoop/conf/core-site.xml]/content: content changed '{md5}315d2770c60bc9a0b3c7dc04e41fda76' to '{md5}ed179cad06cca9faad64e6fff87201cb'
Notice: /Stage[main]/Hadoop::Common_mapred_app/File[/etc/hadoop/conf/taskcontroller.cfg]/ensure: defined content as '{md5}fa9d342175e04ccf0fa1aba8291bedba'
Notice: /Stage[main]/Hadoop::Datanode/File[/data/1/hdfs]/owner: owner changed 'root' to 'hdfs'
Notice: /Stage[main]/Hadoop::Datanode/File[/data/1/hdfs]/group: group changed 'root' to 'hdfs'
Notice: /Stage[main]/Hadoop::Common_yarn/File[/etc/hadoop/conf/container-executor.cfg]/content: content changed '{md5}8afd041c79a90945ebfdd10ccbc43d9d' to '{md5}6aa9a3efa8965dfee06966b8fcb22e19'
Notice: /Stage[main]/Hadoop::Common_hdfs/File[/etc/hadoop/conf/hdfs-site.xml]/content: content changed '{md5}06f0cc5cad0018c16ef3dcb404efb74e' to '{md5}d6cd9cdc2e82f58ce240bfd43d89c8c6'
Notice: /Stage[main]/Hadoop::Datanode/File[/data/2/hdfs]/owner: owner changed 'root' to 'hdfs'
Notice: /Stage[main]/Hadoop::Datanode/File[/data/2/hdfs]/group: group changed 'root' to 'hdfs'
Notice: /Stage[main]/Hadoop::Common_mapred_app/File[/etc/hadoop/conf/mapred-site.xml]/content: content changed '{md5}1b058ae210b2e2dafd3407b6bf7dad43' to '{md5}e1555af50524c355a1e375d476057f01'
Notice: /Stage[main]/Hadoop::Nodemanager/File[/data/1/yarn]/owner: owner changed 'root' to 'yarn'
Notice: /Stage[main]/Hadoop::Nodemanager/File[/data/1/yarn]/group: group changed 'root' to 'yarn'
Notice: /Stage[main]/Hadoop::Common_yarn/File[/etc/hadoop/conf/yarn-site.xml]/content: content changed '{md5}f679a7750b76612f7468f61a7c6b75e7' to '{md5}67b5aa29694448b22408de7e9fe178fa'
Error: Could not start Service[hadoop-yarn-nodemanager]: Execution of '/sbin/service hadoop-yarn-nodemanager start' returned 1: starting nodemanager, logging to /var/log/hadoop-yarn/yarn-yarn-nodemanager-localhost.localdomain.out
Failed to start Hadoop nodemanager. Return value: 1[FAILED]
Error: /Stage[main]/Hadoop::Nodemanager/Service[hadoop-yarn-nodemanager]/ensure: change from stopped to running failed: Could not start Service[hadoop-yarn-nodemanager]: Execution of '/sbin/service hadoop-yarn-nodemanager start' returned 1: starting nodemanager, logging to /var/log/hadoop-yarn/yarn-yarn-nodemanager-localhost.localdomain.out
Failed to start Hadoop nodemanager. Return value: 1[FAILED]
Notice: /Stage[main]/Hadoop::Nodemanager/Service[hadoop-yarn-nodemanager]: Triggered 'refresh' from 4 events
Notice: /Stage[main]/Hadoop::Datanode/File[/etc/default/hadoop-hdfs-datanode]/content: content changed '{md5}0c3a4aff527c8fb16d826bf59c9bb985' to '{md5}8aaf7169ae9f4cf6863f2e8c96e8ecb8'
Error: Could not start Service[hadoop-hdfs-datanode]: Execution of '/sbin/service hadoop-hdfs-datanode start' returned 1: starting datanode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-datanode-localhost.localdomain.out
Failed to start Hadoop datanode. Return value: 1[FAILED]
Error: /Stage[main]/Hadoop::Datanode/Service[hadoop-hdfs-datanode]/ensure: change from stopped to running failed: Could not start Service[hadoop-hdfs-datanode]: Execution of '/sbin/service hadoop-hdfs-datanode start' returned 1: starting datanode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-datanode-localhost.localdomain.out
Failed to start Hadoop datanode. Return value: 1[FAILED]
{code}
My environment is as follows:
{code}
sekikn@ubuntu:~/bigtop/provisioner/docker$ docker version
Client:
Version: 1.10.3
API version: 1.22
Go version: go1.6.1
Git commit: 20f81dd
Built: Wed, 20 Apr 2016 14:19:16 -0700
OS/Arch: linux/amd64
Server:
Version: 1.10.3
API version: 1.22
Go version: go1.6.1
Git commit: 20f81dd
Built: Wed, 20 Apr 2016 14:19:16 -0700
OS/Arch: linux/amd64
sekikn@ubuntu:~/bigtop/provisioner/docker$ docker-compose version
docker-compose version 1.5.2, build unknown
docker-py version: 1.8.0
CPython version: 2.7.12
OpenSSL version: OpenSSL 1.0.2g 1 Mar 2016
sekikn@ubuntu:~/bigtop/provisioner/docker$ puppet -V
3.8.5
sekikn@ubuntu:~/bigtop/provisioner/docker$ cat /etc/issue
Ubuntu 16.04.1 LTS \n \l
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)