You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bigtop.apache.org by "Konstantinos Tsakalozos (JIRA)" <ji...@apache.org> on 2016/06/03 09:29:59 UTC
[jira] [Comment Edited] (BIGTOP-2466) HBase puppet scripts
co-deploy ZK and Datanode along with HBase
[ https://issues.apache.org/jira/browse/BIGTOP-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15313896#comment-15313896 ]
Konstantinos Tsakalozos edited comment on BIGTOP-2466 at 6/3/16 9:29 AM:
-------------------------------------------------------------------------
For completeness let me show how deploying only hbase roles fails.
Starting with this site.yaml:
{noformat}
root@jackal-local-machine-4:/home/ubuntu/bigtop.release/bigtop-1.1.0# cat bigtop-deploy/puppet/hieradata/site.yaml
bigtop::bigtop_repo_uri: http://bigtop-repos.s3.amazonaws.com/releases/1.1.0/ubuntu/trusty/x86_64
bigtop::hadoop_head_node: jackal-local-machine-5
bigtop::jdk_preinstalled: true
bigtop::roles:
- hbase-client
- hbase-master
- hbase-server
bigtop::roles_enabled: true
hadoop::common_hdfs::hadoop_namenode_host: jackal-local-machine-5
hadoop::common_hdfs::namenode_datanode_registration_ip_hostname_check: true
hadoop::common_mapred_app::jobtracker_host: jackal-local-machine-6
hadoop::common_mapred_app::mapreduce_jobhistory_host: jackal-local-machine-6
hadoop::common_yarn::hadoop_ps_host: jackal-local-machine-6
hadoop::common_yarn::hadoop_rm_host: jackal-local-machine-6
hadoop::common_yarn::yarn_nodemanager_vmem_check_enabled: false
hadoop::hadoop_storage_dirs:
- /data/1
- /data/2
hadoop_hbase::common_config::zookeeper_quorum: 10.0.3.165
{noformat}
And doing a:
{noformat}
puppet apply -d --modulepath="bigtop-deploy/puppet/modules:/etc/puppet/modules" bigtop-deploy/puppet/manifests/site.pp
{noformat}
You get his error:
{noformat}
Error: Could not find dependency Package[hadoop-hdfs] for Exec[init hdfs] at /home/ubuntu/bigtop.release/bigtop-1.1.0/bigtop-deploy/puppet/modules/hadoop/manifests/init.pp:87
{noformat}
was (Author: ktsakalozos):
For completeness let me show how deploying only hbase roles fails.
Starting with this site.yaml:
root@jackal-local-machine-4:/home/ubuntu/bigtop.release/bigtop-1.1.0# cat bigtop-deploy/puppet/hieradata/site.yaml
bigtop::bigtop_repo_uri: http://bigtop-repos.s3.amazonaws.com/releases/1.1.0/ubuntu/trusty/x86_64
bigtop::hadoop_head_node: jackal-local-machine-5
bigtop::jdk_preinstalled: true
bigtop::roles:
- hbase-client
- hbase-master
- hbase-server
bigtop::roles_enabled: true
hadoop::common_hdfs::hadoop_namenode_host: jackal-local-machine-5
hadoop::common_hdfs::namenode_datanode_registration_ip_hostname_check: true
hadoop::common_mapred_app::jobtracker_host: jackal-local-machine-6
hadoop::common_mapred_app::mapreduce_jobhistory_host: jackal-local-machine-6
hadoop::common_yarn::hadoop_ps_host: jackal-local-machine-6
hadoop::common_yarn::hadoop_rm_host: jackal-local-machine-6
hadoop::common_yarn::yarn_nodemanager_vmem_check_enabled: false
hadoop::hadoop_storage_dirs:
- /data/1
- /data/2
hadoop_hbase::common_config::zookeeper_quorum: 10.0.3.165
And doing a:
puppet apply -d --modulepath="bigtop-deploy/puppet/modules:/etc/puppet/modules" bigtop-deploy/puppet/manifests/site.pp
You get his error:
Error: Could not find dependency Package[hadoop-hdfs] for Exec[init hdfs] at /home/ubuntu/bigtop.release/bigtop-1.1.0/bigtop-deploy/puppet/modules/hadoop/manifests/init.pp:87
> HBase puppet scripts co-deploy ZK and Datanode along with HBase
> ---------------------------------------------------------------
>
> Key: BIGTOP-2466
> URL: https://issues.apache.org/jira/browse/BIGTOP-2466
> Project: Bigtop
> Issue Type: Improvement
> Components: hbase
> Affects Versions: 1.1.0
> Reporter: Konstantinos Tsakalozos
> Priority: Minor
>
> Deploying the only HBase roles fails because it is expected to also co-deploy a datanode. Similarly, the puppet HBase scripts deploy a Zookeeper along with HBase.
> In deployments where ZK and HDFS are already provided on other nodes you end up needlessly deploying services along with HBase and waste resources.
> We should have a way to deploy only HBase (without auxiliary services) from the puppet scripts.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)