You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ambari.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2017/08/02 13:52:00 UTC
[jira] [Commented] (AMBARI-21581) Replace Hard Coded conf-select
Structures
[ https://issues.apache.org/jira/browse/AMBARI-21581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16110941#comment-16110941 ]
Hudson commented on AMBARI-21581:
---------------------------------
SUCCESS: Integrated in Jenkins build Ambari-trunk-Commit #7835 (See [https://builds.apache.org/job/Ambari-trunk-Commit/7835/])
AMBARI-21581 - Replace Hard Coded conf-select Structures (jhurley: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=97ccf3bff96f8717c185776c7c000496df90daa0])
* (edit) ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py
* (edit) ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/package/scripts/zookeeper_client.py
* (edit) ambari-server/src/main/resources/common-services/SPARK/2.2.0/package/scripts/spark_client.py
* (edit) ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/accumulo_client.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/namenode.py
* (delete) ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/upgrade.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/datanode.py
* (edit) ambari-server/src/main/resources/custom_actions/scripts/install_packages.py
* (edit) ambari-server/src/test/python/stacks/utils/RMFTestCase.py
* (edit) ambari-server/src/test/python/stacks/2.1/STORM/test_storm_supervisor_prod.py
* (edit) ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_client.py
* (edit) ambari-server/src/main/resources/common-services/RANGER/1.0.0.3.0/package/scripts/ranger_tagsync.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_regionserver.py
* (edit) ambari-server/src/main/resources/common-services/FLUME/1.4.0.2.0/package/scripts/flume_handler.py
* (edit) ambari-server/src/main/resources/common-services/SPARK/2.2.0/package/scripts/spark_thrift_server.py
* (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/yarn_client.py
* (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/supervisor_prod.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_client.py
* (edit) ambari-server/src/test/python/custom_actions/test_ru_set_all.py
* (edit) ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/spark_thrift_server.py
* (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/nodemanager.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_client.py
* (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/application_timeline_server.py
* (edit) ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java
* (edit) ambari-server/src/main/resources/common-services/TEZ/0.9.0.3.0/package/scripts/tez_client.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HDFS/test_hdfs_client.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/journalnode.py
* (edit) ambari-server/src/test/python/stacks/2.1/STORM/test_storm_nimbus_prod.py
* (edit) ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py
* (edit) ambari-server/src/main/resources/common-services/PIG/0.16.1.3.0/package/scripts/pig_client.py
* (add) ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_packages.json
* (edit) ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py
* (edit) ambari-server/src/test/python/stacks/2.1/HIVE/test_hive_metastore.py
* (edit) ambari-server/src/test/python/stacks/2.2/SPARK/test_job_history_server.py
* (edit) ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/superset.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py
* (edit) ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop_client.py
* (edit) ambari-server/src/test/python/stacks/2.3/MAHOUT/test_mahout_client.py
* (edit) ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py
* (edit) ambari-server/src/main/resources/common-services/OOZIE/4.2.0.3.0/package/scripts/oozie_client.py
* (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/nimbus.py
* (edit) ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py
* (add) ambari-server/src/main/resources/stacks/HDP/3.0/properties/stack_packages.json
* (edit) ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/SQOOP/test_sqoop.py
* (edit) ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/mapreduce2_client.py
* (edit) ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py
* (edit) ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_client.py
* (edit) ambari-server/src/main/resources/common-services/KAFKA/0.10.0.3.0/package/scripts/kafka_broker.py
* (edit) ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_thrift_server.py
* (edit) ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/webhcat_server.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_client.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/snamenode.py
* (edit) ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py
* (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/supervisor_prod.py
* (edit) ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_server.py
* (edit) ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn_client.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/hive_server.py
* (edit) ambari-server/src/test/python/stacks/2.2/RANGER/test_ranger_admin.py
* (edit) ambari-server/src/main/resources/common-services/OOZIE/4.2.0.3.0/package/scripts/oozie_server.py
* (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/supervisor.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HIVE/test_webhcat_server.py
* (edit) ambari-server/src/test/python/stacks/2.2/common/test_conf_select.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py
* (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/mapreduce2_client.py
* (edit) ambari-server/src/test/python/stacks/2.1/STORM/test_storm_supervisor.py
* (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/ui_server.py
* (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/historyserver.py
* (edit) ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
* (edit) ambari-server/src/main/java/org/apache/ambari/server/upgrade/FinalUpgradeCatalog.java
* (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_client.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HDFS/test_nfsgateway.py
* (edit) ambari-server/src/test/python/stacks/2.2/SPARK/test_spark_client.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
* (edit) ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/spark_client.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/hive_server_interactive.py
* (edit) ambari-server/src/main/resources/common-services/SQOOP/1.4.4.3.0/package/scripts/sqoop_client.py
* (edit) ambari-server/src/test/python/stacks/2.1/STORM/test_storm_ui_server.py
* (edit) ambari-server/src/test/python/stacks/2.3/SPARK/test_spark_thrift_server.py
* (edit) ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/pig_client.py
* (edit) ambari-server/src/test/python/stacks/2.1/YARN/test_apptimelineserver.py
* (edit) ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/ranger_tagsync.py
* (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/drpc_server.py
* (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/drpc_server.py
* (edit) ambari-server/src/main/resources/common-services/RANGER/1.0.0.3.0/package/scripts/ranger_admin.py
* (edit) ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/mahout_client.py
* (edit) ambari-server/src/main/resources/common-services/SPARK/2.2.0/package/scripts/job_history_server.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_client.py
* (edit) ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/tez_client.py
* (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/pacemaker.py
* (edit) ambari-server/src/main/resources/common-services/RANGER_KMS/1.0.0.3.0/package/scripts/upgrade.py
* (edit) ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/upgrade.py
* (edit) ambari-server/src/main/resources/common-services/RANGER/1.0.0.3.0/package/scripts/upgrade.py
* (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/nimbus_prod.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/journalnode.py
* (edit) ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py
* (edit) ambari-server/src/test/python/stacks/2.1/STORM/test_storm_nimbus.py
* (edit) ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py
* (edit) ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_client.py
* (edit) ambari-server/src/main/resources/common-services/ATLAS/0.7.0.3.0/package/scripts/metadata_server.py
* (edit) ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/nfsgateway.py
* (edit) ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/livy_server.py
* (edit) ambari-server/src/test/python/stacks/2.2/SLIDER/test_slider_client.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server.py
* (edit) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py
* (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/nimbus.py
* (edit) ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_client.py
* (edit) ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/slider_client.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_server.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/nfsgateway.py
* (edit) ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/phoenix_queryserver.py
* (edit) ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/atlas_client.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/hive_client.py
* (edit) ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml
* (edit) ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka_broker.py
* (edit) ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/druid_node.py
* (edit) ambari-server/src/main/resources/common-services/RANGER/1.0.0.3.0/package/scripts/ranger_usersync.py
* (edit) ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_client.py
* (edit) ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_client.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py
* (edit) ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.3.0/package/scripts/master.py
* (edit) ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/package/scripts/resourcemanager.py
* (edit) ambari-server/src/main/resources/custom_actions/scripts/ru_set_all.py
* (edit) ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/hbase_client.py
* (edit) ambari-server/src/main/resources/common-services/HBASE/2.0.0.3.0/package/scripts/phoenix_queryserver.py
* (edit) ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/ranger_usersync.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/hdfs_client.py
* (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/nimbus_prod.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py
* (edit) ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/accumulo_script.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/snamenode.py
* (edit) ambari-server/src/main/resources/common-services/SLIDER/0.60.0.2.2/package/scripts/slider_client.py
* (edit) ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/package/scripts/zkfc_slave.py
* (edit) ambari-server/src/main/resources/common-services/ATLAS/0.7.0.3.0/package/scripts/atlas_client.py
* (edit) ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/historyserver.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_yarn_client.py
* (edit) ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/ranger_admin.py
* (edit) ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/livy2_server.py
* (edit) ambari-server/src/test/python/stacks/2.2/RANGER/test_ranger_usersync.py
* (edit) ambari-server/src/main/resources/common-services/SPARK/2.2.0/package/scripts/livy_server.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/HBASE/test_phoenix_queryserver.py
* (edit) ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/job_history_server.py
* (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/ui_server.py
* (edit) ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/package/scripts/zookeeper_server.py
* (delete) ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/upgrade.py
* (edit) ambari-server/src/main/resources/common-services/RANGER_KMS/1.0.0.3.0/package/scripts/kms_server.py
* (edit) ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py
* (edit) ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms_server.py
* (edit) ambari-server/src/test/python/stacks/2.2/ACCUMULO/test_accumulo_client.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py
* (edit) ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_client.py
* (edit) ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/job_history_server.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/PIG/test_pig_client.py
* (edit) ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/supervisor.py
* (edit) ambari-server/src/main/resources/common-services/KNOX/0.5.0.3.0/package/scripts/knox_gateway.py
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_select_packages.json
* (edit) ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py
* (edit) ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/package/scripts/pacemaker.py
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/properties/stack_select_packages.json
* (edit) ambari-server/src/test/python/stacks/2.1/STORM/test_storm_drpc_server.py
* (edit) ambari-server/src/main/resources/common-services/HIVE/2.1.0.3.0/package/scripts/hive_metastore.py
> Replace Hard Coded conf-select Structures
> -----------------------------------------
>
> Key: AMBARI-21581
> URL: https://issues.apache.org/jira/browse/AMBARI-21581
> Project: Ambari
> Issue Type: Task
> Affects Versions: 2.6.0
> Reporter: Jonathan Hurley
> Assignee: Jonathan Hurley
> Priority: Critical
> Fix For: 2.6.0
>
> Attachments: AMBARI-21581.patch
>
>
> Currently, both the stack-select and conf-select python files have hard-coded structures inside of them which contain mappings for things like configuration directories and stack-select component names.
> This code is really HDP-specific in that it reverses the mappings of the RPMs and allows us to remap {{/etc/<component>/conf}} to {{/usr/hdp/current/<stack-component>/conf}}
> {code:title=conf_select.py}
> _PACKAGE_DIRS = {
> "atlas": [
> {
> "conf_dir": "/etc/atlas/conf",
> "current_dir": "{0}/current/atlas-client/conf".format(STACK_ROOT_PATTERN)
> }
> ],
> "accumulo": [
> {
> "conf_dir": "/etc/accumulo/conf",
> "current_dir": "{0}/current/accumulo-client/conf".format(STACK_ROOT_PATTERN)
> }
> ],
> ...
> {code}
> With the coming of management packs replacing stacks, we can no longer hard code this in Python. My suggestion is to begin moving this data into a property of some sort. Today, {{cluster-env}} exists and would be the only place for it (where the stack-feature tools are now). However, I believe that {{cluster-env}} is also going away and being replaced with something similar to "Cluster Settings".
> In any event, this Jira is to track the work needed to replace this logic.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)