You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ambari.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2017/10/03 13:26:00 UTC
[jira] [Commented] (AMBARI-22095) Make hooks stack agnostic

    [ https://issues.apache.org/jira/browse/AMBARI-22095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16189701#comment-16189701 ] 

Hudson commented on AMBARI-22095:
---------------------------------

FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #8194 (See [https://builds.apache.org/job/Ambari-trunk-Commit/8194/])
AMBARI-22095 Make hooks stack agnostic (dsen) (dsen: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=5b36cdfd87b756eba922dfd1ac5419552f4d375f])
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-START/files/task-log4j.properties
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-START/files/checkForFormat.sh
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py
* (edit) ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
* (add) ambari-server/src/main/resources/stack-hooks/before-START/scripts/rack_awareness.py
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/exclude_hosts_list.j2
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-START/templates/include_hosts_list.j2
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py
* (add) ambari-server/src/main/resources/stack-hooks/after-INSTALL/scripts/shared_initialization.py
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py
* (add) ambari-server/src/main/resources/stack-hooks/before-START/templates/topology_mappings.data.j2
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/rack_awareness.py
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/hook.py
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/files/task-log4j.properties
* (edit) ambari-server/src/main/resources/stacks/HDP/2.3.ECS/services/ECS/package/scripts/ecs_client.py
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-START/scripts/shared_initialization.py
* (add) ambari-server/src/main/resources/stack-hooks/before-ANY/scripts/hook.py
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/files/changeToSecureUid.sh
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-START/templates/topology_mappings.data.j2
* (edit) ambari-agent/src/main/python/ambari_agent/FileCache.py
* (add) ambari-server/src/main/resources/stack-hooks/before-START/scripts/shared_initialization.py
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/params.py
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-START/templates/hadoop-metrics2.properties.j2
* (add) ambari-server/src/main/resources/stack-hooks/before-INSTALL/scripts/params.py
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-ANY/scripts/hook.py
* (edit) ambari-server/src/test/python/TestResourceFilesKeeper.py
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-START/templates/health_check.j2
* (edit) ambari-server/src/main/resources/scripts/post-user-creation-hook.sh
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-INSTALL/scripts/shared_initialization.py
* (add) ambari-server/src/main/resources/stack-hooks/before-START/files/fast-hdfs-resource.jar
* (edit) ambari-agent/pom.xml
* (add) ambari-server/src/main/resources/stack-hooks/before-START/scripts/hook.py
* (add) ambari-server/src/main/resources/stack-hooks/after-INSTALL/scripts/params.py
* (add) ambari-server/src/main/resources/stack-hooks/before-START/templates/hadoop-metrics2.properties.j2
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py
* (edit) ambari-server/src/main/python/ambari_server/setupMpacks.py
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/files/fast-hdfs-resource.jar
* (edit) ambari-server/src/main/resources/scripts/Ambaripreupload.py
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-START/scripts/params.py
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/hadoop-metrics2.properties.j2
* (add) ambari-server/src/main/resources/stack-hooks/before-INSTALL/scripts/shared_initialization.py
* (add) ambari-server/src/main/resources/stack-hooks/before-START/templates/commons-logging.properties.j2
* (edit) ambari-server/src/test/java/org/apache/ambari/server/api/services/AmbariMetaInfoTest.java
* (edit) ambari-server/src/main/java/org/apache/ambari/server/stack/StackDirectory.java
* (edit) ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ExecutionCommandWrapper.java
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py
* (edit) ambari-agent/src/test/python/ambari_agent/TestFileCache.py
* (edit) ambari-server/src/main/java/org/apache/ambari/server/agent/HeartbeatMonitor.java
* (add) ambari-server/src/main/resources/stack-hooks/before-INSTALL/scripts/repo_initialization.py
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/after-INSTALL/scripts/params.py
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/health_check.j2
* (edit) ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py
* (edit) ambari-server/src/main/python/ambari_server/serverConfiguration.py
* (add) ambari-server/src/main/resources/stack-hooks/before-START/scripts/custom_extensions.py
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/repo_initialization.py
* (edit) ambari-server/src/main/assemblies/server.xml
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-START/templates/commons-logging.properties.j2
* (edit) ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py
* (edit) ambari-server/src/test/python/stacks/2.0.6/hooks/before-ANY/test_before_any.py
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-INSTALL/scripts/hook.py
* (add) ambari-server/src/main/resources/stack-hooks/before-INSTALL/scripts/hook.py
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/params.py
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-START/files/topology_script.py
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/files/checkForFormat.sh
* (add) ambari-server/src/main/resources/stack-hooks/after-INSTALL/scripts/hook.py
* (edit) ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/files/topology_script.py
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-ANY/scripts/shared_initialization.py
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/after-INSTALL/scripts/shared_initialization.py
* (add) ambari-server/src/main/resources/stack-hooks/before-START/files/topology_script.py
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-START/templates/exclude_hosts_list.j2
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py
* (edit) ambari-server/src/test/python/stacks/utils/RMFTestCase.py
* (edit) ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-START/scripts/hook.py
* (add) ambari-server/src/main/resources/stack-hooks/before-START/templates/include_hosts_list.j2
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/after-INSTALL/scripts/hook.py
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-RESTART/scripts/hook.py
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/shared_initialization.py
* (add) ambari-server/src/main/resources/stack-hooks/before-ANY/scripts/shared_initialization.py
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-START/files/fast-hdfs-resource.jar
* (add) ambari-server/src/main/resources/stack-hooks/before-START/scripts/params.py
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/include_hosts_list.j2
* (add) ambari-server/src/main/resources/stack-hooks/before-ANY/scripts/params.py
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/topology_mappings.data.j2
* (add) ambari-server/src/main/resources/stack-hooks/before-START/files/task-log4j.properties
* (edit) ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java
* (add) ambari-server/src/main/resources/stack-hooks/before-START/files/checkForFormat.sh
* (edit) contrib/management-packs/hdf-ambari-mpack/src/main/assemblies/hdf-ambari-mpack.xml
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-INSTALL/scripts/params.py
* (edit) ambari-server/pom.xml
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/commons-logging.properties.j2
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-START/scripts/rack_awareness.py
* (add) ambari-server/src/main/resources/stack-hooks/before-START/templates/exclude_hosts_list.j2
* (add) ambari-server/src/main/resources/stack-hooks/before-START/templates/health_check.j2
* (add) ambari-server/src/main/resources/stack-hooks/before-RESTART/scripts/hook.py
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-ANY/scripts/params.py
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/custom_extensions.py
* (add) ambari-server/src/main/resources/stack-hooks/before-ANY/files/changeToSecureUid.sh
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-ANY/files/changeToSecureUid.sh
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/hook.py
* (edit) ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java
* (delete) ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-INSTALL/scripts/repo_initialization.py
* (delete) ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-RESTART/scripts/hook.py


> Make hooks stack agnostic
> -------------------------
>
>                 Key: AMBARI-22095
>                 URL: https://issues.apache.org/jira/browse/AMBARI-22095
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 3.0.0
>            Reporter: Dmytro Sen
>            Assignee: Dmytro Sen
>            Priority: Critical
>             Fix For: 3.0.0
>
>         Attachments: AMBARI-22095.patch
>
>
> - At present hooks are located inside stacks 
> - For all HDP stacks, we use https://github.com/apache/ambari/tree/trunk/ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks
> - Similarly for all HDF stacks, we use https://github.com/apache/ambari/tree/trunk/contrib/management-packs/hdf-ambari-mpack/src/main/resources/stacks/HDF/2.0/hooks
> - Any time the HDP hooks are updated, the changes have to be ported to HDF branch. This is a maintenance overhead
> - In Ambari 3.0 when we move to mpack based operations model, this will be a maint nightmare. 
> *Requirements*
> # We should look into making hooks stack agnostic. 
> # Hooks should be moved from resources/stacks/HDP/2.0.6/hooks to resources/stack-hooks
> # Hooks should be entirely config driven
> # There should be no HDP or HDF specific logic in the hooks (hdp-select, conf-select, hdf-select, hdfconf-select) 
> # There should be no service specific logic in the hooks (for example: hardcoded service user to service group mapping etc. this should be stack driven see: AMBARI-20034).
> # Any service specific logic that doesn't make sense should be removed and refactored into respective service definitions.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)