You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Ivan Kozlov <ik...@hortonworks.com> on 2014/04/15 18:54:46 UTC
Re: Review Request 18628: Refactor temporary shared resource for HDFS and MR
initialization
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18628/
-----------------------------------------------------------
(Updated April 15, 2014, 4:54 p.m.)
Review request for Ambari and Andrew Onischuk.
Bugs: AMBARI-4885
https://issues.apache.org/jira/browse/AMBARI-4885
Repository: ambari
Description
-------
Refactor (and remove if possible) the resource that has been created in a previous jira. All logic specific to MR should be moved to MR scripts. The same thing for HDFS.
Partially done. Diff for 1-st stack.
Diffs (updated)
-----
ambari-agent/src/main/python/resource_management/libraries/script/hook.py 19b5204
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/after-INSTALL/scripts/hook.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/after-INSTALL/scripts/params.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/after-INSTALL/scripts/shared_initialization.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/after-INSTALL/templates/hadoop-env.sh.j2 PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/hook.py 51e5cd2
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/params.py fa19ca3
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 6a37d02
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-RESTART/scripts/hook.py 7042602
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/hook.py 4ee2214
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/params.py 2db9b4b
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/shared_initialization.py c5828d3
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/templates/hadoop-env.sh.j2 76ac3f3
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/templates/hdfs.conf.j2 d58a6f5
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/templates/slaves.j2 4a9e713
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/templates/taskcontroller.cfg.j2 3d5f4f2
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/scripts/datanode.py 57fdb35
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/scripts/hdfs.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/scripts/hdfs_client.py 40bde89
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/scripts/namenode.py 2f26c98
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/scripts/params.py 98d536c
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/scripts/snamenode.py b2a3bd1
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/templates/hdfs.conf.j2 PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/templates/slaves.j2 PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/package/scripts/mapreduce.py 94fe24e
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/package/scripts/params.py 1a77f95
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/package/templates/taskcontroller.cfg.j2 PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/hook.py 51e5cd2
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/params.py 341d86f
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py c7c0c70
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-RESTART/scripts/hook.py 7042602
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py 4ee2214
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py f1ef84a
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py 3ffcc76
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/hdfs.conf.j2 d58a6f5
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/slaves.j2 4a9e713
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/taskcontroller.cfg.j2 3d5f4f2
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/datanode.py 57fdb35
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs_client.py 6220326
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/journalnode.py a707e2e
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/namenode.py 2179292
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/params.py d55488d
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/snamenode.py b2a3bd1
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/zkfc_slave.py f415f24
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/templates/hdfs.conf.j2 PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/templates/slaves.j2 PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/package/scripts/params.py 64e8f81
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/package/scripts/yarn.py 4c71566
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/package/templates/taskcontroller.cfg.j2 PRE-CREATION
ambari-server/src/test/python/stacks/1.3.2/HDFS/test_datanode.py 6f51a8c
ambari-server/src/test/python/stacks/1.3.2/HDFS/test_namenode.py 6efa948
ambari-server/src/test/python/stacks/1.3.2/HDFS/test_snamenode.py 058c19a
ambari-server/src/test/python/stacks/1.3.2/MAPREDUCE/test_mapreduce_client.py 3606595
ambari-server/src/test/python/stacks/1.3.2/MAPREDUCE/test_mapreduce_historyserver.py 7367b82
ambari-server/src/test/python/stacks/1.3.2/MAPREDUCE/test_mapreduce_jobtracker.py faa27fd
ambari-server/src/test/python/stacks/1.3.2/MAPREDUCE/test_mapreduce_tasktracker.py d725c90
ambari-server/src/test/python/stacks/1.3.2/hooks/after-INSTALL/test_after_install.py PRE-CREATION
ambari-server/src/test/python/stacks/1.3.2/hooks/before-INSTALL/test_before_install.py 3867f48
ambari-server/src/test/python/stacks/1.3.2/hooks/before-START/test_before_start.py 2663839
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py 4ee2571
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py 380f0f8
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 9c659ea
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_snamenode.py fc96be5
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_zkfc.py d33e063
ambari-server/src/test/python/stacks/2.0.6/HIVE/_test_hive_service_check.py PRE-CREATION
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_service_check.py 00c24d4
ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 25525c4
ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_client.py 95d5b48
ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py e966b46
ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py e5f38b7
ambari-server/src/test/python/stacks/2.0.6/YARN/test_yarn_client.py 25c72f6
ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py 8559db0
ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 88448e6
ambari-server/src/test/python/stacks/2.1/YARN/test_apptimelineserver.py 0a35276
Diff: https://reviews.apache.org/r/18628/diff/
Testing
-------
security and default deployment on multinode cluster. deployment of mr, hdfs and "only client" components on separate hosts (default and secure cluster).
Thanks,
Ivan Kozlov
Re: Review Request 18628: Refactor temporary shared resource for HDFS and MR
initialization
Posted by Ivan Kozlov <ik...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/18628/
-----------------------------------------------------------
(Updated April 16, 2014, 3:14 p.m.)
Review request for Ambari and Andrew Onischuk.
Bugs: AMBARI-4885
https://issues.apache.org/jira/browse/AMBARI-4885
Repository: ambari
Description
-------
Refactor (and remove if possible) the resource that has been created in a previous jira. All logic specific to MR should be moved to MR scripts. The same thing for HDFS.
Partially done. Diff for 1-st stack.
Diffs (updated)
-----
ambari-agent/src/main/python/resource_management/libraries/script/hook.py 19b5204
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/after-INSTALL/scripts/hook.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/after-INSTALL/scripts/params.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/after-INSTALL/scripts/shared_initialization.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/after-INSTALL/templates/hadoop-env.sh.j2 PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/hook.py 51e5cd2
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/params.py fa19ca3
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-INSTALL/scripts/shared_initialization.py 6a37d02
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-RESTART/scripts/hook.py 7042602
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/hook.py 075a6b6
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/params.py 2db9b4b
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/scripts/shared_initialization.py c6cc432
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/templates/hadoop-env.sh.j2 76ac3f3
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/templates/hdfs.conf.j2 d58a6f5
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/templates/slaves.j2 4a9e713
ambari-server/src/main/resources/stacks/HDP/1.3.2/hooks/before-START/templates/taskcontroller.cfg.j2 3d5f4f2
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/scripts/datanode.py 57fdb35
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/scripts/hdfs.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/scripts/hdfs_client.py 40bde89
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/scripts/namenode.py 2f26c98
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/scripts/params.py 98d536c
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/scripts/snamenode.py b2a3bd1
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/templates/hdfs.conf.j2 PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HDFS/package/templates/slaves.j2 PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/package/scripts/mapreduce.py 4b26814
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/package/scripts/params.py 1a77f95
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/MAPREDUCE/package/templates/taskcontroller.cfg.j2 PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/hook.py 51e5cd2
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/params.py 341d86f
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py c7c0c70
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-RESTART/scripts/hook.py 7042602
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/hook.py 075a6b6
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/params.py f1ef84a
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/scripts/shared_initialization.py f132c2e
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/hdfs.conf.j2 d58a6f5
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/slaves.j2 4a9e713
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-START/templates/taskcontroller.cfg.j2 3d5f4f2
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/datanode.py 57fdb35
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs.py PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/hdfs_client.py 6220326
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/journalnode.py a707e2e
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/namenode.py 2179292
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/params.py d55488d
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/snamenode.py b2a3bd1
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/scripts/zkfc_slave.py f415f24
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/templates/hdfs.conf.j2 PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/package/templates/slaves.j2 PRE-CREATION
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/package/scripts/params.py 64e8f81
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/package/scripts/yarn.py 4c71566
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/YARN/package/templates/taskcontroller.cfg.j2 PRE-CREATION
ambari-server/src/test/python/stacks/1.3.2/HDFS/test_datanode.py 6f51a8c
ambari-server/src/test/python/stacks/1.3.2/HDFS/test_namenode.py 6efa948
ambari-server/src/test/python/stacks/1.3.2/HDFS/test_snamenode.py 058c19a
ambari-server/src/test/python/stacks/1.3.2/MAPREDUCE/test_mapreduce_client.py d581903
ambari-server/src/test/python/stacks/1.3.2/MAPREDUCE/test_mapreduce_historyserver.py d22f4ba
ambari-server/src/test/python/stacks/1.3.2/MAPREDUCE/test_mapreduce_jobtracker.py 255ba1a
ambari-server/src/test/python/stacks/1.3.2/MAPREDUCE/test_mapreduce_tasktracker.py bab7f71
ambari-server/src/test/python/stacks/1.3.2/hooks/after-INSTALL/test_after_install.py PRE-CREATION
ambari-server/src/test/python/stacks/1.3.2/hooks/before-INSTALL/test_before_install.py 3867f48
ambari-server/src/test/python/stacks/1.3.2/hooks/before-START/test_before_start.py 1ef57aa
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py 4ee2571
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py 380f0f8
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 9c659ea
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_snamenode.py fc96be5
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_zkfc.py d33e063
ambari-server/src/test/python/stacks/2.0.6/HIVE/_test_hive_service_check.py PRE-CREATION
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_service_check.py 00c24d4
ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 25525c4
ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_client.py 95d5b48
ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py e966b46
ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py e5f38b7
ambari-server/src/test/python/stacks/2.0.6/YARN/test_yarn_client.py 25c72f6
ambari-server/src/test/python/stacks/2.0.6/hooks/before-INSTALL/test_before_install.py 8559db0
ambari-server/src/test/python/stacks/2.0.6/hooks/before-START/test_before_start.py 7efb056
ambari-server/src/test/python/stacks/2.1/YARN/test_apptimelineserver.py 0a35276
Diff: https://reviews.apache.org/r/18628/diff/
Testing
-------
security and default deployment on multinode cluster. deployment of mr, hdfs and "only client" components on separate hosts (default and secure cluster).
Thanks,
Ivan Kozlov