You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Robert Nettleton <rn...@hortonworks.com> on 2016/01/08 21:42:18 UTC
Review Request 42085: Add INSTALL_ONLY support to Blueprints on a
per-component basis
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42085/
-----------------------------------------------------------
Review request for Ambari, Mahadev Konar and Sumit Mohanty.
Bugs: AMBARI-14555
https://issues.apache.org/jira/browse/AMBARI-14555
Repository: ambari
Description
-------
This patch implements the new Blueprint support described in AMBARI-14555.
Blueprints will now support syntax to allow users to mark certain components in a Blueprint host group as "INSTALL_ONLY", meaning that Ambari will install them, but that these components will not be started initially.
This patch implements the following:
1. Adds a new column to the "hostgroup_component" table in the Ambari DB, "provision_action", to store the provisioning state desired for a component.
2. Moves the ProvisionAction enumerated type to a top-level class, in order to share this across the entire Blueprints implementation.
3. Adds code to the UpgradeCatalog implementation for Ambari 2.2.1, in order to add the new column to older databases during an upgrade, and adds a unit test for this as well.
4. Refactors the HostGroup interface and implementation, in order to support more configuration data being attached to a given host group component.
5. Updates various sections of the code that use the HostGroup interface, to accomodate these changes.
6. Adds handling code to store and read this new Blueprint syntax during the POST/GET calls on the BlueprintResourceProvider.
7. Modifies the HostComponentResourceProvider's start() method, such that a set of components that are marked as "INSTALL_ONLY" can be passed in. Modifies the start() implementation to use a new set of predicates to filter out any components that should not be started prior to the start() operation being executed on the host.
8. Updates various unit tests to accomodate interface and implementation changes.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRenderer.java 3705ceb
ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorBlueprintProcessor.java d57c17d
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintResourceProvider.java f3100b5
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExportBlueprintRequest.java 8c8b89d
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentResourceProvider.java 194d75f
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionAction.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java 7b1de26
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostGroupComponentEntity.java 984c549
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostGroupComponentEntityPK.java fb9011b
ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java 6bfee93
ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintFactory.java b8ce749
ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintImpl.java 88052b0
ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java 1c293ee
ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java c3c04db
ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java e78300c
ambari-server/src/main/java/org/apache/ambari/server/topology/Component.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/topology/HostGroup.java 07e3e88
ambari-server/src/main/java/org/apache/ambari/server/topology/HostGroupImpl.java b89e7e4
ambari-server/src/main/java/org/apache/ambari/server/topology/HostRequest.java 440638c
ambari-server/src/main/java/org/apache/ambari/server/topology/LogicalRequest.java bd9f2e0
ambari-server/src/main/java/org/apache/ambari/server/topology/RequiredPasswordValidator.java e26de3f
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog221.java db60d77
ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql d93a3c2
ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql aa8ced1
ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql b534344
ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 941fc6e
ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql dd517f8
ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql f837f9e
ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 239d27e
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/package/scripts/common.py c427c83
ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java 522d902
ambari-server/src/test/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorBlueprintProcessorTest.java 514e6ab
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 0384b45
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ExportBlueprintRequestTest.java 94ba90f
ambari-server/src/test/java/org/apache/ambari/server/orm/entities/HostGroupComponentEntityTest.java c0efd0d
ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintFactoryTest.java 3a3b6dc
ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java 3addfc4
ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java 304cded
ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartTest.java 1354a72
ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterTopologyImplTest.java 08aa3d3
ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java f4ded70
ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java 47169f4
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog221Test.java 319024b
Diff: https://reviews.apache.org/r/42085/diff/
Testing
-------
1. Tested a 3-node HDFS HA cluster using a Blueprint with the original syntax (no new additions), and this deployment succeeded.
2. Tested a 3-node HDFS HA cluster with certain components marked as "INSTALL_ONLY" with the new syntax. Verified that the cluster deployed properly, and that the marked components were installed, but were not started by the Blueprints deployment.
3. Ran "mvn clean test" in the ambari-server project:
"Tests in error:
UpgradeCatalog221Test.testExecuteDMLUpdates:158 » NullPointer
Tests run: 3741, Failures: 0, Errors: 1, Skipped: 29"
The one failure seen also occurs in trunk without any patches applied, so this current patch does not introduce any new failures.
Thanks,
Robert Nettleton
Re: Review Request 42085: Add INSTALL_ONLY support to Blueprints on a
per-component basis
Posted by Robert Nettleton <rn...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42085/
-----------------------------------------------------------
(Updated Jan. 8, 2016, 8:52 p.m.)
Review request for Ambari, Mahadev Konar and Sumit Mohanty.
Changes
-------
Updated patch to remove extraneous file changes not required to implement AMBARI-14555.
Bugs: AMBARI-14555
https://issues.apache.org/jira/browse/AMBARI-14555
Repository: ambari
Description
-------
This patch implements the new Blueprint support described in AMBARI-14555.
Blueprints will now support syntax to allow users to mark certain components in a Blueprint host group as "INSTALL_ONLY", meaning that Ambari will install them, but that these components will not be started initially.
This patch implements the following:
1. Adds a new column to the "hostgroup_component" table in the Ambari DB, "provision_action", to store the provisioning state desired for a component.
2. Moves the ProvisionAction enumerated type to a top-level class, in order to share this across the entire Blueprints implementation.
3. Adds code to the UpgradeCatalog implementation for Ambari 2.2.1, in order to add the new column to older databases during an upgrade, and adds a unit test for this as well.
4. Refactors the HostGroup interface and implementation, in order to support more configuration data being attached to a given host group component.
5. Updates various sections of the code that use the HostGroup interface, to accomodate these changes.
6. Adds handling code to store and read this new Blueprint syntax during the POST/GET calls on the BlueprintResourceProvider.
7. Modifies the HostComponentResourceProvider's start() method, such that a set of components that are marked as "INSTALL_ONLY" can be passed in. Modifies the start() implementation to use a new set of predicates to filter out any components that should not be started prior to the start() operation being executed on the host.
8. Updates various unit tests to accomodate interface and implementation changes.
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRenderer.java 3705ceb
ambari-server/src/main/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorBlueprintProcessor.java d57c17d
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintResourceProvider.java f3100b5
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExportBlueprintRequest.java 8c8b89d
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentResourceProvider.java 194d75f
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionAction.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ProvisionClusterRequest.java 7b1de26
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostGroupComponentEntity.java 984c549
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostGroupComponentEntityPK.java fb9011b
ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java 6bfee93
ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintFactory.java b8ce749
ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintImpl.java 88052b0
ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintValidatorImpl.java 1c293ee
ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java c3c04db
ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java e78300c
ambari-server/src/main/java/org/apache/ambari/server/topology/Component.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/topology/HostGroup.java 07e3e88
ambari-server/src/main/java/org/apache/ambari/server/topology/HostGroupImpl.java b89e7e4
ambari-server/src/main/java/org/apache/ambari/server/topology/HostRequest.java 440638c
ambari-server/src/main/java/org/apache/ambari/server/topology/LogicalRequest.java bd9f2e0
ambari-server/src/main/java/org/apache/ambari/server/topology/RequiredPasswordValidator.java e26de3f
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog221.java 5cde24b
ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql d93a3c2
ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql aa8ced1
ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql b534344
ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 941fc6e
ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql dd517f8
ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql f837f9e
ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 239d27e
ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java 522d902
ambari-server/src/test/java/org/apache/ambari/server/api/services/stackadvisor/StackAdvisorBlueprintProcessorTest.java 514e6ab
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 0384b45
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ExportBlueprintRequestTest.java 94ba90f
ambari-server/src/test/java/org/apache/ambari/server/orm/entities/HostGroupComponentEntityTest.java c0efd0d
ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintFactoryTest.java 3a3b6dc
ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java 3addfc4
ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintValidatorImplTest.java 304cded
ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartTest.java 1354a72
ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterTopologyImplTest.java 08aa3d3
ambari-server/src/test/java/org/apache/ambari/server/topology/RequiredPasswordValidatorTest.java f4ded70
ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java 47169f4
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog221Test.java 319024b
Diff: https://reviews.apache.org/r/42085/diff/
Testing
-------
1. Tested a 3-node HDFS HA cluster using a Blueprint with the original syntax (no new additions), and this deployment succeeded.
2. Tested a 3-node HDFS HA cluster with certain components marked as "INSTALL_ONLY" with the new syntax. Verified that the cluster deployed properly, and that the marked components were installed, but were not started by the Blueprints deployment.
3. Ran "mvn clean test" in the ambari-server project:
"Tests in error:
UpgradeCatalog221Test.testExecuteDMLUpdates:158 » NullPointer
Tests run: 3741, Failures: 0, Errors: 1, Skipped: 29"
The one failure seen also occurs in trunk without any patches applied, so this current patch does not introduce any new failures.
Thanks,
Robert Nettleton