You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by John Speidel <js...@hortonworks.com> on 2014/06/03 22:36:20 UTC
Review Request 22218: Provide topology validation when creating blueprint
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22218/
-----------------------------------------------------------
Review request for Ambari, Nate Cole and Sid Wagle.
Bugs: AMBARI-6015
https://issues.apache.org/jira/browse/AMBARI-6015
Repository: ambari
Description
-------
Provide topology validation when a blueprint is being created.
Topology is validated based on dependency and cardinality information provided in the associated Ambari stack definition. I addition to validation, some components, mostly clients, are auto-deployed if not present in the topology. Again, this behavior is specified in the corresponding stack.
If topology validation fails, a 400 response is returned with a descriptive message:
{ status: 400 message: "Cluster Topology validation failed. Invalid service component count: [MYSQL_SERVER(actual=0, required=1), HIVE_METASTORE(actual=0, required=1), HIVE_SERVER(actual=0, required=1)]. To disable topology validation and create the blueprint, add the following to the end the url: '?validate_topology=false'" }
To create a blueprint that fails validation, a user can disable topology validation in the URL via '?validate_topology=false'.
The following cases are not currently supported by blueprint topology validation and will require a user to disable validation when creating a blueprint.
- external reference to MYSQL_SERVER or HIVE_METASTORE
- HA topology with > 1 NN and < 1SNN
- any other topology in which the default cardinality is is conditional on other topology information. HA is one example of this.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java 3e5a224
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BaseBlueprintProcessor.java PRE-CREATION
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintResourceProvider.java 5e04141
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterResourceProvider.java 2aa9ea1
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DefaultProviderModule.java cdc30d2
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostGroupEntity.java f0422e3
ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java d6d000f
ambari-server/src/main/resources/properties.json 9c6dd9c
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/metainfo.xml 6e057d2
ambari-server/src/main/resources/stacks/HDP/1.3.2/services/ZOOKEEPER/metainfo.xml d3e6ee0
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/FLUME/metainfo.xml 0e25020
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/metainfo.xml 3a181b8
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/metainfo.xml bf4e40b
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/metainfo.xml 4b498a7
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/metainfo.xml d6b8fd2
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/metainfo.xml fc09417
ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/metainfo.xml f878d77
ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/metainfo.xml 71c7837
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintResourceProviderTest.java 99a1120
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterResourceProviderTest.java 9f51ac7
Diff: https://reviews.apache.org/r/22218/diff/
Testing
-------
Functional testing:
- Tested various topologies
-- tests with auto-deploy
-- negative tests
-- positive tests without auto-deploy
- Tested disabling of topology validation
- Tested creating of cluster via blueprint
All unit tests pass:
Results :
Tests run: 1647, Failures: 0, Errors: 0, Skipped: 13
...
Total run:561
Total errors:0
Total failures:0
OK
Thanks,
John Speidel
Re: Review Request 22218: Provide topology validation when creating blueprint
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22218/#review44672
-----------------------------------------------------------
Ship it!
Ship It!
- Nate Cole
On June 3, 2014, 4:36 p.m., John Speidel wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22218/
> -----------------------------------------------------------
>
> (Updated June 3, 2014, 4:36 p.m.)
>
>
> Review request for Ambari, Nate Cole and Sid Wagle.
>
>
> Bugs: AMBARI-6015
> https://issues.apache.org/jira/browse/AMBARI-6015
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Provide topology validation when a blueprint is being created.
> Topology is validated based on dependency and cardinality information provided in the associated Ambari stack definition. I addition to validation, some components, mostly clients, are auto-deployed if not present in the topology. Again, this behavior is specified in the corresponding stack.
>
> If topology validation fails, a 400 response is returned with a descriptive message:
> { status: 400 message: "Cluster Topology validation failed. Invalid service component count: [MYSQL_SERVER(actual=0, required=1), HIVE_METASTORE(actual=0, required=1), HIVE_SERVER(actual=0, required=1)]. To disable topology validation and create the blueprint, add the following to the end the url: '?validate_topology=false'" }
>
> To create a blueprint that fails validation, a user can disable topology validation in the URL via '?validate_topology=false'.
> The following cases are not currently supported by blueprint topology validation and will require a user to disable validation when creating a blueprint.
> - external reference to MYSQL_SERVER or HIVE_METASTORE
> - HA topology with > 1 NN and < 1SNN
> - any other topology in which the default cardinality is is conditional on other topology information. HA is one example of this.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java 3e5a224
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BaseBlueprintProcessor.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintResourceProvider.java 5e04141
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterResourceProvider.java 2aa9ea1
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DefaultProviderModule.java cdc30d2
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostGroupEntity.java f0422e3
> ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java d6d000f
> ambari-server/src/main/resources/properties.json 9c6dd9c
> ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/metainfo.xml 6e057d2
> ambari-server/src/main/resources/stacks/HDP/1.3.2/services/ZOOKEEPER/metainfo.xml d3e6ee0
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/FLUME/metainfo.xml 0e25020
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/metainfo.xml 3a181b8
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/metainfo.xml bf4e40b
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/metainfo.xml 4b498a7
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/metainfo.xml d6b8fd2
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/metainfo.xml fc09417
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/metainfo.xml f878d77
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/metainfo.xml 71c7837
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintResourceProviderTest.java 99a1120
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterResourceProviderTest.java 9f51ac7
>
> Diff: https://reviews.apache.org/r/22218/diff/
>
>
> Testing
> -------
>
> Functional testing:
> - Tested various topologies
> -- tests with auto-deploy
> -- negative tests
> -- positive tests without auto-deploy
> - Tested disabling of topology validation
> - Tested creating of cluster via blueprint
>
> All unit tests pass:
> Results :
>
> Tests run: 1647, Failures: 0, Errors: 0, Skipped: 13
> ...
> Total run:561
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> John Speidel
>
>
Re: Review Request 22218: Provide topology validation when creating blueprint
Posted by John Speidel <js...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22218/#review44666
-----------------------------------------------------------
Due to code freeze deadlines, this patch doesn't fully cover all new code with unit tests.
These will be added in a separate patch.
Also, new code is not as well formed as I would like, but again, due to time constraints this will need to be improved in another patch.
- John Speidel
On June 3, 2014, 8:36 p.m., John Speidel wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22218/
> -----------------------------------------------------------
>
> (Updated June 3, 2014, 8:36 p.m.)
>
>
> Review request for Ambari, Nate Cole and Sid Wagle.
>
>
> Bugs: AMBARI-6015
> https://issues.apache.org/jira/browse/AMBARI-6015
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Provide topology validation when a blueprint is being created.
> Topology is validated based on dependency and cardinality information provided in the associated Ambari stack definition. I addition to validation, some components, mostly clients, are auto-deployed if not present in the topology. Again, this behavior is specified in the corresponding stack.
>
> If topology validation fails, a 400 response is returned with a descriptive message:
> { status: 400 message: "Cluster Topology validation failed. Invalid service component count: [MYSQL_SERVER(actual=0, required=1), HIVE_METASTORE(actual=0, required=1), HIVE_SERVER(actual=0, required=1)]. To disable topology validation and create the blueprint, add the following to the end the url: '?validate_topology=false'" }
>
> To create a blueprint that fails validation, a user can disable topology validation in the URL via '?validate_topology=false'.
> The following cases are not currently supported by blueprint topology validation and will require a user to disable validation when creating a blueprint.
> - external reference to MYSQL_SERVER or HIVE_METASTORE
> - HA topology with > 1 NN and < 1SNN
> - any other topology in which the default cardinality is is conditional on other topology information. HA is one example of this.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractControllerResourceProvider.java 3e5a224
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BaseBlueprintProcessor.java PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintResourceProvider.java 5e04141
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterResourceProvider.java 2aa9ea1
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/DefaultProviderModule.java cdc30d2
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostGroupEntity.java f0422e3
> ambari-server/src/main/java/org/apache/ambari/server/state/DependencyInfo.java d6d000f
> ambari-server/src/main/resources/properties.json 9c6dd9c
> ambari-server/src/main/resources/stacks/HDP/1.3.2/services/HIVE/metainfo.xml 6e057d2
> ambari-server/src/main/resources/stacks/HDP/1.3.2/services/ZOOKEEPER/metainfo.xml d3e6ee0
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/FLUME/metainfo.xml 0e25020
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/GANGLIA/metainfo.xml 3a181b8
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HBASE/metainfo.xml bf4e40b
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HDFS/metainfo.xml 4b498a7
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/metainfo.xml d6b8fd2
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/ZOOKEEPER/metainfo.xml fc09417
> ambari-server/src/main/resources/stacks/HDP/2.1/services/FALCON/metainfo.xml f878d77
> ambari-server/src/main/resources/stacks/HDP/2.1/services/TEZ/metainfo.xml 71c7837
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintResourceProviderTest.java 99a1120
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterResourceProviderTest.java 9f51ac7
>
> Diff: https://reviews.apache.org/r/22218/diff/
>
>
> Testing
> -------
>
> Functional testing:
> - Tested various topologies
> -- tests with auto-deploy
> -- negative tests
> -- positive tests without auto-deploy
> - Tested disabling of topology validation
> - Tested creating of cluster via blueprint
>
> All unit tests pass:
> Results :
>
> Tests run: 1647, Failures: 0, Errors: 0, Skipped: 13
> ...
> Total run:561
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> John Speidel
>
>