You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Robert Nettleton <rn...@hortonworks.com> on 2015/05/17 21:47:53 UTC
Review Request 34334: Register Yarn ResourceManager HA configuration
properties with the Blueprint config processor
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34334/
-----------------------------------------------------------
Review request for Ambari, Mahadev Konar, Robert Levas, and Sumit Mohanty.
Bugs: AMBARI-10411
https://issues.apache.org/jira/browse/AMBARI-10411
Repository: ambari
Description
-------
This patch addresses AMBARI-10411.
The BlueprintConfigurationProcessor was not properly handling the dynamic property names for Yarn ResourceManager HA scenarios, in which some property names include the name of the specific resource manager. This would cause certain Yarn HA properties to contain hostname information in an exported Blueprint, which would make the Blueprint less portable across different cluster types.
This patch fixes this issue by:
1. Adding some code to dynamically register the appropriate PropertyUpdater instances, based on the Yarn HA configuration specified in yarn-site.xml.
2. Adds a property updater for the "yarn.resourcemanager.zk-address", which was previously not considered by the BlueprintConfigurationProcessor. This allows this property to now be set automatically by the framework, and will also allow the Blueprint export process to properly mask the hostnames in this property.
3. Adds a property updater for "yarn.resourcemanager.webapp.https.address", for the same reasons listed above.
4. Adds a new unit test to verify this change.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java d7ff543
ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java 6a7a79b
ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java e0e79b3
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java bebc884
Diff: https://reviews.apache.org/r/34334/diff/
Testing
-------
1. Ran the ambari-server unit tests, all passing.
2. Deployed a 3-node Yarn cluster with my patch applied, using the Ambari UI, enabled Yarn RM HA, and then exported the Blueprint. I verified that these properties are now masked, and no hostname information appears in the exported Blueprint.
3. Deployed a separate 3-node Yarn cluster with a Blueprint that does not set "yarn.resourcemanager.zk-address", and verified that the cluster started up properly. This verifies that the BlueprintConfigurationProcessor is properly updating the topology information to the list of Zookeeper servers in the cluster.
Thanks,
Robert Nettleton
Re: Review Request 34334: Register Yarn ResourceManager HA
configuration properties with the Blueprint config processor
Posted by Sumit Mohanty <sm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34334/#review84154
-----------------------------------------------------------
Ship it!
Ship It!
- Sumit Mohanty
On May 17, 2015, 7:47 p.m., Robert Nettleton wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34334/
> -----------------------------------------------------------
>
> (Updated May 17, 2015, 7:47 p.m.)
>
>
> Review request for Ambari, Mahadev Konar, Robert Levas, and Sumit Mohanty.
>
>
> Bugs: AMBARI-10411
> https://issues.apache.org/jira/browse/AMBARI-10411
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This patch addresses AMBARI-10411.
>
> The BlueprintConfigurationProcessor was not properly handling the dynamic property names for Yarn ResourceManager HA scenarios, in which some property names include the name of the specific resource manager. This would cause certain Yarn HA properties to contain hostname information in an exported Blueprint, which would make the Blueprint less portable across different cluster types.
>
> This patch fixes this issue by:
>
> 1. Adding some code to dynamically register the appropriate PropertyUpdater instances, based on the Yarn HA configuration specified in yarn-site.xml.
> 2. Adds a property updater for the "yarn.resourcemanager.zk-address", which was previously not considered by the BlueprintConfigurationProcessor. This allows this property to now be set automatically by the framework, and will also allow the Blueprint export process to properly mask the hostnames in this property.
> 3. Adds a property updater for "yarn.resourcemanager.webapp.https.address", for the same reasons listed above.
> 4. Adds a new unit test to verify this change.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java d7ff543
> ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java 6a7a79b
> ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java e0e79b3
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java bebc884
>
> Diff: https://reviews.apache.org/r/34334/diff/
>
>
> Testing
> -------
>
> 1. Ran the ambari-server unit tests, all passing.
> 2. Deployed a 3-node Yarn cluster with my patch applied, using the Ambari UI, enabled Yarn RM HA, and then exported the Blueprint. I verified that these properties are now masked, and no hostname information appears in the exported Blueprint.
> 3. Deployed a separate 3-node Yarn cluster with a Blueprint that does not set "yarn.resourcemanager.zk-address", and verified that the cluster started up properly. This verifies that the BlueprintConfigurationProcessor is properly updating the topology information to the list of Zookeeper servers in the cluster.
>
>
> Thanks,
>
> Robert Nettleton
>
>
Re: Review Request 34334: Register Yarn ResourceManager HA
configuration properties with the Blueprint config processor
Posted by Robert Levas <rl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34334/#review84090
-----------------------------------------------------------
Ship it!
Ship It!
- Robert Levas
On May 17, 2015, 3:47 p.m., Robert Nettleton wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34334/
> -----------------------------------------------------------
>
> (Updated May 17, 2015, 3:47 p.m.)
>
>
> Review request for Ambari, Mahadev Konar, Robert Levas, and Sumit Mohanty.
>
>
> Bugs: AMBARI-10411
> https://issues.apache.org/jira/browse/AMBARI-10411
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This patch addresses AMBARI-10411.
>
> The BlueprintConfigurationProcessor was not properly handling the dynamic property names for Yarn ResourceManager HA scenarios, in which some property names include the name of the specific resource manager. This would cause certain Yarn HA properties to contain hostname information in an exported Blueprint, which would make the Blueprint less portable across different cluster types.
>
> This patch fixes this issue by:
>
> 1. Adding some code to dynamically register the appropriate PropertyUpdater instances, based on the Yarn HA configuration specified in yarn-site.xml.
> 2. Adds a property updater for the "yarn.resourcemanager.zk-address", which was previously not considered by the BlueprintConfigurationProcessor. This allows this property to now be set automatically by the framework, and will also allow the Blueprint export process to properly mask the hostnames in this property.
> 3. Adds a property updater for "yarn.resourcemanager.webapp.https.address", for the same reasons listed above.
> 4. Adds a new unit test to verify this change.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java d7ff543
> ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java 6a7a79b
> ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java e0e79b3
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java bebc884
>
> Diff: https://reviews.apache.org/r/34334/diff/
>
>
> Testing
> -------
>
> 1. Ran the ambari-server unit tests, all passing.
> 2. Deployed a 3-node Yarn cluster with my patch applied, using the Ambari UI, enabled Yarn RM HA, and then exported the Blueprint. I verified that these properties are now masked, and no hostname information appears in the exported Blueprint.
> 3. Deployed a separate 3-node Yarn cluster with a Blueprint that does not set "yarn.resourcemanager.zk-address", and verified that the cluster started up properly. This verifies that the BlueprintConfigurationProcessor is properly updating the topology information to the list of Zookeeper servers in the cluster.
>
>
> Thanks,
>
> Robert Nettleton
>
>