You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Robert Nettleton <rn...@hortonworks.com> on 2015/04/20 17:04:49 UTC
Review Request 33360: Fix Blueprint configuration processor to handle
FQDN properties for HA clusters
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33360/
-----------------------------------------------------------
Review request for Ambari, John Speidel, Mahadev Konar, and Sumit Mohanty.
Bugs: AMBARI-10487
https://issues.apache.org/jira/browse/AMBARI-10487
Repository: ambari
Description
-------
This patch implements a fix for AMBARI-10487.
The BlueprintConfigurationProcessor was incorrectly throwing an exception while processing some HDFS and Yarn properties when an HA cluster was being deployed.
The custom handling logic in the config processor was incorrectly throwing an error when more than one instance of an HA-capable service was included in the HA cluster. In particular, the processor was not accepting FQDN hostnames as valid values for HDFS and Yarn properties.
This patch adddresses this problem by:
1. Adding handling code in the SingleHostTopologyUpdater to address this issue in HA scenarios for HDFS, Yarn, and Oozie deployments.
2. Modifying the MultipleHostPropertyUpdater to return the original property value if the property does not match any patterns that require hostname substitution.
3. Updating the internal PropertyUpdater interface to pass in the name of the property being processed. This was done mainly to facilitate simpler debug/error messages, since previously the updater threw an exception without mentioning the actual property involved in the processing failure.
4. Adding unit tests to verify the fixes in this patch.
In some future release of Ambari, we should probably build an HA-aware property updater, but at this point it makes more sense to handle this in the existing updaters, in order to support FQDNs as quickly as possible.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java 8247c63
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 1839a5f
Diff: https://reviews.apache.org/r/33360/diff/
Testing
-------
1. Ran the ambari-server unit tests.
2. Deployed a 3 node HDFS NameNode HA cluster, using a Blueprint that only included FQDN hostnames as property values.
3. Deployed a 3 node HDFS NameNode HA cluster, using a Blueprint that included the usage of the %HOSTGROUP% substitution syntax.
4. Deployed a 3 node Yarn ResourceManager HA cluster, using a Blueprint that only included FQDN hostnames as property values.
5. Deployed a 3 node Yarn ResourceManager HA cluster, using a Blueprint that included the usage of the %HOSTGROUP% substitution syntax in certain property values (HOSTGROUP support for remaining Yarn properties to be added in a subsequent patch).
Thanks,
Robert Nettleton
Re: Review Request 33360: Fix Blueprint configuration processor to
handle FQDN properties for HA clusters
Posted by Mahadev Konar <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33360/#review80791
-----------------------------------------------------------
Ship it!
Ship It!
- Mahadev Konar
On April 20, 2015, 3:04 p.m., Robert Nettleton wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33360/
> -----------------------------------------------------------
>
> (Updated April 20, 2015, 3:04 p.m.)
>
>
> Review request for Ambari, John Speidel, Mahadev Konar, and Sumit Mohanty.
>
>
> Bugs: AMBARI-10487
> https://issues.apache.org/jira/browse/AMBARI-10487
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This patch implements a fix for AMBARI-10487.
>
> The BlueprintConfigurationProcessor was incorrectly throwing an exception while processing some HDFS and Yarn properties when an HA cluster was being deployed.
>
> The custom handling logic in the config processor was incorrectly throwing an error when more than one instance of an HA-capable service was included in the HA cluster. In particular, the processor was not accepting FQDN hostnames as valid values for HDFS and Yarn properties.
>
> This patch adddresses this problem by:
>
> 1. Adding handling code in the SingleHostTopologyUpdater to address this issue in HA scenarios for HDFS, Yarn, and Oozie deployments.
> 2. Modifying the MultipleHostPropertyUpdater to return the original property value if the property does not match any patterns that require hostname substitution.
> 3. Updating the internal PropertyUpdater interface to pass in the name of the property being processed. This was done mainly to facilitate simpler debug/error messages, since previously the updater threw an exception without mentioning the actual property involved in the processing failure.
> 4. Adding unit tests to verify the fixes in this patch.
>
>
> In some future release of Ambari, we should probably build an HA-aware property updater, but at this point it makes more sense to handle this in the existing updaters, in order to support FQDNs as quickly as possible.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java 8247c63
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 1839a5f
>
> Diff: https://reviews.apache.org/r/33360/diff/
>
>
> Testing
> -------
>
> 1. Ran the ambari-server unit tests.
> 2. Deployed a 3 node HDFS NameNode HA cluster, using a Blueprint that only included FQDN hostnames as property values.
> 3. Deployed a 3 node HDFS NameNode HA cluster, using a Blueprint that included the usage of the %HOSTGROUP% substitution syntax.
> 4. Deployed a 3 node Yarn ResourceManager HA cluster, using a Blueprint that only included FQDN hostnames as property values.
> 5. Deployed a 3 node Yarn ResourceManager HA cluster, using a Blueprint that included the usage of the %HOSTGROUP% substitution syntax in certain property values (HOSTGROUP support for remaining Yarn properties to be added in a subsequent patch).
>
>
> Thanks,
>
> Robert Nettleton
>
>
Re: Review Request 33360: Fix Blueprint configuration processor to
handle FQDN properties for HA clusters
Posted by John Speidel <js...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33360/#review80754
-----------------------------------------------------------
Ship it!
Ship It!
- John Speidel
On April 20, 2015, 3:04 p.m., Robert Nettleton wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33360/
> -----------------------------------------------------------
>
> (Updated April 20, 2015, 3:04 p.m.)
>
>
> Review request for Ambari, John Speidel, Mahadev Konar, and Sumit Mohanty.
>
>
> Bugs: AMBARI-10487
> https://issues.apache.org/jira/browse/AMBARI-10487
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This patch implements a fix for AMBARI-10487.
>
> The BlueprintConfigurationProcessor was incorrectly throwing an exception while processing some HDFS and Yarn properties when an HA cluster was being deployed.
>
> The custom handling logic in the config processor was incorrectly throwing an error when more than one instance of an HA-capable service was included in the HA cluster. In particular, the processor was not accepting FQDN hostnames as valid values for HDFS and Yarn properties.
>
> This patch adddresses this problem by:
>
> 1. Adding handling code in the SingleHostTopologyUpdater to address this issue in HA scenarios for HDFS, Yarn, and Oozie deployments.
> 2. Modifying the MultipleHostPropertyUpdater to return the original property value if the property does not match any patterns that require hostname substitution.
> 3. Updating the internal PropertyUpdater interface to pass in the name of the property being processed. This was done mainly to facilitate simpler debug/error messages, since previously the updater threw an exception without mentioning the actual property involved in the processing failure.
> 4. Adding unit tests to verify the fixes in this patch.
>
>
> In some future release of Ambari, we should probably build an HA-aware property updater, but at this point it makes more sense to handle this in the existing updaters, in order to support FQDNs as quickly as possible.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java 8247c63
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java 1839a5f
>
> Diff: https://reviews.apache.org/r/33360/diff/
>
>
> Testing
> -------
>
> 1. Ran the ambari-server unit tests.
> 2. Deployed a 3 node HDFS NameNode HA cluster, using a Blueprint that only included FQDN hostnames as property values.
> 3. Deployed a 3 node HDFS NameNode HA cluster, using a Blueprint that included the usage of the %HOSTGROUP% substitution syntax.
> 4. Deployed a 3 node Yarn ResourceManager HA cluster, using a Blueprint that only included FQDN hostnames as property values.
> 5. Deployed a 3 node Yarn ResourceManager HA cluster, using a Blueprint that included the usage of the %HOSTGROUP% substitution syntax in certain property values (HOSTGROUP support for remaining Yarn properties to be added in a subsequent patch).
>
>
> Thanks,
>
> Robert Nettleton
>
>