You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Robert Nettleton <rn...@hortonworks.com> on 2014/10/30 01:48:26 UTC
Review Request 27366: Adds Blueprint export handling for Knox and
Zookeeper properties in HDP 2.2 stack
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27366/
-----------------------------------------------------------
Review request for Ambari, John Speidel and Nate Cole.
Repository: ambari
Description
-------
This patch implements a fix for AMBARI-8021.
Some new properties introduced in the HDP 2.2 stack are not
being handled properly during an export of a Blueprint from
a running cluster. These properties are all related to
either Zookeeper or Knox for several HDP 2.2 services
(Yarn, Kafka, Oozie, Slider, etc).
Currently, the Blueprint processor allows the hostname
information for these new properties to be included
in the exported Blueprint. This is incorrect, since
hostname information is not generally useful within
a Blueprint.
This patch addresses the problem by adding PropertyUpdater
handlers for each new property.
This patch also adds unit tests and new assertions to
existing unit tests in order to verify this change.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java be379c9
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java cf06064
Diff: https://reviews.apache.org/r/27366/diff/
Testing
-------
1. Ran the ambari-server unit test suite in trunk and 1.7.0, all tests are passing.
2. Manually verified the fix in trunk.
3. Manually verified the fix in 1.7.0.
Thanks,
Robert Nettleton
Re: Review Request 27366: Adds Blueprint export handling for Knox and
Zookeeper properties in HDP 2.2 stack
Posted by John Speidel <js...@hortonworks.com>.
> On Oct. 30, 2014, 3:09 p.m., John Speidel wrote:
> > Ship It!
merged to trunk and 1.7.0
All tests pass in both branches.
Results :
Tests run: 2126, Failures: 0, Errors: 0, Skipped: 15
...
Total run:679
Total errors:0
Total failures:0
OK
- John
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27366/#review59200
-----------------------------------------------------------
On Oct. 30, 2014, 12:48 a.m., Robert Nettleton wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27366/
> -----------------------------------------------------------
>
> (Updated Oct. 30, 2014, 12:48 a.m.)
>
>
> Review request for Ambari, John Speidel and Nate Cole.
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This patch implements a fix for AMBARI-8021.
>
> Some new properties introduced in the HDP 2.2 stack are not
> being handled properly during an export of a Blueprint from
> a running cluster. These properties are all related to
> either Zookeeper or Knox for several HDP 2.2 services
> (Yarn, Kafka, Oozie, Slider, etc).
>
> Currently, the Blueprint processor allows the hostname
> information for these new properties to be included
> in the exported Blueprint. This is incorrect, since
> hostname information is not generally useful within
> a Blueprint.
>
> This patch addresses the problem by adding PropertyUpdater
> handlers for each new property.
>
> This patch also adds unit tests and new assertions to
> existing unit tests in order to verify this change.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java be379c9
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java cf06064
>
> Diff: https://reviews.apache.org/r/27366/diff/
>
>
> Testing
> -------
>
> 1. Ran the ambari-server unit test suite in trunk and 1.7.0, all tests are passing.
> 2. Manually verified the fix in trunk.
> 3. Manually verified the fix in 1.7.0.
>
>
> Thanks,
>
> Robert Nettleton
>
>
Re: Review Request 27366: Adds Blueprint export handling for Knox and
Zookeeper properties in HDP 2.2 stack
Posted by John Speidel <js...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27366/#review59200
-----------------------------------------------------------
Ship it!
Ship It!
- John Speidel
On Oct. 30, 2014, 12:48 a.m., Robert Nettleton wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27366/
> -----------------------------------------------------------
>
> (Updated Oct. 30, 2014, 12:48 a.m.)
>
>
> Review request for Ambari, John Speidel and Nate Cole.
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This patch implements a fix for AMBARI-8021.
>
> Some new properties introduced in the HDP 2.2 stack are not
> being handled properly during an export of a Blueprint from
> a running cluster. These properties are all related to
> either Zookeeper or Knox for several HDP 2.2 services
> (Yarn, Kafka, Oozie, Slider, etc).
>
> Currently, the Blueprint processor allows the hostname
> information for these new properties to be included
> in the exported Blueprint. This is incorrect, since
> hostname information is not generally useful within
> a Blueprint.
>
> This patch addresses the problem by adding PropertyUpdater
> handlers for each new property.
>
> This patch also adds unit tests and new assertions to
> existing unit tests in order to verify this change.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java be379c9
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java cf06064
>
> Diff: https://reviews.apache.org/r/27366/diff/
>
>
> Testing
> -------
>
> 1. Ran the ambari-server unit test suite in trunk and 1.7.0, all tests are passing.
> 2. Manually verified the fix in trunk.
> 3. Manually verified the fix in 1.7.0.
>
>
> Thanks,
>
> Robert Nettleton
>
>
Re: Review Request 27366: Adds Blueprint export handling for Knox and
Zookeeper properties in HDP 2.2 stack
Posted by Nate Cole <nc...@hortonworks.com>.
> On Oct. 30, 2014, 7:22 a.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java, lines 1017-1019
> > <https://reviews.apache.org/r/27366/diff/1/?file=742190#file742190line1017>
> >
> > Just a general question since I can't see context: are tehse getting set even if, say, Slider is not installed? Are properties getting set if they don't belong?
>
> Robert Nettleton wrote:
> These map.put() calls basically register "updater" instances that are used to inspect a given cluster's configuration. These calls won't cause the actual properties to be set in the configuration, if they are not already there. These maps are used by the config processor to inspect the current configuration, and then handle any hostname substitution (during cluster setup) or hostname removal (during a Blueprint export).
>
> For example: If Slider is not installed, the updater registered for the "slider.zookeeper.quorum" property won't be used.
Oh, ok - that's great! Thanks for clarifying.
- Nate
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27366/#review59181
-----------------------------------------------------------
On Oct. 29, 2014, 8:48 p.m., Robert Nettleton wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27366/
> -----------------------------------------------------------
>
> (Updated Oct. 29, 2014, 8:48 p.m.)
>
>
> Review request for Ambari, John Speidel and Nate Cole.
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This patch implements a fix for AMBARI-8021.
>
> Some new properties introduced in the HDP 2.2 stack are not
> being handled properly during an export of a Blueprint from
> a running cluster. These properties are all related to
> either Zookeeper or Knox for several HDP 2.2 services
> (Yarn, Kafka, Oozie, Slider, etc).
>
> Currently, the Blueprint processor allows the hostname
> information for these new properties to be included
> in the exported Blueprint. This is incorrect, since
> hostname information is not generally useful within
> a Blueprint.
>
> This patch addresses the problem by adding PropertyUpdater
> handlers for each new property.
>
> This patch also adds unit tests and new assertions to
> existing unit tests in order to verify this change.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java be379c9
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java cf06064
>
> Diff: https://reviews.apache.org/r/27366/diff/
>
>
> Testing
> -------
>
> 1. Ran the ambari-server unit test suite in trunk and 1.7.0, all tests are passing.
> 2. Manually verified the fix in trunk.
> 3. Manually verified the fix in 1.7.0.
>
>
> Thanks,
>
> Robert Nettleton
>
>
Re: Review Request 27366: Adds Blueprint export handling for Knox and
Zookeeper properties in HDP 2.2 stack
Posted by Robert Nettleton <rn...@hortonworks.com>.
> On Oct. 30, 2014, 11:22 a.m., Nate Cole wrote:
> > When filling out review requests, please make sure you fill in the Branch and Bugs fields to make it easier to see where the patch is being applied.
Thanks for the review!
I've filled out the requested fields. I indicated two branches, since this will be merged to trunk and 1.7.0.
> On Oct. 30, 2014, 11:22 a.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java, lines 1017-1019
> > <https://reviews.apache.org/r/27366/diff/1/?file=742190#file742190line1017>
> >
> > Just a general question since I can't see context: are tehse getting set even if, say, Slider is not installed? Are properties getting set if they don't belong?
These map.put() calls basically register "updater" instances that are used to inspect a given cluster's configuration. These calls won't cause the actual properties to be set in the configuration, if they are not already there. These maps are used by the config processor to inspect the current configuration, and then handle any hostname substitution (during cluster setup) or hostname removal (during a Blueprint export).
For example: If Slider is not installed, the updater registered for the "slider.zookeeper.quorum" property won't be used.
- Robert
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27366/#review59181
-----------------------------------------------------------
On Oct. 30, 2014, 12:48 a.m., Robert Nettleton wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27366/
> -----------------------------------------------------------
>
> (Updated Oct. 30, 2014, 12:48 a.m.)
>
>
> Review request for Ambari, John Speidel and Nate Cole.
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This patch implements a fix for AMBARI-8021.
>
> Some new properties introduced in the HDP 2.2 stack are not
> being handled properly during an export of a Blueprint from
> a running cluster. These properties are all related to
> either Zookeeper or Knox for several HDP 2.2 services
> (Yarn, Kafka, Oozie, Slider, etc).
>
> Currently, the Blueprint processor allows the hostname
> information for these new properties to be included
> in the exported Blueprint. This is incorrect, since
> hostname information is not generally useful within
> a Blueprint.
>
> This patch addresses the problem by adding PropertyUpdater
> handlers for each new property.
>
> This patch also adds unit tests and new assertions to
> existing unit tests in order to verify this change.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java be379c9
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java cf06064
>
> Diff: https://reviews.apache.org/r/27366/diff/
>
>
> Testing
> -------
>
> 1. Ran the ambari-server unit test suite in trunk and 1.7.0, all tests are passing.
> 2. Manually verified the fix in trunk.
> 3. Manually verified the fix in 1.7.0.
>
>
> Thanks,
>
> Robert Nettleton
>
>
Re: Review Request 27366: Adds Blueprint export handling for Knox and
Zookeeper properties in HDP 2.2 stack
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27366/#review59181
-----------------------------------------------------------
Ship it!
When filling out review requests, please make sure you fill in the Branch and Bugs fields to make it easier to see where the patch is being applied.
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
<https://reviews.apache.org/r/27366/#comment100475>
Just a general question since I can't see context: are tehse getting set even if, say, Slider is not installed? Are properties getting set if they don't belong?
- Nate Cole
On Oct. 29, 2014, 8:48 p.m., Robert Nettleton wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27366/
> -----------------------------------------------------------
>
> (Updated Oct. 29, 2014, 8:48 p.m.)
>
>
> Review request for Ambari, John Speidel and Nate Cole.
>
>
> Repository: ambari
>
>
> Description
> -------
>
> This patch implements a fix for AMBARI-8021.
>
> Some new properties introduced in the HDP 2.2 stack are not
> being handled properly during an export of a Blueprint from
> a running cluster. These properties are all related to
> either Zookeeper or Knox for several HDP 2.2 services
> (Yarn, Kafka, Oozie, Slider, etc).
>
> Currently, the Blueprint processor allows the hostname
> information for these new properties to be included
> in the exported Blueprint. This is incorrect, since
> hostname information is not generally useful within
> a Blueprint.
>
> This patch addresses the problem by adding PropertyUpdater
> handlers for each new property.
>
> This patch also adds unit tests and new assertions to
> existing unit tests in order to verify this change.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java be379c9
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java cf06064
>
> Diff: https://reviews.apache.org/r/27366/diff/
>
>
> Testing
> -------
>
> 1. Ran the ambari-server unit test suite in trunk and 1.7.0, all tests are passing.
> 2. Manually verified the fix in trunk.
> 3. Manually verified the fix in 1.7.0.
>
>
> Thanks,
>
> Robert Nettleton
>
>