You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Mugdha Varadkar <mu...@freestoneinfotech.com> on 2016/11/29 17:42:33 UTC
Review Request 54169: AMBARI-19007 Atlas to support configuration of
hooks from separate cluster
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54169/
-----------------------------------------------------------
Review request for Ambari, Alejandro Fernandez, Gautam Borad, Madhan Neethiraj, and Sumit Mohanty.
Bugs: AMBARI-19007
https://issues.apache.org/jira/browse/AMBARI-19007
Repository: ambari
Description
-------
1) Introduced new flag for all the hook suported components(HIVE,STORM,SQOOP,FALCON)under there respective env config-type.
2) If ATLAS service is present/selected for install, stack-advisor will set the hook flag. This flag value is used to recommend the expected configuration required for hook to work.
3) For Blue-print based installation user need to set hook flag and add common-atlas application properties under each hook config-type eg: hive-atlas-application.properties config-type which are require to communicate with external ATLAS
Diffs
-----
ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py a1d2f95
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java ec846f8
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-env.xml 09cced6
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py 4429253
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py 0fb21d0
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml 150f629
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py 5e2c709
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py bcc598a
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 01e5f00
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py fe3f34a
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-env.xml f682e97
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml f7823d2
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py 283f54d
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py 68f06db
ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml 9547335
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py e753e98
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py bda4fe2
ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml ab350dc
ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/configuration/storm-site.xml b71f4a9
ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 775dbab
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java f2dd099
ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 39cbbce
Diff: https://reviews.apache.org/r/54169/diff/
Testing
-------
Tested Atlas installation via UI and blue-print. With blueprint used config_recommendation_strategy as NEVER_APPLY and ALWAYS_APPLY_DONT_OVERRIDE_CUSTOM_VALUES
mvn -DskipPythonTests -Dtest=BlueprintConfigurationProcessorTest test
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Running org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
Tests run: 167, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.101 sec - in org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
Results :
Tests run: 167, Failures: 0, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 24.484s
[INFO] Finished at: Tue Nov 29 11:40:30 UTC 2016
[INFO] Final Memory: 62M/797M
[INFO] ------------------------------------------------------------------------
Thanks,
Mugdha Varadkar
Re: Review Request 54169: AMBARI-19007 Atlas to support configuration
of hooks from separate cluster
Posted by Alejandro Fernandez <af...@hortonworks.com>.
> On Nov. 29, 2016, 6:53 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml, line 32
> > <https://reviews.apache.org/r/54169/diff/1/?file=1572153#file1572153line32>
> >
> > This property will be added automatically after an Ambari Upgrade with the value "false". Should this be smarter to detect if Atlas and Storm are in the cluster and if so then set the value to true?
>
> Mugdha Varadkar wrote:
> Hi Alejandro,
> Yes there is a way this can be handled in UpgradeCatalog250.java. This fix will is getting handled in https://issues.apache.org/jira/browse/AMBARI-19024
Mugdha, a lot of the items I brought up are being deferred to another Jira. In my opinion, they should be done as part of this commit. Typically, it's ok to create another Jira for additional functionality, but not very common to fix more issues with the current patch. Thanks.
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54169/#review157296
-----------------------------------------------------------
On Nov. 29, 2016, 5:42 p.m., Mugdha Varadkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54169/
> -----------------------------------------------------------
>
> (Updated Nov. 29, 2016, 5:42 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Madhan Neethiraj, and Sumit Mohanty.
>
>
> Bugs: AMBARI-19007
> https://issues.apache.org/jira/browse/AMBARI-19007
>
>
> Repository: ambari
>
>
> Description
> -------
>
> 1) Introduced new flag for all the hook suported components(HIVE,STORM,SQOOP,FALCON)under there respective env config-type.
> 2) If ATLAS service is present/selected for install, stack-advisor will set the hook flag. This flag value is used to recommend the expected configuration required for hook to work.
> 3) For Blue-print based installation user need to set hook flag and add common-atlas application properties under each hook config-type eg: hive-atlas-application.properties config-type which are require to communicate with external ATLAS
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py a1d2f95
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java ec846f8
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-env.xml 09cced6
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py 4429253
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py 0fb21d0
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml 150f629
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py 5e2c709
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py bcc598a
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 01e5f00
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py fe3f34a
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-env.xml f682e97
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml f7823d2
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py 283f54d
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py 68f06db
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml 9547335
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py e753e98
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py bda4fe2
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml ab350dc
> ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/configuration/storm-site.xml b71f4a9
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 775dbab
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java f2dd099
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 39cbbce
>
> Diff: https://reviews.apache.org/r/54169/diff/
>
>
> Testing
> -------
>
> Tested Atlas installation via UI and blue-print. With blueprint used config_recommendation_strategy as NEVER_APPLY and ALWAYS_APPLY_DONT_OVERRIDE_CUSTOM_VALUES
>
>
> mvn -DskipPythonTests -Dtest=BlueprintConfigurationProcessorTest test
>
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
> Running org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
> Tests run: 167, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.101 sec - in org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
>
> Results :
>
> Tests run: 167, Failures: 0, Errors: 0, Skipped: 0
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 24.484s
> [INFO] Finished at: Tue Nov 29 11:40:30 UTC 2016
> [INFO] Final Memory: 62M/797M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Mugdha Varadkar
>
>
Re: Review Request 54169: AMBARI-19007 Atlas to support configuration
of hooks from separate cluster
Posted by Mugdha Varadkar <mu...@freestoneinfotech.com>.
> On Nov. 29, 2016, 6:53 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml, line 32
> > <https://reviews.apache.org/r/54169/diff/1/?file=1572153#file1572153line32>
> >
> > This property will be added automatically after an Ambari Upgrade with the value "false". Should this be smarter to detect if Atlas and Storm are in the cluster and if so then set the value to true?
Hi Alejandro,
Yes there is a way this can be handled in UpgradeCatalog250.java. This fix will is getting handled in https://issues.apache.org/jira/browse/AMBARI-19024
> On Nov. 29, 2016, 6:53 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml, line 24
> > <https://reviews.apache.org/r/54169/diff/1/?file=1572153#file1572153line24>
> >
> > If a user uses the latest version of Ambari, should this property exist in all versions of HDP (2.3 - 2.6)? If so, then putting it in common-services is ok.
For Storm, this property will be availabe from HDP 2.3
HDP-2.3 extends STORM/0.10.0 version and ATLAS service is available from HDP-2.3
> On Nov. 29, 2016, 6:53 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-env.xml, line 25
> > <https://reviews.apache.org/r/54169/diff/1/?file=1572149#file1572149line25>
> >
> > Same comments regarding storm.atlas.hook property on Ambari upgrade. What should the behavior be if Sqoop and Atlas are both on the cluster?
This fix will is getting handled in https://issues.apache.org/jira/browse/AMBARI-19024
> On Nov. 29, 2016, 6:53 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml, line 24
> > <https://reviews.apache.org/r/54169/diff/1/?file=1572144#file1572144line24>
> >
> > Same comments regarding storm.atlas.hook property on Ambari upgrade. What should the behavior be if Sqoop and Atlas are both on the cluster?
This fix will is getting handled in https://issues.apache.org/jira/browse/AMBARI-19024
> On Nov. 29, 2016, 6:53 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-env.xml, line 24
> > <https://reviews.apache.org/r/54169/diff/1/?file=1572141#file1572141line24>
> >
> > Same comments regarding storm.atlas.hook property on Ambari upgrade. What should the behavior be if Sqoop and Atlas are both on the cluster?
This fix will is getting handled in https://issues.apache.org/jira/browse/AMBARI-19024
- Mugdha
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54169/#review157296
-----------------------------------------------------------
On Nov. 29, 2016, 5:42 p.m., Mugdha Varadkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54169/
> -----------------------------------------------------------
>
> (Updated Nov. 29, 2016, 5:42 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Madhan Neethiraj, and Sumit Mohanty.
>
>
> Bugs: AMBARI-19007
> https://issues.apache.org/jira/browse/AMBARI-19007
>
>
> Repository: ambari
>
>
> Description
> -------
>
> 1) Introduced new flag for all the hook suported components(HIVE,STORM,SQOOP,FALCON)under there respective env config-type.
> 2) If ATLAS service is present/selected for install, stack-advisor will set the hook flag. This flag value is used to recommend the expected configuration required for hook to work.
> 3) For Blue-print based installation user need to set hook flag and add common-atlas application properties under each hook config-type eg: hive-atlas-application.properties config-type which are require to communicate with external ATLAS
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py a1d2f95
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java ec846f8
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-env.xml 09cced6
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py 4429253
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py 0fb21d0
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml 150f629
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py 5e2c709
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py bcc598a
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 01e5f00
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py fe3f34a
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-env.xml f682e97
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml f7823d2
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py 283f54d
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py 68f06db
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml 9547335
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py e753e98
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py bda4fe2
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml ab350dc
> ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/configuration/storm-site.xml b71f4a9
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 775dbab
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java f2dd099
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 39cbbce
>
> Diff: https://reviews.apache.org/r/54169/diff/
>
>
> Testing
> -------
>
> Tested Atlas installation via UI and blue-print. With blueprint used config_recommendation_strategy as NEVER_APPLY and ALWAYS_APPLY_DONT_OVERRIDE_CUSTOM_VALUES
>
>
> mvn -DskipPythonTests -Dtest=BlueprintConfigurationProcessorTest test
>
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
> Running org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
> Tests run: 167, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.101 sec - in org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
>
> Results :
>
> Tests run: 167, Failures: 0, Errors: 0, Skipped: 0
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 24.484s
> [INFO] Finished at: Tue Nov 29 11:40:30 UTC 2016
> [INFO] Final Memory: 62M/797M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Mugdha Varadkar
>
>
Re: Review Request 54169: AMBARI-19007 Atlas to support configuration
of hooks from separate cluster
Posted by Mugdha Varadkar <mu...@freestoneinfotech.com>.
> On Nov. 29, 2016, 6:53 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml, line 32
> > <https://reviews.apache.org/r/54169/diff/1/?file=1572153#file1572153line32>
> >
> > This property will be added automatically after an Ambari Upgrade with the value "false". Should this be smarter to detect if Atlas and Storm are in the cluster and if so then set the value to true?
>
> Mugdha Varadkar wrote:
> Hi Alejandro,
> Yes there is a way this can be handled in UpgradeCatalog250.java. This fix will is getting handled in https://issues.apache.org/jira/browse/AMBARI-19024
>
> Alejandro Fernandez wrote:
> Mugdha, a lot of the items I brought up are being deferred to another Jira. In my opinion, they should be done as part of this commit. Typically, it's ok to create another Jira for additional functionality, but not very common to fix more issues with the current patch. Thanks.
Updated in latest patch
- Mugdha
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54169/#review157296
-----------------------------------------------------------
On Dec. 1, 2016, 6:59 a.m., Mugdha Varadkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54169/
> -----------------------------------------------------------
>
> (Updated Dec. 1, 2016, 6:59 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Madhan Neethiraj, and Sumit Mohanty.
>
>
> Bugs: AMBARI-19007
> https://issues.apache.org/jira/browse/AMBARI-19007
>
>
> Repository: ambari
>
>
> Description
> -------
>
> 1) Introduced new flag for all the hook suported components(HIVE,STORM,SQOOP,FALCON)under there respective env config-type.
> 2) If ATLAS service is present/selected for install, stack-advisor will set the hook flag. This flag value is used to recommend the expected configuration required for hook to work.
> 3) For Blue-print based installation user need to set hook flag and add common-atlas application properties under each hook config-type eg: hive-atlas-application.properties config-type which are require to communicate with external ATLAS
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py a1d2f95
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java ec846f8
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java 52de784
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-env.xml 09cced6
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py 4429253
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py 0fb21d0
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml 150f629
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py 5e2c709
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py bcc598a
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 01e5f00
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py fe3f34a
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-env.xml f682e97
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml f7823d2
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py 283f54d
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py 68f06db
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml 9547335
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py e753e98
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py bda4fe2
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml ab350dc
> ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/configuration/storm-site.xml b71f4a9
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 775dbab
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java f2dd099
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java ce0b387
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 39cbbce
>
> Diff: https://reviews.apache.org/r/54169/diff/
>
>
> Testing
> -------
>
> Tested Atlas installation via UI and blue-print. With blueprint used config_recommendation_strategy as NEVER_APPLY and ALWAYS_APPLY_DONT_OVERRIDE_CUSTOM_VALUES
>
>
> mvn -DskipPythonTests -Dtest=BlueprintConfigurationProcessorTest test
>
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
> Running org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
> Tests run: 167, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.101 sec - in org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
>
> Results :
>
> Tests run: 167, Failures: 0, Errors: 0, Skipped: 0
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 24.484s
> [INFO] Finished at: Tue Nov 29 11:40:30 UTC 2016
> [INFO] Final Memory: 62M/797M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Mugdha Varadkar
>
>
Re: Review Request 54169: AMBARI-19007 Atlas to support configuration
of hooks from separate cluster
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54169/#review157296
-----------------------------------------------------------
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-env.xml (line 24)
<https://reviews.apache.org/r/54169/#comment227839>
Same comments regarding storm.atlas.hook property on Ambari upgrade. What should the behavior be if Sqoop and Atlas are both on the cluster?
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml (line 24)
<https://reviews.apache.org/r/54169/#comment227838>
Same comments regarding storm.atlas.hook property on Ambari upgrade. What should the behavior be if Sqoop and Atlas are both on the cluster?
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-env.xml (line 25)
<https://reviews.apache.org/r/54169/#comment227837>
Same comments regarding storm.atlas.hook property on Ambari upgrade. What should the behavior be if Sqoop and Atlas are both on the cluster?
ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml (line 24)
<https://reviews.apache.org/r/54169/#comment227832>
If a user uses the latest version of Ambari, should this property exist in all versions of HDP (2.3 - 2.6)? If so, then putting it in common-services is ok.
ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml (line 32)
<https://reviews.apache.org/r/54169/#comment227836>
This property will be added automatically after an Ambari Upgrade with the value "false". Should this be smarter to detect if Atlas and Storm are in the cluster and if so then set the value to true?
- Alejandro Fernandez
On Nov. 29, 2016, 5:42 p.m., Mugdha Varadkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54169/
> -----------------------------------------------------------
>
> (Updated Nov. 29, 2016, 5:42 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Madhan Neethiraj, and Sumit Mohanty.
>
>
> Bugs: AMBARI-19007
> https://issues.apache.org/jira/browse/AMBARI-19007
>
>
> Repository: ambari
>
>
> Description
> -------
>
> 1) Introduced new flag for all the hook suported components(HIVE,STORM,SQOOP,FALCON)under there respective env config-type.
> 2) If ATLAS service is present/selected for install, stack-advisor will set the hook flag. This flag value is used to recommend the expected configuration required for hook to work.
> 3) For Blue-print based installation user need to set hook flag and add common-atlas application properties under each hook config-type eg: hive-atlas-application.properties config-type which are require to communicate with external ATLAS
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py a1d2f95
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java ec846f8
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-env.xml 09cced6
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py 4429253
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py 0fb21d0
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml 150f629
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py 5e2c709
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py bcc598a
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 01e5f00
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py fe3f34a
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-env.xml f682e97
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml f7823d2
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py 283f54d
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py 68f06db
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml 9547335
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py e753e98
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py bda4fe2
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml ab350dc
> ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/configuration/storm-site.xml b71f4a9
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 775dbab
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java f2dd099
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 39cbbce
>
> Diff: https://reviews.apache.org/r/54169/diff/
>
>
> Testing
> -------
>
> Tested Atlas installation via UI and blue-print. With blueprint used config_recommendation_strategy as NEVER_APPLY and ALWAYS_APPLY_DONT_OVERRIDE_CUSTOM_VALUES
>
>
> mvn -DskipPythonTests -Dtest=BlueprintConfigurationProcessorTest test
>
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
> Running org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
> Tests run: 167, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.101 sec - in org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
>
> Results :
>
> Tests run: 167, Failures: 0, Errors: 0, Skipped: 0
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 24.484s
> [INFO] Finished at: Tue Nov 29 11:40:30 UTC 2016
> [INFO] Final Memory: 62M/797M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Mugdha Varadkar
>
>
Re: Review Request 54169: AMBARI-19007 Atlas to support configuration
of hooks from separate cluster
Posted by Mugdha Varadkar <mu...@freestoneinfotech.com>.
> On Dec. 1, 2016, 5:53 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java, line 329
> > <https://reviews.apache.org/r/54169/diff/2/?file=1573671#file1573671line329>
> >
> > Sorry to nitpick here, these 4 functions are nearly identical. Can you parametrize the service and config type?
Updated in latest patch
- Mugdha
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54169/#review157617
-----------------------------------------------------------
On Dec. 2, 2016, 1:01 p.m., Mugdha Varadkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54169/
> -----------------------------------------------------------
>
> (Updated Dec. 2, 2016, 1:01 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Madhan Neethiraj, and Sumit Mohanty.
>
>
> Bugs: AMBARI-19007
> https://issues.apache.org/jira/browse/AMBARI-19007
>
>
> Repository: ambari
>
>
> Description
> -------
>
> 1) Introduced new flag for all the hook suported components(HIVE,STORM,SQOOP,FALCON)under there respective env config-type.
> 2) If ATLAS service is present/selected for install, stack-advisor will set the hook flag. This flag value is used to recommend the expected configuration required for hook to work.
> 3) For Blue-print based installation user need to set hook flag and add common-atlas application properties under each hook config-type eg: hive-atlas-application.properties config-type which are require to communicate with external ATLAS
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py a1d2f95
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java ec846f8
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java 52de784
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-env.xml 09cced6
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py 4429253
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py 0fb21d0
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml 150f629
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py 5e2c709
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py bcc598a
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 01e5f00
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py fe3f34a
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-env.xml f682e97
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml f7823d2
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py 283f54d
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py 68f06db
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml 9547335
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py aca0681
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py bda4fe2
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml ab350dc
> ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/configuration/storm-site.xml b71f4a9
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 775dbab
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java f2dd099
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java ce0b387
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 39cbbce
>
> Diff: https://reviews.apache.org/r/54169/diff/
>
>
> Testing
> -------
>
> Tested Atlas installation via UI and blue-print. With blueprint used config_recommendation_strategy as NEVER_APPLY and ALWAYS_APPLY_DONT_OVERRIDE_CUSTOM_VALUES
>
>
> mvn -DskipPythonTests -Dtest=BlueprintConfigurationProcessorTest test
>
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
> Running org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
> Tests run: 167, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.101 sec - in org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
>
> Results :
>
> Tests run: 167, Failures: 0, Errors: 0, Skipped: 0
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 24.484s
> [INFO] Finished at: Tue Nov 29 11:40:30 UTC 2016
> [INFO] Final Memory: 62M/797M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Mugdha Varadkar
>
>
Re: Review Request 54169: AMBARI-19007 Atlas to support configuration
of hooks from separate cluster
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54169/#review157617
-----------------------------------------------------------
Fix it, then Ship it!
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java (line 329)
<https://reviews.apache.org/r/54169/#comment228226>
Sorry to nitpick here, these 4 functions are nearly identical. Can you parametrize the service and config type?
- Alejandro Fernandez
On Dec. 1, 2016, 6:59 a.m., Mugdha Varadkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54169/
> -----------------------------------------------------------
>
> (Updated Dec. 1, 2016, 6:59 a.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Madhan Neethiraj, and Sumit Mohanty.
>
>
> Bugs: AMBARI-19007
> https://issues.apache.org/jira/browse/AMBARI-19007
>
>
> Repository: ambari
>
>
> Description
> -------
>
> 1) Introduced new flag for all the hook suported components(HIVE,STORM,SQOOP,FALCON)under there respective env config-type.
> 2) If ATLAS service is present/selected for install, stack-advisor will set the hook flag. This flag value is used to recommend the expected configuration required for hook to work.
> 3) For Blue-print based installation user need to set hook flag and add common-atlas application properties under each hook config-type eg: hive-atlas-application.properties config-type which are require to communicate with external ATLAS
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py a1d2f95
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java ec846f8
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java 52de784
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-env.xml 09cced6
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py 4429253
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py 0fb21d0
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml 150f629
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py 5e2c709
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py bcc598a
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 01e5f00
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py fe3f34a
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-env.xml f682e97
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml f7823d2
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py 283f54d
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py 68f06db
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml 9547335
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py e753e98
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py bda4fe2
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml ab350dc
> ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/configuration/storm-site.xml b71f4a9
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 775dbab
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java f2dd099
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java ce0b387
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 39cbbce
>
> Diff: https://reviews.apache.org/r/54169/diff/
>
>
> Testing
> -------
>
> Tested Atlas installation via UI and blue-print. With blueprint used config_recommendation_strategy as NEVER_APPLY and ALWAYS_APPLY_DONT_OVERRIDE_CUSTOM_VALUES
>
>
> mvn -DskipPythonTests -Dtest=BlueprintConfigurationProcessorTest test
>
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
> Running org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
> Tests run: 167, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.101 sec - in org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
>
> Results :
>
> Tests run: 167, Failures: 0, Errors: 0, Skipped: 0
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 24.484s
> [INFO] Finished at: Tue Nov 29 11:40:30 UTC 2016
> [INFO] Final Memory: 62M/797M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Mugdha Varadkar
>
>
Re: Review Request 54169: AMBARI-19007 Atlas to support configuration
of hooks from separate cluster
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54169/#review157776
-----------------------------------------------------------
Ship it!
Ship It!
- Alejandro Fernandez
On Dec. 2, 2016, 1:01 p.m., Mugdha Varadkar wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54169/
> -----------------------------------------------------------
>
> (Updated Dec. 2, 2016, 1:01 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Madhan Neethiraj, and Sumit Mohanty.
>
>
> Bugs: AMBARI-19007
> https://issues.apache.org/jira/browse/AMBARI-19007
>
>
> Repository: ambari
>
>
> Description
> -------
>
> 1) Introduced new flag for all the hook suported components(HIVE,STORM,SQOOP,FALCON)under there respective env config-type.
> 2) If ATLAS service is present/selected for install, stack-advisor will set the hook flag. This flag value is used to recommend the expected configuration required for hook to work.
> 3) For Blue-print based installation user need to set hook flag and add common-atlas application properties under each hook config-type eg: hive-atlas-application.properties config-type which are require to communicate with external ATLAS
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py a1d2f95
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java ec846f8
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java 52de784
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-env.xml 09cced6
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py 4429253
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py 0fb21d0
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml 150f629
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py 5e2c709
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py bcc598a
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 01e5f00
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py fe3f34a
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-env.xml f682e97
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml f7823d2
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py 283f54d
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py 68f06db
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml 9547335
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py aca0681
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py bda4fe2
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml ab350dc
> ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/configuration/storm-site.xml b71f4a9
> ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 775dbab
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java f2dd099
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java ce0b387
> ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 39cbbce
>
> Diff: https://reviews.apache.org/r/54169/diff/
>
>
> Testing
> -------
>
> Tested Atlas installation via UI and blue-print. With blueprint used config_recommendation_strategy as NEVER_APPLY and ALWAYS_APPLY_DONT_OVERRIDE_CUSTOM_VALUES
>
>
> mvn -DskipPythonTests -Dtest=BlueprintConfigurationProcessorTest test
>
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
> Running org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
> Tests run: 167, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.101 sec - in org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
>
> Results :
>
> Tests run: 167, Failures: 0, Errors: 0, Skipped: 0
>
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 24.484s
> [INFO] Finished at: Tue Nov 29 11:40:30 UTC 2016
> [INFO] Final Memory: 62M/797M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Mugdha Varadkar
>
>
Re: Review Request 54169: AMBARI-19007 Atlas to support configuration
of hooks from separate cluster
Posted by Mugdha Varadkar <mu...@freestoneinfotech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54169/
-----------------------------------------------------------
(Updated Dec. 2, 2016, 1:01 p.m.)
Review request for Ambari, Alejandro Fernandez, Gautam Borad, Madhan Neethiraj, and Sumit Mohanty.
Bugs: AMBARI-19007
https://issues.apache.org/jira/browse/AMBARI-19007
Repository: ambari
Description
-------
1) Introduced new flag for all the hook suported components(HIVE,STORM,SQOOP,FALCON)under there respective env config-type.
2) If ATLAS service is present/selected for install, stack-advisor will set the hook flag. This flag value is used to recommend the expected configuration required for hook to work.
3) For Blue-print based installation user need to set hook flag and add common-atlas application properties under each hook config-type eg: hive-atlas-application.properties config-type which are require to communicate with external ATLAS
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py a1d2f95
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java ec846f8
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java 52de784
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-env.xml 09cced6
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py 4429253
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py 0fb21d0
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml 150f629
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py 5e2c709
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py bcc598a
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 01e5f00
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py fe3f34a
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-env.xml f682e97
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml f7823d2
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py 283f54d
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py 68f06db
ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml 9547335
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py aca0681
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py bda4fe2
ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml ab350dc
ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/configuration/storm-site.xml b71f4a9
ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 775dbab
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java f2dd099
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java ce0b387
ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 39cbbce
Diff: https://reviews.apache.org/r/54169/diff/
Testing
-------
Tested Atlas installation via UI and blue-print. With blueprint used config_recommendation_strategy as NEVER_APPLY and ALWAYS_APPLY_DONT_OVERRIDE_CUSTOM_VALUES
mvn -DskipPythonTests -Dtest=BlueprintConfigurationProcessorTest test
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Running org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
Tests run: 167, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.101 sec - in org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
Results :
Tests run: 167, Failures: 0, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 24.484s
[INFO] Finished at: Tue Nov 29 11:40:30 UTC 2016
[INFO] Final Memory: 62M/797M
[INFO] ------------------------------------------------------------------------
Thanks,
Mugdha Varadkar
Re: Review Request 54169: AMBARI-19007 Atlas to support configuration
of hooks from separate cluster
Posted by Mugdha Varadkar <mu...@freestoneinfotech.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54169/
-----------------------------------------------------------
(Updated Dec. 1, 2016, 6:59 a.m.)
Review request for Ambari, Alejandro Fernandez, Gautam Borad, Madhan Neethiraj, and Sumit Mohanty.
Changes
-------
Addressed AMBARI-19024(Ambari upgrade scenario) fix in latest patch
Bugs: AMBARI-19007
https://issues.apache.org/jira/browse/AMBARI-19007
Repository: ambari
Description
-------
1) Introduced new flag for all the hook suported components(HIVE,STORM,SQOOP,FALCON)under there respective env config-type.
2) If ATLAS service is present/selected for install, stack-advisor will set the hook flag. This flag value is used to recommend the expected configuration required for hook to work.
3) For Blue-print based installation user need to set hook flag and add common-atlas application properties under each hook config-type eg: hive-atlas-application.properties config-type which are require to communicate with external ATLAS
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py a1d2f95
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java ec846f8
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java 52de784
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-env.xml 09cced6
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py 4429253
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py 0fb21d0
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml 150f629
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py 5e2c709
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py bcc598a
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 01e5f00
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py fe3f34a
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-env.xml f682e97
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml f7823d2
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py 283f54d
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py 68f06db
ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml 9547335
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py e753e98
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py bda4fe2
ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml ab350dc
ambari-server/src/main/resources/stacks/HDP/2.3/services/STORM/configuration/storm-site.xml b71f4a9
ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 775dbab
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java f2dd099
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java ce0b387
ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 39cbbce
Diff: https://reviews.apache.org/r/54169/diff/
Testing
-------
Tested Atlas installation via UI and blue-print. With blueprint used config_recommendation_strategy as NEVER_APPLY and ALWAYS_APPLY_DONT_OVERRIDE_CUSTOM_VALUES
mvn -DskipPythonTests -Dtest=BlueprintConfigurationProcessorTest test
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true
Running org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
Tests run: 167, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.101 sec - in org.apache.ambari.server.controller.internal.BlueprintConfigurationProcessorTest
Results :
Tests run: 167, Failures: 0, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 24.484s
[INFO] Finished at: Tue Nov 29 11:40:30 UTC 2016
[INFO] Final Memory: 62M/797M
[INFO] ------------------------------------------------------------------------
Thanks,
Mugdha Varadkar