You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Andrew Onischuk <ao...@hortonworks.com> on 2015/05/22 19:16:47 UTC
Review Request 34602: Optimize creating hdfs resources to Webhdfs
instead of hadoop fs jar calls
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34602/
-----------------------------------------------------------
Review request for Ambari and Myroslav Papirkovskyy.
Bugs: AMBARI-11347
https://issues.apache.org/jira/browse/AMBARI-11347
Repository: ambari
Description
-------
After a conversation with Shivani in Mahadev, seems like we need this for
Ambari-2.1.
Tested scenarios:
1\. Deploy
2\. HA
3\. Security
The implementation as well checks if the current system is hdfs to use this
(since only hdfs support it) and if webhdfs is enabled.
Diffs
-----
ambari-common/src/main/python/resource_management/libraries/functions/jmx.py PRE-CREATION
ambari-common/src/main/python/resource_management/libraries/functions/namenode_ha_utils.py PRE-CREATION
ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py 21ad8ed
ambari-common/src/main/python/resource_management/libraries/resources/hdfs_resource.py 92a043f
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog210.java 69c82ef
ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/params.py 205d012
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 4a63e3c
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py 031f5ab
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py 1f62ee7
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml 3cff24c
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py 3900967
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py 8642ba5
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py 839d980
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 21976f4
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 934aadf
ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/params.py 54f0ba0
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py bc168c6
ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/params_linux.py fc39bc6
ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/service_check.py 5f33b1d
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py 34d1cb8
ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py ab29a95
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 90b23de
ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hadoop-env.xml 21b92a2
ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml 0068e3e
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py cc404eb
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py c57287b
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_service_check.py 36abd40
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 3859236
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_service_check.py 1a68b61
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 2bf2af7
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 54c85e4
ambari-server/src/test/python/stacks/2.0.6/PIG/test_pig_service_check.py 79e3697
ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 494e138
ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_service_check.py 6c19bc4
ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py 9025b48
ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 7fc5bce
ambari-server/src/test/python/stacks/2.1/TEZ/test_service_check.py 1eeba7e
ambari-server/src/test/python/stacks/2.2/PIG/test_pig_service_check.py 039b14d
ambari-server/src/test/python/stacks/2.2/SPARK/test_job_history_server.py 4e8ec42
ambari-server/src/test/python/stacks/2.3/MAHOUT/test_mahout_service_check.py 65b04f7
ambari-server/src/test/python/stacks/utils/RMFTestCase.py a186527
Diff: https://reviews.apache.org/r/34602/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk
Re: Review Request 34602: Optimize creating hdfs resources by using
Webhdfs instead of hadoop fs jar calls
Posted by Andrew Onischuk <ao...@hortonworks.com>.
> On May 22, 2015, 5:46 p.m., Alejandro Fernandez wrote:
> > ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py, line 134
> > <https://reviews.apache.org/r/34602/diff/2/?file=969482#file969482line134>
> >
> > Perhaps add some doc for what this class does.
Thanks for you review Alejandro. I'll commit this fixes in my next patches.
- Andrew
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34602/#review84940
-----------------------------------------------------------
On May 22, 2015, 5:33 p.m., Andrew Onischuk wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34602/
> -----------------------------------------------------------
>
> (Updated May 22, 2015, 5:33 p.m.)
>
>
> Review request for Ambari and Myroslav Papirkovskyy.
>
>
> Bugs: AMBARI-11347
> https://issues.apache.org/jira/browse/AMBARI-11347
>
>
> Repository: ambari
>
>
> Description
> -------
>
> After a conversation with Shivani in Mahadev, seems like we need this for
> Ambari-2.1.
>
> Tested scenarios:
> 1\. Deploy
> 2\. HA
> 3\. Security
>
> The implementation as well checks if the current system is hdfs to use this
> (since only hdfs support it) and if webhdfs is enabled.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/jmx.py PRE-CREATION
> ambari-common/src/main/python/resource_management/libraries/functions/namenode_ha_utils.py PRE-CREATION
> ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py 21ad8ed
> ambari-common/src/main/python/resource_management/libraries/resources/hdfs_resource.py 970e1e9
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog210.java 69c82ef
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/params.py 205d012
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 4a63e3c
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py 031f5ab
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py 01f082b
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml 3cff24c
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py 3900967
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py 8642ba5
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py b29eb8e
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 21976f4
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 647892a
> ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/params.py 54f0ba0
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py bc168c6
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/params_linux.py d7ff0ec
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/service_check.py b090a04
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py 348f49d
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py ab29a95
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 2198bcc
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hadoop-env.xml 21b92a2
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml 0068e3e
> ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py cc404eb
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py c57287b
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_service_check.py 36abd40
> ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 357ee56
> ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_service_check.py 1a68b61
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 2bf2af7
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 54c85e4
> ambari-server/src/test/python/stacks/2.0.6/PIG/test_pig_service_check.py 4e56be3
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 2b1b581
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_service_check.py 6c19bc4
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py b97d50d
> ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 7fc5bce
> ambari-server/src/test/python/stacks/2.1/TEZ/test_service_check.py 1eeba7e
> ambari-server/src/test/python/stacks/2.2/PIG/test_pig_service_check.py dcdae89
> ambari-server/src/test/python/stacks/2.2/SPARK/test_job_history_server.py 489037c
> ambari-server/src/test/python/stacks/2.3/MAHOUT/test_mahout_service_check.py 65b04f7
> ambari-server/src/test/python/stacks/utils/RMFTestCase.py a186527
>
> Diff: https://reviews.apache.org/r/34602/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Andrew Onischuk
>
>
Re: Review Request 34602: Optimize creating hdfs resources by using
Webhdfs instead of hadoop fs jar calls
Posted by Andrew Onischuk <ao...@hortonworks.com>.
> On May 22, 2015, 5:46 p.m., Alejandro Fernandez wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/jmx.py, line 23
> > <https://reviews.apache.org/r/34602/diff/2/?file=969480#file969480line23>
> >
> > HDFS's utils.py has get_value_from_jmx().
> > We should avoid code duplication by keeping these functions in a single place.
It's already removed in the patch from there and move to the common
- Andrew
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34602/#review84940
-----------------------------------------------------------
On May 22, 2015, 5:33 p.m., Andrew Onischuk wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34602/
> -----------------------------------------------------------
>
> (Updated May 22, 2015, 5:33 p.m.)
>
>
> Review request for Ambari and Myroslav Papirkovskyy.
>
>
> Bugs: AMBARI-11347
> https://issues.apache.org/jira/browse/AMBARI-11347
>
>
> Repository: ambari
>
>
> Description
> -------
>
> After a conversation with Shivani in Mahadev, seems like we need this for
> Ambari-2.1.
>
> Tested scenarios:
> 1\. Deploy
> 2\. HA
> 3\. Security
>
> The implementation as well checks if the current system is hdfs to use this
> (since only hdfs support it) and if webhdfs is enabled.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/jmx.py PRE-CREATION
> ambari-common/src/main/python/resource_management/libraries/functions/namenode_ha_utils.py PRE-CREATION
> ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py 21ad8ed
> ambari-common/src/main/python/resource_management/libraries/resources/hdfs_resource.py 970e1e9
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog210.java 69c82ef
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/params.py 205d012
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 4a63e3c
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py 031f5ab
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py 01f082b
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml 3cff24c
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py 3900967
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py 8642ba5
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py b29eb8e
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 21976f4
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 647892a
> ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/params.py 54f0ba0
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py bc168c6
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/params_linux.py d7ff0ec
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/service_check.py b090a04
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py 348f49d
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py ab29a95
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 2198bcc
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hadoop-env.xml 21b92a2
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml 0068e3e
> ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py cc404eb
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py c57287b
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_service_check.py 36abd40
> ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 357ee56
> ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_service_check.py 1a68b61
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 2bf2af7
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 54c85e4
> ambari-server/src/test/python/stacks/2.0.6/PIG/test_pig_service_check.py 4e56be3
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 2b1b581
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_service_check.py 6c19bc4
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py b97d50d
> ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 7fc5bce
> ambari-server/src/test/python/stacks/2.1/TEZ/test_service_check.py 1eeba7e
> ambari-server/src/test/python/stacks/2.2/PIG/test_pig_service_check.py dcdae89
> ambari-server/src/test/python/stacks/2.2/SPARK/test_job_history_server.py 489037c
> ambari-server/src/test/python/stacks/2.3/MAHOUT/test_mahout_service_check.py 65b04f7
> ambari-server/src/test/python/stacks/utils/RMFTestCase.py a186527
>
> Diff: https://reviews.apache.org/r/34602/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Andrew Onischuk
>
>
Re: Review Request 34602: Optimize creating hdfs resources by using
Webhdfs instead of hadoop fs jar calls
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34602/#review84940
-----------------------------------------------------------
Any unit tests needed?
ambari-common/src/main/python/resource_management/libraries/functions/jmx.py
<https://reviews.apache.org/r/34602/#comment136387>
HDFS's utils.py has get_value_from_jmx().
We should avoid code duplication by keeping these functions in a single place.
ambari-common/src/main/python/resource_management/libraries/functions/namenode_ha_utils.py
<https://reviews.apache.org/r/34602/#comment136388>
This is also copied in alert_ha_namenode_healthy.py
We should try to keep this logic in a single place.
ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py
<https://reviews.apache.org/r/34602/#comment136390>
Perhaps add some doc for what this class does.
- Alejandro Fernandez
On May 22, 2015, 5:33 p.m., Andrew Onischuk wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34602/
> -----------------------------------------------------------
>
> (Updated May 22, 2015, 5:33 p.m.)
>
>
> Review request for Ambari and Myroslav Papirkovskyy.
>
>
> Bugs: AMBARI-11347
> https://issues.apache.org/jira/browse/AMBARI-11347
>
>
> Repository: ambari
>
>
> Description
> -------
>
> After a conversation with Shivani in Mahadev, seems like we need this for
> Ambari-2.1.
>
> Tested scenarios:
> 1\. Deploy
> 2\. HA
> 3\. Security
>
> The implementation as well checks if the current system is hdfs to use this
> (since only hdfs support it) and if webhdfs is enabled.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/jmx.py PRE-CREATION
> ambari-common/src/main/python/resource_management/libraries/functions/namenode_ha_utils.py PRE-CREATION
> ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py 21ad8ed
> ambari-common/src/main/python/resource_management/libraries/resources/hdfs_resource.py 970e1e9
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog210.java 69c82ef
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/params.py 205d012
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 4a63e3c
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py 031f5ab
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py 01f082b
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml 3cff24c
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py 3900967
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py 8642ba5
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py b29eb8e
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 21976f4
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 647892a
> ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/params.py 54f0ba0
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py bc168c6
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/params_linux.py d7ff0ec
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/service_check.py b090a04
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py 348f49d
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py ab29a95
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 2198bcc
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hadoop-env.xml 21b92a2
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml 0068e3e
> ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py cc404eb
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py c57287b
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_service_check.py 36abd40
> ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 357ee56
> ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_service_check.py 1a68b61
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 2bf2af7
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 54c85e4
> ambari-server/src/test/python/stacks/2.0.6/PIG/test_pig_service_check.py 4e56be3
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 2b1b581
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_service_check.py 6c19bc4
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py b97d50d
> ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 7fc5bce
> ambari-server/src/test/python/stacks/2.1/TEZ/test_service_check.py 1eeba7e
> ambari-server/src/test/python/stacks/2.2/PIG/test_pig_service_check.py dcdae89
> ambari-server/src/test/python/stacks/2.2/SPARK/test_job_history_server.py 489037c
> ambari-server/src/test/python/stacks/2.3/MAHOUT/test_mahout_service_check.py 65b04f7
> ambari-server/src/test/python/stacks/utils/RMFTestCase.py a186527
>
> Diff: https://reviews.apache.org/r/34602/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Andrew Onischuk
>
>
Re: Review Request 34602: Optimize creating hdfs resources by using
Webhdfs instead of hadoop fs jar calls
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34602/
-----------------------------------------------------------
(Updated May 22, 2015, 5:33 p.m.)
Review request for Ambari and Myroslav Papirkovskyy.
Summary (updated)
-----------------
Optimize creating hdfs resources by using Webhdfs instead of hadoop fs jar calls
Bugs: AMBARI-11347
https://issues.apache.org/jira/browse/AMBARI-11347
Repository: ambari
Description
-------
After a conversation with Shivani in Mahadev, seems like we need this for
Ambari-2.1.
Tested scenarios:
1\. Deploy
2\. HA
3\. Security
The implementation as well checks if the current system is hdfs to use this
(since only hdfs support it) and if webhdfs is enabled.
Diffs
-----
ambari-common/src/main/python/resource_management/libraries/functions/jmx.py PRE-CREATION
ambari-common/src/main/python/resource_management/libraries/functions/namenode_ha_utils.py PRE-CREATION
ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py 21ad8ed
ambari-common/src/main/python/resource_management/libraries/resources/hdfs_resource.py 970e1e9
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog210.java 69c82ef
ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/params.py 205d012
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 4a63e3c
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py 031f5ab
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py 01f082b
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml 3cff24c
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py 3900967
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py 8642ba5
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py b29eb8e
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 21976f4
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 647892a
ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/params.py 54f0ba0
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py bc168c6
ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/params_linux.py d7ff0ec
ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/service_check.py b090a04
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py 348f49d
ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py ab29a95
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 2198bcc
ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hadoop-env.xml 21b92a2
ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml 0068e3e
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py cc404eb
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py c57287b
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_service_check.py 36abd40
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 357ee56
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_service_check.py 1a68b61
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 2bf2af7
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 54c85e4
ambari-server/src/test/python/stacks/2.0.6/PIG/test_pig_service_check.py 4e56be3
ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 2b1b581
ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_service_check.py 6c19bc4
ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py b97d50d
ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 7fc5bce
ambari-server/src/test/python/stacks/2.1/TEZ/test_service_check.py 1eeba7e
ambari-server/src/test/python/stacks/2.2/PIG/test_pig_service_check.py dcdae89
ambari-server/src/test/python/stacks/2.2/SPARK/test_job_history_server.py 489037c
ambari-server/src/test/python/stacks/2.3/MAHOUT/test_mahout_service_check.py 65b04f7
ambari-server/src/test/python/stacks/utils/RMFTestCase.py a186527
Diff: https://reviews.apache.org/r/34602/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk
Re: Review Request 34602: Optimize creating hdfs resources to Webhdfs
instead of hadoop fs jar calls
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34602/
-----------------------------------------------------------
(Updated May 22, 2015, 5:33 p.m.)
Review request for Ambari and Myroslav Papirkovskyy.
Bugs: AMBARI-11347
https://issues.apache.org/jira/browse/AMBARI-11347
Repository: ambari
Description
-------
After a conversation with Shivani in Mahadev, seems like we need this for
Ambari-2.1.
Tested scenarios:
1\. Deploy
2\. HA
3\. Security
The implementation as well checks if the current system is hdfs to use this
(since only hdfs support it) and if webhdfs is enabled.
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/jmx.py PRE-CREATION
ambari-common/src/main/python/resource_management/libraries/functions/namenode_ha_utils.py PRE-CREATION
ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py 21ad8ed
ambari-common/src/main/python/resource_management/libraries/resources/hdfs_resource.py 970e1e9
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog210.java 69c82ef
ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/params.py 205d012
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 4a63e3c
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py 031f5ab
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py 01f082b
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml 3cff24c
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py 3900967
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py 8642ba5
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py b29eb8e
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 21976f4
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 647892a
ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/params.py 54f0ba0
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py bc168c6
ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/params_linux.py d7ff0ec
ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/service_check.py b090a04
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py 348f49d
ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py ab29a95
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 2198bcc
ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hadoop-env.xml 21b92a2
ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml 0068e3e
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py cc404eb
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py c57287b
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_service_check.py 36abd40
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 357ee56
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_service_check.py 1a68b61
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 2bf2af7
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 54c85e4
ambari-server/src/test/python/stacks/2.0.6/PIG/test_pig_service_check.py 4e56be3
ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 2b1b581
ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_service_check.py 6c19bc4
ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py b97d50d
ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 7fc5bce
ambari-server/src/test/python/stacks/2.1/TEZ/test_service_check.py 1eeba7e
ambari-server/src/test/python/stacks/2.2/PIG/test_pig_service_check.py dcdae89
ambari-server/src/test/python/stacks/2.2/SPARK/test_job_history_server.py 489037c
ambari-server/src/test/python/stacks/2.3/MAHOUT/test_mahout_service_check.py 65b04f7
ambari-server/src/test/python/stacks/utils/RMFTestCase.py a186527
Diff: https://reviews.apache.org/r/34602/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk
Re: Review Request 34602: Optimize creating hdfs resources to Webhdfs
instead of hadoop fs jar calls
Posted by Myroslav Papirkovskyy <mp...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34602/#review84937
-----------------------------------------------------------
Ship it!
Ship It!
- Myroslav Papirkovskyy
On Травень 22, 2015, 8:16 після полудня, Andrew Onischuk wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34602/
> -----------------------------------------------------------
>
> (Updated Травень 22, 2015, 8:16 після полудня)
>
>
> Review request for Ambari and Myroslav Papirkovskyy.
>
>
> Bugs: AMBARI-11347
> https://issues.apache.org/jira/browse/AMBARI-11347
>
>
> Repository: ambari
>
>
> Description
> -------
>
> After a conversation with Shivani in Mahadev, seems like we need this for
> Ambari-2.1.
>
> Tested scenarios:
> 1\. Deploy
> 2\. HA
> 3\. Security
>
> The implementation as well checks if the current system is hdfs to use this
> (since only hdfs support it) and if webhdfs is enabled.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/jmx.py PRE-CREATION
> ambari-common/src/main/python/resource_management/libraries/functions/namenode_ha_utils.py PRE-CREATION
> ambari-common/src/main/python/resource_management/libraries/providers/hdfs_resource.py 21ad8ed
> ambari-common/src/main/python/resource_management/libraries/resources/hdfs_resource.py 92a043f
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog210.java 69c82ef
> ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/params.py 205d012
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py 4a63e3c
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py 031f5ab
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py 1f62ee7
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/configuration/hadoop-env.xml 3cff24c
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py 3900967
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_ha_state.py 8642ba5
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py 839d980
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/utils.py 21976f4
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py 934aadf
> ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/params.py 54f0ba0
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py bc168c6
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/params_linux.py fc39bc6
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/service_check.py 5f33b1d
> ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py 34d1cb8
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py ab29a95
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 90b23de
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/configuration/hadoop-env.xml 21b92a2
> ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/configuration/hadoop-env.xml 0068e3e
> ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py cc404eb
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py c57287b
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_service_check.py 36abd40
> ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 3859236
> ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_service_check.py 1a68b61
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 2bf2af7
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_service_check.py 54c85e4
> ambari-server/src/test/python/stacks/2.0.6/PIG/test_pig_service_check.py 79e3697
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 494e138
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_service_check.py 6c19bc4
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py 9025b48
> ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 7fc5bce
> ambari-server/src/test/python/stacks/2.1/TEZ/test_service_check.py 1eeba7e
> ambari-server/src/test/python/stacks/2.2/PIG/test_pig_service_check.py 039b14d
> ambari-server/src/test/python/stacks/2.2/SPARK/test_job_history_server.py 4e8ec42
> ambari-server/src/test/python/stacks/2.3/MAHOUT/test_mahout_service_check.py 65b04f7
> ambari-server/src/test/python/stacks/utils/RMFTestCase.py a186527
>
> Diff: https://reviews.apache.org/r/34602/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Andrew Onischuk
>
>