You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Andrew Onischuk <ao...@hortonworks.com> on 2015/09/24 17:03:26 UTC
Review Request 38719: Detect non-compliant python versions and do not
attempt to start Ambari Agent
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38719/
-----------------------------------------------------------
Review request for Ambari and Vitalyi Brodetskyi.
Bugs: AMBARI-13229
https://issues.apache.org/jira/browse/AMBARI-13229
Repository: ambari
Description
-------
We had an issue where the root user's ~/.bashrc file had Python 3 in the PATH.
We blindly tried to execute tasks and hdp-select blew up with a syntax issue.
The vast majority of our tasks will fail with python 3 and as users start to
more heavily adopt newer version of python for data science tasks there is
risk that they will leak into being used by our user accounts, specifically
root for root agents, and the non-root user for non-root agents.
I would propose that we check the version of python before attempting to start
the ambari-agent, and fail with a non-zero exit code if we detect a non-
compliant version.
Diffs
-----
ambari-common/src/main/python/resource_management/libraries/functions/get_hdp_version.py e8fdbb6
ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py 0c42823
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py a3c02a6
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/service_check.py 6ec3996
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py 77e487f
ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms.py fafe1ec
ambari-server/src/main/resources/custom_actions/scripts/ru_set_all.py 6e5ddd2
ambari-server/src/test/python/custom_actions/test_ru_set_all.py b102b4e
ambari-server/src/test/python/stacks/2.0.6/FLUME/test_flume.py c3722c6
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_client.py ff25933
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py 8db555f
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_regionserver.py 2cb05c0
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_phoenix_queryserver.py 64e7845
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py efb17be
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_hdfs_client.py 4948d01
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py becc82b
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py b242c87
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_nfsgateway.py 62458c1
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_client.py 79bcc73
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 275d1d4
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_webhcat_server.py aca7664
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_client.py e8196d6
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 0ce5ce7
ambari-server/src/test/python/stacks/2.0.6/PIG/test_pig_client.py 2f42520
ambari-server/src/test/python/stacks/2.0.6/SQOOP/test_sqoop.py 2c0b8c8
ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 45c7978
ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_client.py 0e52264
ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 2692420
ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py fb7d847
ambari-server/src/test/python/stacks/2.0.6/YARN/test_yarn_client.py 21fbb9d
ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_client.py 79fd74b
ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_server.py afc4bc7
ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_client.py 9b08a90
ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 793e8dc
ambari-server/src/test/python/stacks/2.1/HIVE/test_hive_metastore.py 6eeacbe
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_drpc_server.py 283c865
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_nimbus.py bbcc15a
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_nimbus_prod.py 03f33f6
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_rest_api_service.py b26913e
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_supervisor.py 84fb64c
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_supervisor_prod.py 649b716
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_ui_server.py 185c66d
ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py 89037eb
ambari-server/src/test/python/stacks/2.1/YARN/test_apptimelineserver.py cb7b06b
ambari-server/src/test/python/stacks/2.2/ACCUMULO/test_accumulo_client.py fd97061
ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py 3337e3f
ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 83bba1f
ambari-server/src/test/python/stacks/2.2/RANGER/test_ranger_admin.py 2009b38
ambari-server/src/test/python/stacks/2.2/RANGER/test_ranger_usersync.py c2b826b
ambari-server/src/test/python/stacks/2.2/SLIDER/test_slider_client.py 3af55f6
ambari-server/src/test/python/stacks/2.2/SPARK/test_job_history_server.py 9cf1bd1
ambari-server/src/test/python/stacks/2.2/SPARK/test_spark_client.py deec6d6
ambari-server/src/test/python/stacks/2.3/MAHOUT/test_mahout_client.py 4776859
ambari-server/src/test/python/stacks/2.3/SPARK/test_spark_thrift_server.py a0b80f0
Diff: https://reviews.apache.org/r/38719/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk
Re: Review Request 38719: Detect non-compliant python versions and do
not attempt to start Ambari Agent
Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38719/#review100415
-----------------------------------------------------------
Ship it!
Ship It!
- Vitalyi Brodetskyi
On Вер. 24, 2015, 3:03 після полудня, Andrew Onischuk wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38719/
> -----------------------------------------------------------
>
> (Updated Вер. 24, 2015, 3:03 після полудня)
>
>
> Review request for Ambari and Vitalyi Brodetskyi.
>
>
> Bugs: AMBARI-13229
> https://issues.apache.org/jira/browse/AMBARI-13229
>
>
> Repository: ambari
>
>
> Description
> -------
>
> We had an issue where the root user's ~/.bashrc file had Python 3 in the PATH.
> We blindly tried to execute tasks and hdp-select blew up with a syntax issue.
> The vast majority of our tasks will fail with python 3 and as users start to
> more heavily adopt newer version of python for data science tasks there is
> risk that they will leak into being used by our user accounts, specifically
> root for root agents, and the non-root user for non-root agents.
> I would propose that we check the version of python before attempting to start
> the ambari-agent, and fail with a non-zero exit code if we detect a non-
> compliant version.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/get_hdp_version.py e8fdbb6
> ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py 0c42823
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py a3c02a6
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/service_check.py 6ec3996
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py 77e487f
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms.py fafe1ec
> ambari-server/src/main/resources/custom_actions/scripts/ru_set_all.py 6e5ddd2
> ambari-server/src/test/python/custom_actions/test_ru_set_all.py b102b4e
> ambari-server/src/test/python/stacks/2.0.6/FLUME/test_flume.py c3722c6
> ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_client.py ff25933
> ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py 8db555f
> ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_regionserver.py 2cb05c0
> ambari-server/src/test/python/stacks/2.0.6/HBASE/test_phoenix_queryserver.py 64e7845
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py efb17be
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_hdfs_client.py 4948d01
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py becc82b
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py b242c87
> ambari-server/src/test/python/stacks/2.0.6/HDFS/test_nfsgateway.py 62458c1
> ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_client.py 79bcc73
> ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 275d1d4
> ambari-server/src/test/python/stacks/2.0.6/HIVE/test_webhcat_server.py aca7664
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_client.py e8196d6
> ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 0ce5ce7
> ambari-server/src/test/python/stacks/2.0.6/PIG/test_pig_client.py 2f42520
> ambari-server/src/test/python/stacks/2.0.6/SQOOP/test_sqoop.py 2c0b8c8
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 45c7978
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_client.py 0e52264
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 2692420
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py fb7d847
> ambari-server/src/test/python/stacks/2.0.6/YARN/test_yarn_client.py 21fbb9d
> ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_client.py 79fd74b
> ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_server.py afc4bc7
> ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_client.py 9b08a90
> ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 793e8dc
> ambari-server/src/test/python/stacks/2.1/HIVE/test_hive_metastore.py 6eeacbe
> ambari-server/src/test/python/stacks/2.1/STORM/test_storm_drpc_server.py 283c865
> ambari-server/src/test/python/stacks/2.1/STORM/test_storm_nimbus.py bbcc15a
> ambari-server/src/test/python/stacks/2.1/STORM/test_storm_nimbus_prod.py 03f33f6
> ambari-server/src/test/python/stacks/2.1/STORM/test_storm_rest_api_service.py b26913e
> ambari-server/src/test/python/stacks/2.1/STORM/test_storm_supervisor.py 84fb64c
> ambari-server/src/test/python/stacks/2.1/STORM/test_storm_supervisor_prod.py 649b716
> ambari-server/src/test/python/stacks/2.1/STORM/test_storm_ui_server.py 185c66d
> ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py 89037eb
> ambari-server/src/test/python/stacks/2.1/YARN/test_apptimelineserver.py cb7b06b
> ambari-server/src/test/python/stacks/2.2/ACCUMULO/test_accumulo_client.py fd97061
> ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py 3337e3f
> ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 83bba1f
> ambari-server/src/test/python/stacks/2.2/RANGER/test_ranger_admin.py 2009b38
> ambari-server/src/test/python/stacks/2.2/RANGER/test_ranger_usersync.py c2b826b
> ambari-server/src/test/python/stacks/2.2/SLIDER/test_slider_client.py 3af55f6
> ambari-server/src/test/python/stacks/2.2/SPARK/test_job_history_server.py 9cf1bd1
> ambari-server/src/test/python/stacks/2.2/SPARK/test_spark_client.py deec6d6
> ambari-server/src/test/python/stacks/2.3/MAHOUT/test_mahout_client.py 4776859
> ambari-server/src/test/python/stacks/2.3/SPARK/test_spark_thrift_server.py a0b80f0
>
> Diff: https://reviews.apache.org/r/38719/diff/
>
>
> Testing
> -------
>
> mvn clean test
>
>
> Thanks,
>
> Andrew Onischuk
>
>
Re: Review Request 38719: Detect non-compliant python versions and do
not attempt to start Ambari Agent
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38719/
-----------------------------------------------------------
(Updated Sept. 28, 2015, 10:24 a.m.)
Review request for Ambari and Vitalyi Brodetskyi.
Bugs: AMBARI-13229
https://issues.apache.org/jira/browse/AMBARI-13229
Repository: ambari
Description
-------
We had an issue where the root user's ~/.bashrc file had Python 3 in the PATH.
We blindly tried to execute tasks and hdp-select blew up with a syntax issue.
The vast majority of our tasks will fail with python 3 and as users start to
more heavily adopt newer version of python for data science tasks there is
risk that they will leak into being used by our user accounts, specifically
root for root agents, and the non-root user for non-root agents.
I would propose that we check the version of python before attempting to start
the ambari-agent, and fail with a non-zero exit code if we detect a non-
compliant version.
Diffs (updated)
-----
ambari-common/src/main/python/resource_management/libraries/functions/get_hdp_version.py e8fdbb6
ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py f4f0efc
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py a3c02a6
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/service_check.py 6ec3996
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py 77e487f
ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms.py fafe1ec
ambari-server/src/main/resources/custom_actions/scripts/ru_set_all.py 6e5ddd2
ambari-server/src/test/python/custom_actions/test_ru_set_all.py b102b4e
ambari-server/src/test/python/stacks/2.0.6/FLUME/test_flume.py be2b87c
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_client.py ff25933
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py f19da4c
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_regionserver.py 2cb05c0
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_phoenix_queryserver.py 0cfc2e3
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py d5a42f0
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_hdfs_client.py 4948d01
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py becc82b
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py b9211a5
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_nfsgateway.py 89b4762
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_client.py 79bcc73
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py e4ca82e
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_webhcat_server.py aca7664
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_client.py e8196d6
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py 9cf426f
ambari-server/src/test/python/stacks/2.0.6/PIG/test_pig_client.py 2f42520
ambari-server/src/test/python/stacks/2.0.6/SQOOP/test_sqoop.py 2c0b8c8
ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py b6d5f42
ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_client.py 0e52264
ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 2692420
ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py fb7d847
ambari-server/src/test/python/stacks/2.0.6/YARN/test_yarn_client.py 21fbb9d
ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_client.py 79fd74b
ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_server.py afc4bc7
ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_client.py 9b08a90
ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py c9166a5
ambari-server/src/test/python/stacks/2.1/HIVE/test_hive_metastore.py 9d42b9f
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_drpc_server.py 283c865
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_nimbus.py bbcc15a
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_nimbus_prod.py 03f33f6
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_rest_api_service.py b26913e
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_supervisor.py 84fb64c
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_supervisor_prod.py 649b716
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_ui_server.py 185c66d
ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py 89037eb
ambari-server/src/test/python/stacks/2.1/YARN/test_apptimelineserver.py cb7b06b
ambari-server/src/test/python/stacks/2.2/ACCUMULO/test_accumulo_client.py 1b518d4
ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py ffb5914
ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 83bba1f
ambari-server/src/test/python/stacks/2.2/RANGER/test_ranger_admin.py 912f187
ambari-server/src/test/python/stacks/2.2/RANGER/test_ranger_usersync.py d10a9fc
ambari-server/src/test/python/stacks/2.2/SLIDER/test_slider_client.py 3af55f6
ambari-server/src/test/python/stacks/2.2/SPARK/test_job_history_server.py 369fdf9
ambari-server/src/test/python/stacks/2.2/SPARK/test_spark_client.py a414dda
ambari-server/src/test/python/stacks/2.3/MAHOUT/test_mahout_client.py a44c5af
ambari-server/src/test/python/stacks/2.3/SPARK/test_spark_thrift_server.py 9e41e11
Diff: https://reviews.apache.org/r/38719/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk