You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Jonathan Hurley <jh...@hortonworks.com> on 2017/11/06 21:17:34 UTC
Review Request 63595: Remove HADOOP_HOME From Environment For Daemons
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63595/
-----------------------------------------------------------
Review request for Ambari, Dmitro Lisnichenko and Nate Cole.
Bugs: AMBARI-22370
https://issues.apache.org/jira/browse/AMBARI-22370
Repository: ambari
Description
-------
The wrapper scripts for services should be setting {{HADOOP_HOME}} correctly regardless of whether or not an upgrade is taking place. There are many spots which we current override this value and pass it in:
- Either via the Python files and setting the {{environment}}
- Or by setting it in configuration files
This should be removed since we must rely on correct {{HADOOP_HOME}} to be set by the wrapper scripts.
Diffs
-----
ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py d8f3d374f5
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py f6f678fa89
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py 933515b01b
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_service_interactive.py eaf95ad5db
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service.py 646090ac27
ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/service_check.py a79b593ce8
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py 400c87c828
ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py 4d636855c3
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 12edc6924f
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_webhcat_server.py f9480eeaff
ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 7f2ed464dc
ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py fad99f6496
ambari-server/src/test/python/stacks/2.3/MAHOUT/test_mahout_service_check.py b80476cab2
ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py 4951c7e2ec
Diff: https://reviews.apache.org/r/63595/diff/1/
Testing
-------
OK
----------------------------------------------------------------------
Total run:1193
Total errors:0
Total failures:0
OK
Thanks,
Jonathan Hurley
Re: Review Request 63595: Remove HADOOP_HOME From Environment For
Daemons
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63595/#review190312
-----------------------------------------------------------
Ship it!
Ship It!
- Nate Cole
On Nov. 6, 2017, 4:17 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63595/
> -----------------------------------------------------------
>
> (Updated Nov. 6, 2017, 4:17 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko and Nate Cole.
>
>
> Bugs: AMBARI-22370
> https://issues.apache.org/jira/browse/AMBARI-22370
>
>
> Repository: ambari
>
>
> Description
> -------
>
> The wrapper scripts for services should be setting {{HADOOP_HOME}} correctly regardless of whether or not an upgrade is taking place. There are many spots which we current override this value and pass it in:
>
> - Either via the Python files and setting the {{environment}}
> - Or by setting it in configuration files
>
> This should be removed since we must rely on correct {{HADOOP_HOME}} to be set by the wrapper scripts.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py d8f3d374f5
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py f6f678fa89
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py 933515b01b
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_service_interactive.py eaf95ad5db
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service.py 646090ac27
> ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/service_check.py a79b593ce8
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py 400c87c828
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py 4d636855c3
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 12edc6924f
> ambari-server/src/test/python/stacks/2.0.6/HIVE/test_webhcat_server.py f9480eeaff
> ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 7f2ed464dc
> ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py fad99f6496
> ambari-server/src/test/python/stacks/2.3/MAHOUT/test_mahout_service_check.py b80476cab2
> ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py 4951c7e2ec
>
>
> Diff: https://reviews.apache.org/r/63595/diff/1/
>
>
> Testing
> -------
>
> OK
> ----------------------------------------------------------------------
> Total run:1193
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 63595: Remove HADOOP_HOME From Environment For
Daemons
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
> On Nov. 7, 2017, 1:10 p.m., Dmitro Lisnichenko wrote:
> > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
> > Line 119 (original), 119 (patched)
> > <https://reviews.apache.org/r/63595/diff/1/?file=1882235#file1882235line119>
> >
> >
will it be a case during upgrade/downgrade: hadoop_conf_dir = format('{stack_root}/current/hadoop-client/conf')?
- Dmitro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63595/#review190288
-----------------------------------------------------------
On Nov. 6, 2017, 11:17 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63595/
> -----------------------------------------------------------
>
> (Updated Nov. 6, 2017, 11:17 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko and Nate Cole.
>
>
> Bugs: AMBARI-22370
> https://issues.apache.org/jira/browse/AMBARI-22370
>
>
> Repository: ambari
>
>
> Description
> -------
>
> The wrapper scripts for services should be setting {{HADOOP_HOME}} correctly regardless of whether or not an upgrade is taking place. There are many spots which we current override this value and pass it in:
>
> - Either via the Python files and setting the {{environment}}
> - Or by setting it in configuration files
>
> This should be removed since we must rely on correct {{HADOOP_HOME}} to be set by the wrapper scripts.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py d8f3d374f5
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py f6f678fa89
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py 933515b01b
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_service_interactive.py eaf95ad5db
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service.py 646090ac27
> ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/service_check.py a79b593ce8
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py 400c87c828
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py 4d636855c3
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 12edc6924f
> ambari-server/src/test/python/stacks/2.0.6/HIVE/test_webhcat_server.py f9480eeaff
> ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 7f2ed464dc
> ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py fad99f6496
> ambari-server/src/test/python/stacks/2.3/MAHOUT/test_mahout_service_check.py b80476cab2
> ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py 4951c7e2ec
>
>
> Diff: https://reviews.apache.org/r/63595/diff/1/
>
>
> Testing
> -------
>
> OK
> ----------------------------------------------------------------------
> Total run:1193
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 63595: Remove HADOOP_HOME From Environment For
Daemons
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On Nov. 7, 2017, 6:10 a.m., Dmitro Lisnichenko wrote:
> > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
> > Line 119 (original), 119 (patched)
> > <https://reviews.apache.org/r/63595/diff/1/?file=1882235#file1882235line119>
> >
> >
>
> Dmitro Lisnichenko wrote:
> will it be a case during upgrade/downgrade: hadoop_conf_dir = format('{stack_root}/current/hadoop-client/conf')?
No, it shouldn't be. /etc/hadoop/conf should aways point to hadoop-client/conf anyway. This just makes it clearer that it's using current/hadoop-client. Functionally, it should be no different.
> On Nov. 7, 2017, 6:10 a.m., Dmitro Lisnichenko wrote:
> > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service.py
> > Line 107 (original)
> > <https://reviews.apache.org/r/63595/diff/1/?file=1882238#file1882238line107>
> >
> > is it intended to remove env passed here?
Yes - it only had hadoop home in it:
environ = {
'HADOOP_HOME': params.hadoop_home
}
No reason to pass an empty dictionary...
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63595/#review190288
-----------------------------------------------------------
On Nov. 6, 2017, 4:17 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63595/
> -----------------------------------------------------------
>
> (Updated Nov. 6, 2017, 4:17 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko and Nate Cole.
>
>
> Bugs: AMBARI-22370
> https://issues.apache.org/jira/browse/AMBARI-22370
>
>
> Repository: ambari
>
>
> Description
> -------
>
> The wrapper scripts for services should be setting {{HADOOP_HOME}} correctly regardless of whether or not an upgrade is taking place. There are many spots which we current override this value and pass it in:
>
> - Either via the Python files and setting the {{environment}}
> - Or by setting it in configuration files
>
> This should be removed since we must rely on correct {{HADOOP_HOME}} to be set by the wrapper scripts.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py d8f3d374f5
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py f6f678fa89
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py 933515b01b
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_service_interactive.py eaf95ad5db
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service.py 646090ac27
> ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/service_check.py a79b593ce8
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py 400c87c828
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py 4d636855c3
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 12edc6924f
> ambari-server/src/test/python/stacks/2.0.6/HIVE/test_webhcat_server.py f9480eeaff
> ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 7f2ed464dc
> ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py fad99f6496
> ambari-server/src/test/python/stacks/2.3/MAHOUT/test_mahout_service_check.py b80476cab2
> ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py 4951c7e2ec
>
>
> Diff: https://reviews.apache.org/r/63595/diff/1/
>
>
> Testing
> -------
>
> OK
> ----------------------------------------------------------------------
> Total run:1193
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 63595: Remove HADOOP_HOME From Environment For
Daemons
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63595/#review190288
-----------------------------------------------------------
Fix it, then Ship it!
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
Line 119 (original), 119 (patched)
<https://reviews.apache.org/r/63595/#comment267580>
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service.py
Line 107 (original)
<https://reviews.apache.org/r/63595/#comment267581>
is it intended to remove env passed here?
- Dmitro Lisnichenko
On Nov. 6, 2017, 11:17 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63595/
> -----------------------------------------------------------
>
> (Updated Nov. 6, 2017, 11:17 p.m.)
>
>
> Review request for Ambari, Dmitro Lisnichenko and Nate Cole.
>
>
> Bugs: AMBARI-22370
> https://issues.apache.org/jira/browse/AMBARI-22370
>
>
> Repository: ambari
>
>
> Description
> -------
>
> The wrapper scripts for services should be setting {{HADOOP_HOME}} correctly regardless of whether or not an upgrade is taking place. There are many spots which we current override this value and pass it in:
>
> - Either via the Python files and setting the {{environment}}
> - Or by setting it in configuration files
>
> This should be removed since we must rely on correct {{HADOOP_HOME}} to be set by the wrapper scripts.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py d8f3d374f5
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py f6f678fa89
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py 933515b01b
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_service_interactive.py eaf95ad5db
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service.py 646090ac27
> ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/service_check.py a79b593ce8
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py 400c87c828
> ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py 4d636855c3
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py 12edc6924f
> ambari-server/src/test/python/stacks/2.0.6/HIVE/test_webhcat_server.py f9480eeaff
> ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 7f2ed464dc
> ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py fad99f6496
> ambari-server/src/test/python/stacks/2.3/MAHOUT/test_mahout_service_check.py b80476cab2
> ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py 4951c7e2ec
>
>
> Diff: https://reviews.apache.org/r/63595/diff/1/
>
>
> Testing
> -------
>
> OK
> ----------------------------------------------------------------------
> Total run:1193
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Jonathan Hurley
>
>