You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Alejandro Fernandez <af...@hortonworks.com> on 2014/10/24 02:32:53 UTC
Review Request 27121: Hiveserver2 and Webhcat fails to start in secure
cluster
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27121/
-----------------------------------------------------------
Review request for Ambari, Andrew Onischuk, Jaimin Jetly, jun aoki, Jonathan Hurley, Sumit Mohanty, Srimanth Gunturi, Sid Wagle, and Yusaku Sako.
Bugs: AMBARI-7942
https://issues.apache.org/jira/browse/AMBARI-7942
Repository: ambari
Description
-------
Deploy a cluster with default configs (did not select Kafka, Slider and Knox)
After enabling security hive server2 and webhcat fails to start.
```
2014-10-23 23:52:27,200 - Error while executing command 'start':
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 122, in execute
method(env)
File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HIVE/package/scripts/hive_server.py", line 44, in start
copy_tarballs_to_hdfs('tez', params.tez_user, params.hdfs_user)
File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/dynamic_variable_interpretation.py", line 228, in copy_tarballs_to_hdfs
user=component_user
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 241, in action_run
raise ex
Fail: Execution of '{kinit_path_local} -kt {hdfs_user_keytab} {hdfs_principal_name};' returned 127. -bash: {kinit_path_local}: command not found
```
Diffs
-----
ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py 5112c45
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/package/scripts/webhcat.py 4eeff8b
Diff: https://reviews.apache.org/r/27121/diff/
Testing
-------
Reproduced the issue on a cluster after enabling security.
Then copied the changed scripts, and was able to start all Hive Components, including WebHcat
Ran unit tests in ambari-server.
----------------------------------------------------------------------
Total run:669
Total errors:0
Total failures:0
OK
Thanks,
Alejandro Fernandez
Re: Review Request 27121: Hiveserver2 and Webhcat fails to start in
secure cluster
Posted by Srimanth Gunturi <sr...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27121/#review58193
-----------------------------------------------------------
Ship it!
Ship It!
- Srimanth Gunturi
On Oct. 24, 2014, 12:40 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27121/
> -----------------------------------------------------------
>
> (Updated Oct. 24, 2014, 12:40 a.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Jaimin Jetly, jun aoki, Jonathan Hurley, Sumit Mohanty, Srimanth Gunturi, Sid Wagle, and Yusaku Sako.
>
>
> Bugs: AMBARI-7942
> https://issues.apache.org/jira/browse/AMBARI-7942
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Deploy a cluster with default configs (did not select Kafka, Slider and Knox)
> After enabling security hive server2 and webhcat fails to start.
>
> ```
> 2014-10-23 23:52:27,200 - Error while executing command 'start':
> Traceback (most recent call last):
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 122, in execute
> method(env)
> File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HIVE/package/scripts/hive_server.py", line 44, in start
> copy_tarballs_to_hdfs('tez', params.tez_user, params.hdfs_user)
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/dynamic_variable_interpretation.py", line 228, in copy_tarballs_to_hdfs
> user=component_user
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
> self.env.run()
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149, in run
> self.run_action(resource, action)
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
> provider_action()
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 241, in action_run
> raise ex
> Fail: Execution of '{kinit_path_local} -kt {hdfs_user_keytab} {hdfs_principal_name};' returned 127. -bash: {kinit_path_local}: command not found
> ```
>
> This is because dynamic_variable_interpretation.py is missing imports for format and Execute, which are used when security is enabled.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py 5112c45
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/package/scripts/webhcat.py 4eeff8b
>
> Diff: https://reviews.apache.org/r/27121/diff/
>
>
> Testing
> -------
>
> Reproduced the issue on a cluster after enabling security.
> Then copied the changed scripts, and was able to start all Hive Components, including WebHcat
>
> Ran unit tests in ambari-server.
>
> ----------------------------------------------------------------------
> Total run:669
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 27121: Hiveserver2 and Webhcat fails to start in
secure cluster
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27121/#review58308
-----------------------------------------------------------
Ship it!
Ship It!
- Jonathan Hurley
On Oct. 23, 2014, 8:40 p.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27121/
> -----------------------------------------------------------
>
> (Updated Oct. 23, 2014, 8:40 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Jaimin Jetly, jun aoki, Jonathan Hurley, Sumit Mohanty, Srimanth Gunturi, Sid Wagle, and Yusaku Sako.
>
>
> Bugs: AMBARI-7942
> https://issues.apache.org/jira/browse/AMBARI-7942
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Deploy a cluster with default configs (did not select Kafka, Slider and Knox)
> After enabling security hive server2 and webhcat fails to start.
>
> ```
> 2014-10-23 23:52:27,200 - Error while executing command 'start':
> Traceback (most recent call last):
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 122, in execute
> method(env)
> File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HIVE/package/scripts/hive_server.py", line 44, in start
> copy_tarballs_to_hdfs('tez', params.tez_user, params.hdfs_user)
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/dynamic_variable_interpretation.py", line 228, in copy_tarballs_to_hdfs
> user=component_user
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
> self.env.run()
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149, in run
> self.run_action(resource, action)
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
> provider_action()
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 241, in action_run
> raise ex
> Fail: Execution of '{kinit_path_local} -kt {hdfs_user_keytab} {hdfs_principal_name};' returned 127. -bash: {kinit_path_local}: command not found
> ```
>
> This is because dynamic_variable_interpretation.py is missing imports for format and Execute, which are used when security is enabled.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py 5112c45
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/package/scripts/webhcat.py 4eeff8b
>
> Diff: https://reviews.apache.org/r/27121/diff/
>
>
> Testing
> -------
>
> Reproduced the issue on a cluster after enabling security.
> Then copied the changed scripts, and was able to start all Hive Components, including WebHcat
>
> Ran unit tests in ambari-server.
>
> ----------------------------------------------------------------------
> Total run:669
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 27121: Hiveserver2 and Webhcat fails to start in
secure cluster
Posted by Alejandro Fernandez <af...@hortonworks.com>.
> On Oct. 24, 2014, 12:43 a.m., Jaimin Jetly wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/package/scripts/webhcat.py, line 26
> > <https://reviews.apache.org/r/27121/diff/1/?file=731775#file731775line26>
> >
> > IS this required ?
Technically no, but I want Ambari to start the practice of only importing what is required. This makes it easier to find out where things come from, helps avoid name collisions, and makes the developer more conscious about what modules/classes/functions they need.
See the comment quoted from Guido van Rossum in http://stackoverflow.com/questions/193919/what-are-good-rules-of-thumb-for-python-imports
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27121/#review58186
-----------------------------------------------------------
On Oct. 24, 2014, 12:40 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27121/
> -----------------------------------------------------------
>
> (Updated Oct. 24, 2014, 12:40 a.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Jaimin Jetly, jun aoki, Jonathan Hurley, Sumit Mohanty, Srimanth Gunturi, Sid Wagle, and Yusaku Sako.
>
>
> Bugs: AMBARI-7942
> https://issues.apache.org/jira/browse/AMBARI-7942
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Deploy a cluster with default configs (did not select Kafka, Slider and Knox)
> After enabling security hive server2 and webhcat fails to start.
>
> ```
> 2014-10-23 23:52:27,200 - Error while executing command 'start':
> Traceback (most recent call last):
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 122, in execute
> method(env)
> File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HIVE/package/scripts/hive_server.py", line 44, in start
> copy_tarballs_to_hdfs('tez', params.tez_user, params.hdfs_user)
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/dynamic_variable_interpretation.py", line 228, in copy_tarballs_to_hdfs
> user=component_user
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
> self.env.run()
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149, in run
> self.run_action(resource, action)
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
> provider_action()
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 241, in action_run
> raise ex
> Fail: Execution of '{kinit_path_local} -kt {hdfs_user_keytab} {hdfs_principal_name};' returned 127. -bash: {kinit_path_local}: command not found
> ```
>
> This is because dynamic_variable_interpretation.py is missing imports for format and Execute, which are used when security is enabled.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py 5112c45
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/package/scripts/webhcat.py 4eeff8b
>
> Diff: https://reviews.apache.org/r/27121/diff/
>
>
> Testing
> -------
>
> Reproduced the issue on a cluster after enabling security.
> Then copied the changed scripts, and was able to start all Hive Components, including WebHcat
>
> Ran unit tests in ambari-server.
>
> ----------------------------------------------------------------------
> Total run:669
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 27121: Hiveserver2 and Webhcat fails to start in
secure cluster
Posted by Jaimin Jetly <ja...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27121/#review58186
-----------------------------------------------------------
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/package/scripts/webhcat.py
<https://reviews.apache.org/r/27121/#comment99134>
IS this required ?
- Jaimin Jetly
On Oct. 24, 2014, 12:40 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27121/
> -----------------------------------------------------------
>
> (Updated Oct. 24, 2014, 12:40 a.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Jaimin Jetly, jun aoki, Jonathan Hurley, Sumit Mohanty, Srimanth Gunturi, Sid Wagle, and Yusaku Sako.
>
>
> Bugs: AMBARI-7942
> https://issues.apache.org/jira/browse/AMBARI-7942
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Deploy a cluster with default configs (did not select Kafka, Slider and Knox)
> After enabling security hive server2 and webhcat fails to start.
>
> ```
> 2014-10-23 23:52:27,200 - Error while executing command 'start':
> Traceback (most recent call last):
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 122, in execute
> method(env)
> File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HIVE/package/scripts/hive_server.py", line 44, in start
> copy_tarballs_to_hdfs('tez', params.tez_user, params.hdfs_user)
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/dynamic_variable_interpretation.py", line 228, in copy_tarballs_to_hdfs
> user=component_user
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
> self.env.run()
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149, in run
> self.run_action(resource, action)
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
> provider_action()
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 241, in action_run
> raise ex
> Fail: Execution of '{kinit_path_local} -kt {hdfs_user_keytab} {hdfs_principal_name};' returned 127. -bash: {kinit_path_local}: command not found
> ```
>
> This is because dynamic_variable_interpretation.py is missing imports for format and Execute, which are used when security is enabled.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py 5112c45
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/package/scripts/webhcat.py 4eeff8b
>
> Diff: https://reviews.apache.org/r/27121/diff/
>
>
> Testing
> -------
>
> Reproduced the issue on a cluster after enabling security.
> Then copied the changed scripts, and was able to start all Hive Components, including WebHcat
>
> Ran unit tests in ambari-server.
>
> ----------------------------------------------------------------------
> Total run:669
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 27121: Hiveserver2 and Webhcat fails to start in
secure cluster
Posted by Jaimin Jetly <ja...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27121/#review58195
-----------------------------------------------------------
Ship it!
Ship It!
- Jaimin Jetly
On Oct. 24, 2014, 12:40 a.m., Alejandro Fernandez wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27121/
> -----------------------------------------------------------
>
> (Updated Oct. 24, 2014, 12:40 a.m.)
>
>
> Review request for Ambari, Andrew Onischuk, Jaimin Jetly, jun aoki, Jonathan Hurley, Sumit Mohanty, Srimanth Gunturi, Sid Wagle, and Yusaku Sako.
>
>
> Bugs: AMBARI-7942
> https://issues.apache.org/jira/browse/AMBARI-7942
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Deploy a cluster with default configs (did not select Kafka, Slider and Knox)
> After enabling security hive server2 and webhcat fails to start.
>
> ```
> 2014-10-23 23:52:27,200 - Error while executing command 'start':
> Traceback (most recent call last):
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 122, in execute
> method(env)
> File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HIVE/package/scripts/hive_server.py", line 44, in start
> copy_tarballs_to_hdfs('tez', params.tez_user, params.hdfs_user)
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/dynamic_variable_interpretation.py", line 228, in copy_tarballs_to_hdfs
> user=component_user
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
> self.env.run()
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149, in run
> self.run_action(resource, action)
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
> provider_action()
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 241, in action_run
> raise ex
> Fail: Execution of '{kinit_path_local} -kt {hdfs_user_keytab} {hdfs_principal_name};' returned 127. -bash: {kinit_path_local}: command not found
> ```
>
> This is because dynamic_variable_interpretation.py is missing imports for format and Execute, which are used when security is enabled.
>
>
> Diffs
> -----
>
> ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py 5112c45
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/package/scripts/webhcat.py 4eeff8b
>
> Diff: https://reviews.apache.org/r/27121/diff/
>
>
> Testing
> -------
>
> Reproduced the issue on a cluster after enabling security.
> Then copied the changed scripts, and was able to start all Hive Components, including WebHcat
>
> Ran unit tests in ambari-server.
>
> ----------------------------------------------------------------------
> Total run:669
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 27121: Hiveserver2 and Webhcat fails to start in
secure cluster
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27121/
-----------------------------------------------------------
(Updated Oct. 24, 2014, 12:40 a.m.)
Review request for Ambari, Andrew Onischuk, Jaimin Jetly, jun aoki, Jonathan Hurley, Sumit Mohanty, Srimanth Gunturi, Sid Wagle, and Yusaku Sako.
Bugs: AMBARI-7942
https://issues.apache.org/jira/browse/AMBARI-7942
Repository: ambari
Description (updated)
-------
Deploy a cluster with default configs (did not select Kafka, Slider and Knox)
After enabling security hive server2 and webhcat fails to start.
```
2014-10-23 23:52:27,200 - Error while executing command 'start':
Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 122, in execute
method(env)
File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/services/HIVE/package/scripts/hive_server.py", line 44, in start
copy_tarballs_to_hdfs('tez', params.tez_user, params.hdfs_user)
File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/dynamic_variable_interpretation.py", line 228, in copy_tarballs_to_hdfs
user=component_user
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 148, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 149, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 115, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 241, in action_run
raise ex
Fail: Execution of '{kinit_path_local} -kt {hdfs_user_keytab} {hdfs_principal_name};' returned 127. -bash: {kinit_path_local}: command not found
```
This is because dynamic_variable_interpretation.py is missing imports for format and Execute, which are used when security is enabled.
Diffs
-----
ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py 5112c45
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/HIVE/package/scripts/webhcat.py 4eeff8b
Diff: https://reviews.apache.org/r/27121/diff/
Testing
-------
Reproduced the issue on a cluster after enabling security.
Then copied the changed scripts, and was able to start all Hive Components, including WebHcat
Ran unit tests in ambari-server.
----------------------------------------------------------------------
Total run:669
Total errors:0
Total failures:0
OK
Thanks,
Alejandro Fernandez