You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Robert Levas <rl...@hortonworks.com> on 2015/08/04 22:29:01 UTC
Review Request 37094: Kerberos: fails check during enable Kerb with
SLES
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37094/
-----------------------------------------------------------
Review request for Ambari, Andrew Onischuk, Dmytro Sen, Jayush Luniya, and Mahadev Konar.
Bugs: AMBARI-12636
https://issues.apache.org/jira/browse/AMBARI-12636
Repository: ambari
Description
-------
When executing the Kerberos service check, the following error occurs:
```
stderr: /var/lib/ambari-agent/data/errors-24.txt
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/service_check.py", line 81, in <module>
KerberosServiceCheck().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 218, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/service_check.py", line 64, in service_check
user=params.smoke_user
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 157, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 152, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 118, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 258, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
result = function(command, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
tries=tries, try_sleep=try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -c /var/lib/ambari-agent/data/tmp/kerberos_service_check_cc_dd529fe1e15538ddfe9ce0347604d64c -kt /etc/security/keytabs/kerberos.service_check.080315.keytab MyCluster-080315@EXAMPLE.COM' returned 1. kinit(v5): Credentials cache permissions incorrect when initializing cache /var/lib/ambari-agent/data/tmp/kerberos_service_check_cc_dd529fe1e15538ddfe9ce0347604d64c
stdout: /var/lib/ambari-agent/data/output-24.txt
Performing kinit using MyCluster-080315@EXAMPLE.COM
2015-08-03 19:11:57,085 - Execute['/usr/bin/kinit -c /var/lib/ambari-agent/data/tmp/kerberos_service_check_cc_dd529fe1e15538ddfe9ce0347604d64c -kt /etc/security/keytabs/kerberos.service_check.080315.keytab MyCluster-080315@EXAMPLE.COM'] {'user': 'ambari-qa'}
2015-08-03 19:11:57,179 - File['/var/lib/ambari-agent/data/tmp/kerberos_service_check_cc_dd529fe1e15538ddfe9ce0347604d64c'] {'action': ['delete']}
```
This error happens only on SLES, however the cause exists on all platforms. The other platforms silently ignore the condition; which, however, does not have any bearing on the results of the _kinit_ test.
#Cause
The "Credentials cache permissions incorrect when initializing cache" issue is caused by the inability to write the Kerberos ticket cache file to the specified location. In the case it is /var/lib/ambari-agent/data/tmp/kerberos_service_check_cc_dd529fe1e15538ddfe9ce0347604d64c. The reason for the write failure is that /var/lib/ambari-agent/data/tmp is not writable by the user executing the _kinit_ call - which is the Ambari smoke test user (typically ambari-qa). The directory's permissions are
```
drwxr-xr-x. 4 root root 4096 Aug 3 22:20 /var/lib/ambari-agent/data/tmp/
```
#Solution
In order for the ambari smoke test user to be able to write to the relevant directory (/var/lib/ambari-agent/data/tmp), the permissions must be set at least as follows
```
drwxrwxr-x. 4 root hadoop 4096 Aug 3 22:20 /var/lib/ambari-agent/data/tmp/
```
However, at the time this directory is created, it is not known what the name of the _hadoop_ group is, so the next best solution is to set the permissions as
```
drwxrwxrwx. 4 root root 4096 Aug 3 22:20 /var/lib/ambari-agent/data/tmp/
```
If the ambari-agent is installed manually via the relevant package manager, the directory is created with the open permissions (777, drwxrwxrwx) via the packages install_helper.sh post install script. However if Ambari installs the agent via SSH, the directory is created with the more restrictive permissions (755, drwxr-xr-x) via the agent bootstrap.py script.
To make these consistent, the following needs to be changed
#bootstrap.py:650
```
command = "sudo mkdir -p {0} ; sudo chown -R {1} {0} ; sudo chmod 755 {3} ; sudo chmod 755 {2} ; sudo chmod 755 {0}".format(
self.TEMP_FOLDER, quote_bash_args(params.user), DEFAULT_AGENT_DATA_FOLDER, DEFAULT_AGENT_LIB_FOLDER)
```
to
```
command = "sudo mkdir -p {0} ; sudo chown -R {1} {0} ; sudo chmod 755 {3} ; sudo chmod 755 {2} ; sudo chmod 777 {0}".format(
self.TEMP_FOLDER, quote_bash_args(params.user), DEFAULT_AGENT_DATA_FOLDER, DEFAULT_AGENT_LIB_FOLDER)
```
*Note:* self.TEMP_FOLDER contains the path to the Ambari agent temp folder (typically, /var/lib/ambari-agent/data/tmp).
Diffs
-----
ambari-server/src/main/python/bootstrap.py accee81
ambari-server/src/test/python/TestBootstrap.py 48d1687
Diff: https://reviews.apache.org/r/37094/diff/
Testing
-------
Manually tested to see that the directory was created as expected.
# Local test results:
test_createTargetDir (TestBootstrap.TestBootstrap) ... ok
# Jenkins test results: PENDING
Thanks,
Robert Levas
Re: Review Request 37094: Kerberos: fails check during enable Kerb
with SLES
Posted by Dmytro Sen <ds...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37094/#review94743
-----------------------------------------------------------
Ship it!
Ship It!
- Dmytro Sen
On Авг. 10, 2015, 2:38 п.п., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37094/
> -----------------------------------------------------------
>
> (Updated Авг. 10, 2015, 2:38 п.п.)
>
>
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, Jayush Luniya, and Mahadev Konar.
>
>
> Bugs: AMBARI-12636
> https://issues.apache.org/jira/browse/AMBARI-12636
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When executing the Kerberos service check, the following error occurs:
> ```
> stderr: /var/lib/ambari-agent/data/errors-24.txt
>
> Traceback (most recent call last):
> File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/service_check.py", line 81, in <module>
> KerberosServiceCheck().execute()
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 218, in execute
> method(env)
> File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/service_check.py", line 64, in service_check
> user=params.smoke_user
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 157, in __init__
> self.env.run()
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 152, in run
> self.run_action(resource, action)
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 118, in run_action
> provider_action()
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 258, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
> result = function(command, **kwargs)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
> tries=tries, try_sleep=try_sleep)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -c /var/lib/ambari-agent/data/tmp/kerberos_service_check_cc_dd529fe1e15538ddfe9ce0347604d64c -kt /etc/security/keytabs/kerberos.service_check.080315.keytab MyCluster-080315@EXAMPLE.COM' returned 1. kinit(v5): Credentials cache permissions incorrect when initializing cache /var/lib/ambari-agent/data/tmp/kerberos_service_check_cc_dd529fe1e15538ddfe9ce0347604d64c
>
> stdout: /var/lib/ambari-agent/data/output-24.txt
>
> Performing kinit using MyCluster-080315@EXAMPLE.COM
> 2015-08-03 19:11:57,085 - Execute['/usr/bin/kinit -c /var/lib/ambari-agent/data/tmp/kerberos_service_check_cc_dd529fe1e15538ddfe9ce0347604d64c -kt /etc/security/keytabs/kerberos.service_check.080315.keytab MyCluster-080315@EXAMPLE.COM'] {'user': 'ambari-qa'}
> 2015-08-03 19:11:57,179 - File['/var/lib/ambari-agent/data/tmp/kerberos_service_check_cc_dd529fe1e15538ddfe9ce0347604d64c'] {'action': ['delete']}
> ```
>
> This error happens only on SLES, however the cause exists on all platforms. The other platforms silently ignore the condition; which, however, does not have any bearing on the results of the _kinit_ test.
>
> #Cause
> The "Credentials cache permissions incorrect when initializing cache" issue is caused by the inability to write the Kerberos ticket cache file to the specified location. In the case it is /var/lib/ambari-agent/data/tmp/kerberos_service_check_cc_dd529fe1e15538ddfe9ce0347604d64c. The reason for the write failure is that /var/lib/ambari-agent/data/tmp is not writable by the user executing the _kinit_ call - which is the Ambari smoke test user (typically ambari-qa). The directory's permissions are
> ```
> drwxr-xr-x. 4 root root 4096 Aug 3 22:20 /var/lib/ambari-agent/data/tmp/
> ```
> #Solution
> In order for the ambari smoke test user to be able to write to the relevant directory (/var/lib/ambari-agent/data/tmp), the permissions must be set at least as follows
> ```
> drwxrwxr-x. 4 root hadoop 4096 Aug 3 22:20 /var/lib/ambari-agent/data/tmp/
> ```
>
> However, at the time this directory is created, it is not known what the name of the _hadoop_ group is, so the next best solution is to set the permissions as
> ```
> drwxrwxrwx. 4 root root 4096 Aug 3 22:20 /var/lib/ambari-agent/data/tmp/
> ```
>
> If the ambari-agent is installed manually via the relevant package manager, the directory is created with the open permissions (777, drwxrwxrwx) via the packages install_helper.sh post install script. However if Ambari installs the agent via SSH, the directory is created with the more restrictive permissions (755, drwxr-xr-x) via the agent bootstrap.py script.
>
> To make these consistent, the following needs to be changed
> #bootstrap.py:650
> ```
> command = "sudo mkdir -p {0} ; sudo chown -R {1} {0} ; sudo chmod 755 {3} ; sudo chmod 755 {2} ; sudo chmod 755 {0}".format(
> self.TEMP_FOLDER, quote_bash_args(params.user), DEFAULT_AGENT_DATA_FOLDER, DEFAULT_AGENT_LIB_FOLDER)
> ```
> to
> ```
> command = "sudo mkdir -p {0} ; sudo chown -R {1} {0} ; sudo chmod 755 {3} ; sudo chmod 755 {2} ; sudo chmod 777 {0}".format(
> self.TEMP_FOLDER, quote_bash_args(params.user), DEFAULT_AGENT_DATA_FOLDER, DEFAULT_AGENT_LIB_FOLDER)
> ```
>
> *Note:* self.TEMP_FOLDER contains the path to the Ambari agent temp folder (typically, /var/lib/ambari-agent/data/tmp).
>
>
> Diffs
> -----
>
> ambari-server/src/main/python/bootstrap.py accee81
> ambari-server/src/test/python/TestBootstrap.py 48d1687
>
> Diff: https://reviews.apache.org/r/37094/diff/
>
>
> Testing
> -------
>
> Manually tested to see that the directory was created as expected.
>
> # Local test results:
> test_createTargetDir (TestBootstrap.TestBootstrap) ... ok
>
> # Jenkins test results:
>
> test_createTargetDir (TestBootstrap.TestBootstrap) ... ok
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 01:12 h
> [INFO] Finished at: 2015-08-04T22:45:57+00:00
> [INFO] Final Memory: 47M/667M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 37094: Kerberos: fails check during enable Kerb
with SLES
Posted by Robert Levas <rl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37094/
-----------------------------------------------------------
(Updated Aug. 10, 2015, 10:38 a.m.)
Review request for Ambari, Andrew Onischuk, Dmytro Sen, Jayush Luniya, and Mahadev Konar.
Bugs: AMBARI-12636
https://issues.apache.org/jira/browse/AMBARI-12636
Repository: ambari
Description
-------
When executing the Kerberos service check, the following error occurs:
```
stderr: /var/lib/ambari-agent/data/errors-24.txt
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/service_check.py", line 81, in <module>
KerberosServiceCheck().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 218, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/service_check.py", line 64, in service_check
user=params.smoke_user
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 157, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 152, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 118, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 258, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
result = function(command, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
tries=tries, try_sleep=try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -c /var/lib/ambari-agent/data/tmp/kerberos_service_check_cc_dd529fe1e15538ddfe9ce0347604d64c -kt /etc/security/keytabs/kerberos.service_check.080315.keytab MyCluster-080315@EXAMPLE.COM' returned 1. kinit(v5): Credentials cache permissions incorrect when initializing cache /var/lib/ambari-agent/data/tmp/kerberos_service_check_cc_dd529fe1e15538ddfe9ce0347604d64c
stdout: /var/lib/ambari-agent/data/output-24.txt
Performing kinit using MyCluster-080315@EXAMPLE.COM
2015-08-03 19:11:57,085 - Execute['/usr/bin/kinit -c /var/lib/ambari-agent/data/tmp/kerberos_service_check_cc_dd529fe1e15538ddfe9ce0347604d64c -kt /etc/security/keytabs/kerberos.service_check.080315.keytab MyCluster-080315@EXAMPLE.COM'] {'user': 'ambari-qa'}
2015-08-03 19:11:57,179 - File['/var/lib/ambari-agent/data/tmp/kerberos_service_check_cc_dd529fe1e15538ddfe9ce0347604d64c'] {'action': ['delete']}
```
This error happens only on SLES, however the cause exists on all platforms. The other platforms silently ignore the condition; which, however, does not have any bearing on the results of the _kinit_ test.
#Cause
The "Credentials cache permissions incorrect when initializing cache" issue is caused by the inability to write the Kerberos ticket cache file to the specified location. In the case it is /var/lib/ambari-agent/data/tmp/kerberos_service_check_cc_dd529fe1e15538ddfe9ce0347604d64c. The reason for the write failure is that /var/lib/ambari-agent/data/tmp is not writable by the user executing the _kinit_ call - which is the Ambari smoke test user (typically ambari-qa). The directory's permissions are
```
drwxr-xr-x. 4 root root 4096 Aug 3 22:20 /var/lib/ambari-agent/data/tmp/
```
#Solution
In order for the ambari smoke test user to be able to write to the relevant directory (/var/lib/ambari-agent/data/tmp), the permissions must be set at least as follows
```
drwxrwxr-x. 4 root hadoop 4096 Aug 3 22:20 /var/lib/ambari-agent/data/tmp/
```
However, at the time this directory is created, it is not known what the name of the _hadoop_ group is, so the next best solution is to set the permissions as
```
drwxrwxrwx. 4 root root 4096 Aug 3 22:20 /var/lib/ambari-agent/data/tmp/
```
If the ambari-agent is installed manually via the relevant package manager, the directory is created with the open permissions (777, drwxrwxrwx) via the packages install_helper.sh post install script. However if Ambari installs the agent via SSH, the directory is created with the more restrictive permissions (755, drwxr-xr-x) via the agent bootstrap.py script.
To make these consistent, the following needs to be changed
#bootstrap.py:650
```
command = "sudo mkdir -p {0} ; sudo chown -R {1} {0} ; sudo chmod 755 {3} ; sudo chmod 755 {2} ; sudo chmod 755 {0}".format(
self.TEMP_FOLDER, quote_bash_args(params.user), DEFAULT_AGENT_DATA_FOLDER, DEFAULT_AGENT_LIB_FOLDER)
```
to
```
command = "sudo mkdir -p {0} ; sudo chown -R {1} {0} ; sudo chmod 755 {3} ; sudo chmod 755 {2} ; sudo chmod 777 {0}".format(
self.TEMP_FOLDER, quote_bash_args(params.user), DEFAULT_AGENT_DATA_FOLDER, DEFAULT_AGENT_LIB_FOLDER)
```
*Note:* self.TEMP_FOLDER contains the path to the Ambari agent temp folder (typically, /var/lib/ambari-agent/data/tmp).
Diffs
-----
ambari-server/src/main/python/bootstrap.py accee81
ambari-server/src/test/python/TestBootstrap.py 48d1687
Diff: https://reviews.apache.org/r/37094/diff/
Testing (updated)
-------
Manually tested to see that the directory was created as expected.
# Local test results:
test_createTargetDir (TestBootstrap.TestBootstrap) ... ok
# Jenkins test results:
test_createTargetDir (TestBootstrap.TestBootstrap) ... ok
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:12 h
[INFO] Finished at: 2015-08-04T22:45:57+00:00
[INFO] Final Memory: 47M/667M
[INFO] ------------------------------------------------------------------------
Thanks,
Robert Levas