You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Robert Levas <rl...@hortonworks.com> on 2015/06/27 13:14:39 UTC
Review Request 35970: Enabling Kerberos on cluster with AMS and no
HDFS fails
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35970/
-----------------------------------------------------------
Review request for Ambari, Emil Anca, Mahadev Konar, Sumit Mohanty, and Tom Beerbower.
Bugs: AMBARI-12180
https://issues.apache.org/jira/browse/AMBARI-12180
Repository: ambari
Description
-------
In a cluster where AMS is installed but HDFS is _not_ installed, enabling Kerberos fails due to the inability for the server-side Kerberos logic to replace ${hadoop-env/hdfs_user} when generating the metadata used to create principals and distribute keytab files.
This condition yields the following principal (when the cluster name is AMSNOHDFS and the realm is EXAMPLE.COM)
```
${hadoop-env/hdfs_user}-AMSNOHDFS@EXAMPLE.COM
```
This is successfully created in the (MIT) KDC. Also, the relative keytab file appears to have been successfully created as well.
However, when distributing the keytab file and setting the ownership attributes, the agent-side script fails with
```
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/kerberos_client.py", line 77, in <module>
KerberosClient().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 216, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/kerberos_client.py", line 67, in set_keytab
self.write_keytab_file()
File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/kerberos_common.py", line 397, in write_keytab_file
group=group)
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 157, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 152, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 118, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 108, in action_create
self.resource.group, mode=self.resource.mode, cd_access=self.resource.cd_access)
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 44, in _ensure_metadata
_user_entity = pwd.getpwnam(user)
KeyError: 'getpwnam(): name not found: ${hadoop-env/hdfs_user}'
```
#Solution:
Remove the HDFS identity reference in AMS and assume the hdfs keytab file will be on the appropriate host(s) when HDFS is installed
Diffs
-----
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json 6010b2f
Diff: https://reviews.apache.org/r/35970/diff/
Testing
-------
Manually tested in cluster with Zookeeper and AMS, not HDFS
#Local tests results:
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 46:29.766s
[INFO] Finished at: Fri Jun 26 22:23:21 EDT 2015
[INFO] Final Memory: 65M/1251M
[INFO] ------------------------------------------------------------------------
#Jenkins test results: PENDING
Thanks,
Robert Levas
Re: Review Request 35970: Enabling Kerberos on cluster with AMS and
no HDFS fails
Posted by Sumit Mohanty <sm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35970/#review89620
-----------------------------------------------------------
Ship it!
Ship It!
- Sumit Mohanty
On June 27, 2015, 11:14 a.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35970/
> -----------------------------------------------------------
>
> (Updated June 27, 2015, 11:14 a.m.)
>
>
> Review request for Ambari, Emil Anca, Mahadev Konar, Sumit Mohanty, and Tom Beerbower.
>
>
> Bugs: AMBARI-12180
> https://issues.apache.org/jira/browse/AMBARI-12180
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In a cluster where AMS is installed but HDFS is _not_ installed, enabling Kerberos fails due to the inability for the server-side Kerberos logic to replace ${hadoop-env/hdfs_user} when generating the metadata used to create principals and distribute keytab files.
>
> This condition yields the following principal (when the cluster name is AMSNOHDFS and the realm is EXAMPLE.COM)
> ```
> ${hadoop-env/hdfs_user}-AMSNOHDFS@EXAMPLE.COM
> ```
>
> This is successfully created in the (MIT) KDC. Also, the relative keytab file appears to have been successfully created as well.
>
> However, when distributing the keytab file and setting the ownership attributes, the agent-side script fails with
> ```
> Traceback (most recent call last):
> File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/kerberos_client.py", line 77, in <module>
> KerberosClient().execute()
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 216, in execute
> method(env)
> File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/kerberos_client.py", line 67, in set_keytab
> self.write_keytab_file()
> File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/kerberos_common.py", line 397, in write_keytab_file
> group=group)
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 157, in __init__
> self.env.run()
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 152, in run
> self.run_action(resource, action)
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 118, in run_action
> provider_action()
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 108, in action_create
> self.resource.group, mode=self.resource.mode, cd_access=self.resource.cd_access)
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 44, in _ensure_metadata
> _user_entity = pwd.getpwnam(user)
> KeyError: 'getpwnam(): name not found: ${hadoop-env/hdfs_user}'
> ```
>
> #Solution:
> Remove the HDFS identity reference in AMS and assume the hdfs keytab file will be on the appropriate host(s) when HDFS is installed
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json 6010b2f
>
> Diff: https://reviews.apache.org/r/35970/diff/
>
>
> Testing
> -------
>
> Manually tested in cluster with Zookeeper and AMS, not HDFS
>
> #Local tests results:
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 46:29.766s
> [INFO] Finished at: Fri Jun 26 22:23:21 EDT 2015
> [INFO] Final Memory: 65M/1251M
> [INFO] ------------------------------------------------------------------------
>
> #Jenkins test results: PENDING
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 35970: Enabling Kerberos on cluster with AMS and
no HDFS fails
Posted by Tom Beerbower <tb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35970/#review89621
-----------------------------------------------------------
Ship it!
Ship It!
- Tom Beerbower
On June 27, 2015, 11:14 a.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35970/
> -----------------------------------------------------------
>
> (Updated June 27, 2015, 11:14 a.m.)
>
>
> Review request for Ambari, Emil Anca, Mahadev Konar, Sumit Mohanty, and Tom Beerbower.
>
>
> Bugs: AMBARI-12180
> https://issues.apache.org/jira/browse/AMBARI-12180
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In a cluster where AMS is installed but HDFS is _not_ installed, enabling Kerberos fails due to the inability for the server-side Kerberos logic to replace ${hadoop-env/hdfs_user} when generating the metadata used to create principals and distribute keytab files.
>
> This condition yields the following principal (when the cluster name is AMSNOHDFS and the realm is EXAMPLE.COM)
> ```
> ${hadoop-env/hdfs_user}-AMSNOHDFS@EXAMPLE.COM
> ```
>
> This is successfully created in the (MIT) KDC. Also, the relative keytab file appears to have been successfully created as well.
>
> However, when distributing the keytab file and setting the ownership attributes, the agent-side script fails with
> ```
> Traceback (most recent call last):
> File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/kerberos_client.py", line 77, in <module>
> KerberosClient().execute()
> File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 216, in execute
> method(env)
> File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/kerberos_client.py", line 67, in set_keytab
> self.write_keytab_file()
> File "/var/lib/ambari-agent/cache/common-services/KERBEROS/1.10.3-10/package/scripts/kerberos_common.py", line 397, in write_keytab_file
> group=group)
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 157, in __init__
> self.env.run()
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 152, in run
> self.run_action(resource, action)
> File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 118, in run_action
> provider_action()
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 108, in action_create
> self.resource.group, mode=self.resource.mode, cd_access=self.resource.cd_access)
> File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 44, in _ensure_metadata
> _user_entity = pwd.getpwnam(user)
> KeyError: 'getpwnam(): name not found: ${hadoop-env/hdfs_user}'
> ```
>
> #Solution:
> Remove the HDFS identity reference in AMS and assume the hdfs keytab file will be on the appropriate host(s) when HDFS is installed
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/kerberos.json 6010b2f
>
> Diff: https://reviews.apache.org/r/35970/diff/
>
>
> Testing
> -------
>
> Manually tested in cluster with Zookeeper and AMS, not HDFS
>
> #Local tests results:
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 46:29.766s
> [INFO] Finished at: Fri Jun 26 22:23:21 EDT 2015
> [INFO] Final Memory: 65M/1251M
> [INFO] ------------------------------------------------------------------------
>
> #Jenkins test results: PENDING
>
>
> Thanks,
>
> Robert Levas
>
>