You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Robert Levas <rl...@hortonworks.com> on 2015/09/17 19:58:03 UTC
Review Request 38466: Hive Metastore did not start when Kerberized
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38466/
-----------------------------------------------------------
Review request for Ambari, Andrew Onischuk and Yusaku Sako.
Bugs: AMBARI-13133
https://issues.apache.org/jira/browse/AMBARI-13133
Repository: ambari
Description
-------
When starting up HiveMetastore under a Kerberized cluster, the following error occurs:
```
resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt /etc/security/keytabs/hive.service.keytab hive/host1.company.com@REALM; ' returned 1. kinit: Keytab contains no suitable keys for hive/host1.company.com@REALM while getting initial credentials
```
This happens when Hive Metastore and HiveServer2 principals are set up distinct from each other.
Hive Metastore is not using hive.metastore.kerberos.principal, but instead it uses hive.server2.authentication.kerberos.principal
Also, the following references hive_conf_dir:
https://github.com/apache/ambari/blob/release-2.1.1/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py#L119-L120
In HDP2.3+ the following file content becomes UNSECURED
/var/lib/ambari-agent/data/structured-out-status.json
We need to either reference hive_server_conf_dir or set hive_conf_dir as hive_server_conf_dir somewhere:
https://github.com/apache/ambari/blob/release-2.1.1/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/status_params.py#L90-L101
*Solution*
Since a kinit call here is unnecessary and the relevant configuration files are being created properly. Simply removing the kinit call (and related variabled) will fix the kinit failure issue.
For the hive_conf_dir issue, setting {{hive_conf_dir = hive_server_conf_dir}} in status_params.py, solves the issue.
Diffs
-----
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_service.py cab6d0d
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/status_params.py a9b6628
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_metastore.py a39fac7
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py ef1ce7f
ambari-server/src/test/python/stacks/2.1/HIVE/test_hive_metastore.py 7075345
Diff: https://reviews.apache.org/r/38466/diff/
Testing
-------
Local test results: PASSED
Jenkins test results: PENDING
Thanks,
Robert Levas
Re: Review Request 38466: Hive Metastore did not start when Kerberized
Posted by Andrew Onischuk <ao...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38466/#review99506
-----------------------------------------------------------
Ship it!
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_service.py (line 82)
<https://reviews.apache.org/r/38466/#comment156395>
Doesn't daemon_cmd require kinit beforehand? I think it's not cause you probably tested it :) But I wonder why is so.
- Andrew Onischuk
On Sept. 17, 2015, 10:34 p.m., Robert Levas wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/38466/
> -----------------------------------------------------------
>
> (Updated Sept. 17, 2015, 10:34 p.m.)
>
>
> Review request for Ambari, Andrew Onischuk and Yusaku Sako.
>
>
> Bugs: AMBARI-13133
> https://issues.apache.org/jira/browse/AMBARI-13133
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When starting up HiveMetastore under a Kerberized cluster, the following error occurs:
> ```
> resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt /etc/security/keytabs/hive.service.keytab hive/host1.company.com@REALM; ' returned 1. kinit: Keytab contains no suitable keys for hive/host1.company.com@REALM while getting initial credentials
> ```
>
> This happens when Hive Metastore and HiveServer2 principals are set up distinct from each other.
> Hive Metastore is not using hive.metastore.kerberos.principal, but instead it uses hive.server2.authentication.kerberos.principal
>
> Also, the following references hive_conf_dir:
> https://github.com/apache/ambari/blob/release-2.1.1/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py#L119-L120
>
> In HDP2.3+ the following file content becomes UNSECURED
> /var/lib/ambari-agent/data/structured-out-status.json
>
> We need to either reference hive_server_conf_dir or set hive_conf_dir as hive_server_conf_dir somewhere:
> https://github.com/apache/ambari/blob/release-2.1.1/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/status_params.py#L90-L101
>
> *Solution*
> Since a kinit call here is unnecessary and the relevant configuration files are being created properly. Simply removing the kinit call (and related variabled) will fix the kinit failure issue.
>
> For the hive_conf_dir issue, setting {{hive_conf_dir = hive_server_conf_dir}} in status_params.py, solves the issue.
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_service.py cab6d0d
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/status_params.py a9b6628
> ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_metastore.py a39fac7
> ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py ef1ce7f
> ambari-server/src/test/python/stacks/2.1/HIVE/test_hive_metastore.py 7075345
>
> Diff: https://reviews.apache.org/r/38466/diff/
>
>
> Testing
> -------
>
> #Local test results: PASSED#
>
> #Jenkins test results:#
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 01:22 h
> [INFO] Finished at: 2015-09-17T22:32:39+00:00
> [INFO] Final Memory: 48M/731M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Robert Levas
>
>
Re: Review Request 38466: Hive Metastore did not start when Kerberized
Posted by Robert Levas <rl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/38466/
-----------------------------------------------------------
(Updated Sept. 17, 2015, 6:34 p.m.)
Review request for Ambari, Andrew Onischuk and Yusaku Sako.
Bugs: AMBARI-13133
https://issues.apache.org/jira/browse/AMBARI-13133
Repository: ambari
Description
-------
When starting up HiveMetastore under a Kerberized cluster, the following error occurs:
```
resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt /etc/security/keytabs/hive.service.keytab hive/host1.company.com@REALM; ' returned 1. kinit: Keytab contains no suitable keys for hive/host1.company.com@REALM while getting initial credentials
```
This happens when Hive Metastore and HiveServer2 principals are set up distinct from each other.
Hive Metastore is not using hive.metastore.kerberos.principal, but instead it uses hive.server2.authentication.kerberos.principal
Also, the following references hive_conf_dir:
https://github.com/apache/ambari/blob/release-2.1.1/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py#L119-L120
In HDP2.3+ the following file content becomes UNSECURED
/var/lib/ambari-agent/data/structured-out-status.json
We need to either reference hive_server_conf_dir or set hive_conf_dir as hive_server_conf_dir somewhere:
https://github.com/apache/ambari/blob/release-2.1.1/ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/status_params.py#L90-L101
*Solution*
Since a kinit call here is unnecessary and the relevant configuration files are being created properly. Simply removing the kinit call (and related variabled) will fix the kinit failure issue.
For the hive_conf_dir issue, setting {{hive_conf_dir = hive_server_conf_dir}} in status_params.py, solves the issue.
Diffs
-----
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_service.py cab6d0d
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/status_params.py a9b6628
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_metastore.py a39fac7
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py ef1ce7f
ambari-server/src/test/python/stacks/2.1/HIVE/test_hive_metastore.py 7075345
Diff: https://reviews.apache.org/r/38466/diff/
Testing (updated)
-------
#Local test results: PASSED#
#Jenkins test results:#
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:22 h
[INFO] Finished at: 2015-09-17T22:32:39+00:00
[INFO] Final Memory: 48M/731M
[INFO] ------------------------------------------------------------------------
Thanks,
Robert Levas