You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by "Zack Marsh (JIRA)" <ji...@apache.org> on 2015/06/23 18:41:01 UTC
[jira] [Created] (AMBARI-12096) Hive Service Check failing after
enabling Kerberos
Zack Marsh created AMBARI-12096:
-----------------------------------
Summary: Hive Service Check failing after enabling Kerberos
Key: AMBARI-12096
URL: https://issues.apache.org/jira/browse/AMBARI-12096
Project: Ambari
Issue Type: Bug
Environment: ambari-2.1.0-1213, hdp-2.3.0.0-2450, sles11sp3
Reporter: Zack Marsh
Priority: Critical
After enabling Kerberos, the Hive Service Check is consistently failing.
Errors as seen in Ambari when the Service Check fails:
stderr:
{code}
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/service_check.py", line 106, in <module>
HiveServiceCheck().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 216, in execute
method(env)
File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/service_check.py", line 102, in service_check
hcat_service_check()
File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, in thunk
return fn(*args, **kwargs)
File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hcat_service_check.py", line 65, in hcat_service_check
logoutput=True)
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 157, in __init__
self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 152, in run
self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 118, in run_action
provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py", line 254, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner
result = function(command, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call
tries=tries, try_sleep=try_sleep)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of '/usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa; env JAVA_HOME=/opt/teradata/jvm64/jdk8 /var/lib/ambari-agent/data/tmp/hcatSmoke.sh hcatsmokeid00270328_date262315 prepare' returned 64. kinit(v5): Client not found in Kerberos database while getting initial credentials
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM from IP 39.0.40.2)
WARNING: Use "yarn jar" to launch YARN applications.
OK
Time taken: 1.612 seconds
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM from IP 39.0.40.2)
{code}
stdout:
{code}
Test connectivity to hive server
Waiting for the Hive server to start...
2015-06-23 12:26:21,859 - Execute['/usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa-PIRIPIRI@PIRIPIRI.HADOOP.TERADATA.COM; '] {'user': 'ambari-qa'}
2015-06-23 12:26:21,910 - Execute['! beeline -u 'jdbc:hive2://piripiri2.labs.teradata.com:10000/;transportMode=binary;principal=hive/_HOST@PIRIPIRI.HADOOP.TERADATA.COM' -e '' 2>&1| awk '{print}'|grep -i -e 'Connection refused' -e 'Invalid URL''] {'path': ['/bin/', '/usr/bin/', '/usr/lib/hive/bin/', '/usr/sbin/'], 'user': 'ambari-qa', 'timeout': 30}
Successfully connected to piripiri2.labs.teradata.com on port 10000
Successfully connected to Hive at piripiri3.labs.teradata.com on port 10000 after 3 seconds
2015-06-23 12:26:25,609 - File['/var/lib/ambari-agent/data/tmp/hcatSmoke.sh'] {'content': StaticFile('hcatSmoke.sh'), 'mode': 0755}
2015-06-23 12:26:25,610 - Writing File['/var/lib/ambari-agent/data/tmp/hcatSmoke.sh'] because it doesn't exist
2015-06-23 12:26:25,610 - Changing permission for /var/lib/ambari-agent/data/tmp/hcatSmoke.sh from 644 to 755
2015-06-23 12:26:25,610 - Execute['/usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa; env JAVA_HOME=/opt/teradata/jvm64/jdk8 /var/lib/ambari-agent/data/tmp/hcatSmoke.sh hcatsmokeid00270328_date262315 prepare'] {'logoutput': True, 'path': ['/usr/sbin', '/usr/local/bin', '/bin', '/usr/bin', '/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/sbin:/sbin:/usr/lib/ambari-server/*:/sbin:/usr/sbin:/usr/local/sbin:/root/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/teradata/bynet/bin:/opt/teradata/jvm64/jdk8/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:.:/opt/teradata/sm3g/bin:/opt/dell/srvadmin/bin:/opt/dell/srvadmin/sbin:/opt/teradata/dswap/sbin:/usr/tdbms/bin:/opt/teradata/gsctools/bin:/opt/teradata/vmf/bin:/opt/teradata/bin:/var/lib/ambari-agent:/var/lib/ambari-agent:/usr/hdp/current/hive-client/bin:/usr/hdp/current/hadoop-client/bin'], 'tries': 3, 'user': 'ambari-qa', 'try_sleep': 5}
kinit(v5): Client not found in Kerberos database while getting initial credentials
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM from IP 39.0.40.2)
WARNING: Use "yarn jar" to launch YARN applications.
OK
Time taken: 1.448 seconds
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM from IP 39.0.40.2)
2015-06-23 12:26:39,610 - Retrying after 5 seconds. Reason: Execution of '/usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa; env JAVA_HOME=/opt/teradata/jvm64/jdk8 /var/lib/ambari-agent/data/tmp/hcatSmoke.sh hcatsmokeid00270328_date262315 prepare' returned 64. kinit(v5): Client not found in Kerberos database while getting initial credentials
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM from IP 39.0.40.2)
WARNING: Use "yarn jar" to launch YARN applications.
OK
Time taken: 1.448 seconds
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM from IP 39.0.40.2)
kinit(v5): Client not found in Kerberos database while getting initial credentials
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM from IP 39.0.40.2)
WARNING: Use "yarn jar" to launch YARN applications.
OK
Time taken: 1.42 seconds
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM from IP 39.0.40.2)
2015-06-23 12:26:58,283 - Retrying after 5 seconds. Reason: Execution of '/usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab ambari-qa; env JAVA_HOME=/opt/teradata/jvm64/jdk8 /var/lib/ambari-agent/data/tmp/hcatSmoke.sh hcatsmokeid00270328_date262315 prepare' returned 64. kinit(v5): Client not found in Kerberos database while getting initial credentials
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM from IP 39.0.40.2)
WARNING: Use "yarn jar" to launch YARN applications.
OK
Time taken: 1.42 seconds
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM from IP 39.0.40.2)
kinit(v5): Client not found in Kerberos database while getting initial credentials
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM from IP 39.0.40.2)
WARNING: Use "yarn jar" to launch YARN applications.
OK
Time taken: 1.612 seconds
WARNING: Use "yarn jar" to launch YARN applications.
FAILED: SemanticException MetaException(message:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): Unauthorized connection for super-user: hive/piripiri2.labs.teradata.com@PIRIPIRI.HADOOP.TERADATA.COM from IP 39.0.40.2)
{code}
The cluster has Hive HA enabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)