You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Dmytro Sen <ds...@hortonworks.com> on 2015/03/03 20:53:35 UTC

Review Request 31639: HiveMetastore in secure mode throws errors

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31639/
-----------------------------------------------------------

Review request for Ambari, Dmitro Lisnichenko and Vitalyi Brodetskyi.


Bugs: AMBARI-9881
    https://issues.apache.org/jira/browse/AMBARI-9881


Repository: ambari


Description
-------

HiveMetastore is "green" and looks like works fine. But in hiveserver2 logs, you can see this:
2015-02-10 14:41:06,023 INFO  [HiveServer2-Handler-Pool: Thread-38]: thrift.ThriftCLIService (ThriftCLIService.java:OpenSession(232)) - Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V6
2015-02-10 14:41:06,039 ERROR [HiveServer2-Handler-Pool: Thread-38]: metadata.Hive (Hive.java:getDelegationToken(2910)) - java.lang.UnsupportedOperationException: getDelegationToken() can be called only in thrift (non local) mode
	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDelegationToken(HiveMetaStoreClient.java:1664)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
	at com.sun.proxy.$Proxy12.getDelegationToken(Unknown Source)
	at org.apache.hadoop.hive.ql.metadata.Hive.getDelegationToken(Hive.java:2908)
	at org.apache.hive.service.cli.CLIService.getDelegationTokenFromMetaStore(CLIService.java:467)
	at org.apache.hive.service.cli.thrift.ThriftCLIService.getDelegationToken(ThriftCLIService.java:340)
	at org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:321)
	at org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:235)
	at org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1253)
	at org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1238)
	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge20S.java:679)
	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
In HiveMetastore logs i see such errors:
2015-02-10 14:46:51,541 ERROR [pool-5-thread-7]: server.TThreadPoolServer (TThreadPoolServer.java:run(215)) - Error occurred during processing of message.
java.lang.RuntimeException: org.apache.thrift.transport.TTransportException
	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$HiveSaslServerTransportFactory.getTransport(HadoopThriftAuthBridge.java:180)
	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge20S.java:726)
	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge20S.java:723)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:356)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1608)
	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge20S.java:723)
	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:189)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.thrift.transport.TTransportException
	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$HiveSaslServerTransportFactory$1.receiveSaslMessage(HadoopThriftAuthBridge.java:197)
	at org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125)
	at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253)
	at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$HiveSaslServerTransportFactory.getTransport(HadoopThriftAuthBridge.java:177)
	... 10 more
looks like it's not good and should be fixed


Diffs
-----

  ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/alerts.json 750f586 
  ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_metastore.py PRE-CREATION 

Diff: https://reviews.apache.org/r/31639/diff/


Testing
-------

all tests passed


Thanks,

Dmytro Sen

Re: Review Request 31639: HiveMetastore in secure mode throws errors

Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31639/#review75161
-----------------------------------------------------------

Ship it!


Ship It!

- Dmitro Lisnichenko


On March 3, 2015, 7:53 p.m., Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31639/
> -----------------------------------------------------------
> 
> (Updated March 3, 2015, 7:53 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-9881
>     https://issues.apache.org/jira/browse/AMBARI-9881
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> HiveMetastore is "green" and looks like works fine. But in hiveserver2 logs, you can see this:
> 2015-02-10 14:41:06,023 INFO  [HiveServer2-Handler-Pool: Thread-38]: thrift.ThriftCLIService (ThriftCLIService.java:OpenSession(232)) - Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V6
> 2015-02-10 14:41:06,039 ERROR [HiveServer2-Handler-Pool: Thread-38]: metadata.Hive (Hive.java:getDelegationToken(2910)) - java.lang.UnsupportedOperationException: getDelegationToken() can be called only in thrift (non local) mode
> 	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDelegationToken(HiveMetaStoreClient.java:1664)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
> 	at com.sun.proxy.$Proxy12.getDelegationToken(Unknown Source)
> 	at org.apache.hadoop.hive.ql.metadata.Hive.getDelegationToken(Hive.java:2908)
> 	at org.apache.hive.service.cli.CLIService.getDelegationTokenFromMetaStore(CLIService.java:467)
> 	at org.apache.hive.service.cli.thrift.ThriftCLIService.getDelegationToken(ThriftCLIService.java:340)
> 	at org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:321)
> 	at org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:235)
> 	at org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1253)
> 	at org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1238)
> 	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> 	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> 	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge20S.java:679)
> 	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:745)
> In HiveMetastore logs i see such errors:
> 2015-02-10 14:46:51,541 ERROR [pool-5-thread-7]: server.TThreadPoolServer (TThreadPoolServer.java:run(215)) - Error occurred during processing of message.
> java.lang.RuntimeException: org.apache.thrift.transport.TTransportException
> 	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$HiveSaslServerTransportFactory.getTransport(HadoopThriftAuthBridge.java:180)
> 	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge20S.java:726)
> 	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge20S.java:723)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:356)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1608)
> 	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge20S.java:723)
> 	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:189)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.thrift.transport.TTransportException
> 	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$HiveSaslServerTransportFactory$1.receiveSaslMessage(HadoopThriftAuthBridge.java:197)
> 	at org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125)
> 	at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253)
> 	at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
> 	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$HiveSaslServerTransportFactory.getTransport(HadoopThriftAuthBridge.java:177)
> 	... 10 more
> looks like it's not good and should be fixed
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/alerts.json 750f586 
>   ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_metastore.py PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/31639/diff/
> 
> 
> Testing
> -------
> 
> all tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 31639: HiveMetastore in secure mode throws errors

Posted by Vitalyi Brodetskyi <vb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31639/#review75160
-----------------------------------------------------------

Ship it!


Ship It!

- Vitalyi Brodetskyi


On Березень 3, 2015, 7:53 після полудня, Dmytro Sen wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31639/
> -----------------------------------------------------------
> 
> (Updated Березень 3, 2015, 7:53 після полудня)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-9881
>     https://issues.apache.org/jira/browse/AMBARI-9881
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> HiveMetastore is "green" and looks like works fine. But in hiveserver2 logs, you can see this:
> 2015-02-10 14:41:06,023 INFO  [HiveServer2-Handler-Pool: Thread-38]: thrift.ThriftCLIService (ThriftCLIService.java:OpenSession(232)) - Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V6
> 2015-02-10 14:41:06,039 ERROR [HiveServer2-Handler-Pool: Thread-38]: metadata.Hive (Hive.java:getDelegationToken(2910)) - java.lang.UnsupportedOperationException: getDelegationToken() can be called only in thrift (non local) mode
> 	at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDelegationToken(HiveMetaStoreClient.java:1664)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90)
> 	at com.sun.proxy.$Proxy12.getDelegationToken(Unknown Source)
> 	at org.apache.hadoop.hive.ql.metadata.Hive.getDelegationToken(Hive.java:2908)
> 	at org.apache.hive.service.cli.CLIService.getDelegationTokenFromMetaStore(CLIService.java:467)
> 	at org.apache.hive.service.cli.thrift.ThriftCLIService.getDelegationToken(ThriftCLIService.java:340)
> 	at org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:321)
> 	at org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:235)
> 	at org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1253)
> 	at org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1238)
> 	at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
> 	at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
> 	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge20S.java:679)
> 	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:745)
> In HiveMetastore logs i see such errors:
> 2015-02-10 14:46:51,541 ERROR [pool-5-thread-7]: server.TThreadPoolServer (TThreadPoolServer.java:run(215)) - Error occurred during processing of message.
> java.lang.RuntimeException: org.apache.thrift.transport.TTransportException
> 	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$HiveSaslServerTransportFactory.getTransport(HadoopThriftAuthBridge.java:180)
> 	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge20S.java:726)
> 	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory$1.run(HadoopThriftAuthBridge20S.java:723)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:356)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1608)
> 	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge20S$Server$TUGIAssumingTransportFactory.getTransport(HadoopThriftAuthBridge20S.java:723)
> 	at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:189)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.thrift.transport.TTransportException
> 	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
> 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
> 	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$HiveSaslServerTransportFactory$1.receiveSaslMessage(HadoopThriftAuthBridge.java:197)
> 	at org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125)
> 	at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253)
> 	at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
> 	at org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$HiveSaslServerTransportFactory.getTransport(HadoopThriftAuthBridge.java:177)
> 	... 10 more
> looks like it's not good and should be fixed
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/alerts.json 750f586 
>   ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_metastore.py PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/31639/diff/
> 
> 
> Testing
> -------
> 
> all tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>