You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Kuldeep (JIRA)" <ji...@apache.org> on 2015/07/31 09:57:04 UTC
[jira] [Comment Edited] (SPARK-9333) JDBC connection to hivethrift
server broken on 1.4.1
[ https://issues.apache.org/jira/browse/SPARK-9333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14648848#comment-14648848 ]
Kuldeep edited comment on SPARK-9333 at 7/31/15 7:56 AM:
---------------------------------------------------------
Seems like I was on spark 1.3.1.
Bigger problems than this with 1.4.1. JDBC connection is not working for me. With 1.4.1 I am getting following on server
{code}
2015-07-31 12:25:29,477 INFO org.apache.hive.service.cli.thrift.ThriftCLIService: Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V6
2015-07-31 12:25:29,480 INFO org.apache.hadoop.hive.metastore.HiveMetaStore: No user is added in admin role, since config is empty
2015-07-31 12:25:29,480 INFO org.apache.hadoop.hive.ql.session.SessionState: No Tez session required at this point. hive.execution.engine=mr.
2015-07-31 12:25:29,481 INFO org.apache.hadoop.hive.ql.session.SessionState: No Tez session required at this point. hive.execution.engine=mr.
2015-07-31 12:25:29,482 WARN org.apache.hive.service.cli.thrift.ThriftCLIService: Error opening session:
java.lang.NullPointerException
at org.apache.spark.sql.hive.thriftserver.SparkSQLSessionManager.openSession(Shim13.scala:244)
at org.apache.hive.service.cli.CLIService.openSessionWithImpersonation(CLIService.java:161)
at org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:265)
at org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:191)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1253)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1238)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:55)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
And this on client
{code}org.apache.thrift.protocol.TProtocolException: Required field 'sessionHandle' is unset! Struct:TCloseSessionReq(sessionHandle:null)
{code}
was (Author: kul):
Seems like I was on spark 1.3.1.
Bigger problems than this with 1.4.1. JDBC connection is not working for me. With 1.4.1 I am getting following on server
{code}
2015-07-31 12:25:29,477 INFO org.apache.hive.service.cli.thrift.ThriftCLIService: Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V6
2015-07-31 12:25:29,480 INFO org.apache.hadoop.hive.metastore.HiveMetaStore: No user is added in admin role, since config is empty
2015-07-31 12:25:29,480 INFO org.apache.hadoop.hive.ql.session.SessionState: No Tez session required at this point. hive.execution.engine=mr.
2015-07-31 12:25:29,481 INFO org.apache.hadoop.hive.ql.session.SessionState: No Tez session required at this point. hive.execution.engine=mr.
2015-07-31 12:25:29,482 WARN org.apache.hive.service.cli.thrift.ThriftCLIService: Error opening session:
java.lang.NullPointerException
at org.apache.spark.sql.hive.thriftserver.SparkSQLSessionManager.openSession(Shim13.scala:244)
at org.apache.hive.service.cli.CLIService.openSessionWithImpersonation(CLIService.java:161)
at org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:265)
at org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:191)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1253)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1238)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:55)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}
And this on client
{code}
java.sql.SQLException: Error while cleaning up the server resources org.apache.thrift.protocol.TProtocolException: Required field 'sessionHandle' is unset! Struct:TCloseSessionReq(sessionHandle:null)
{code}
> JDBC connection to hivethrift server broken on 1.4.1
> ----------------------------------------------------
>
> Key: SPARK-9333
> URL: https://issues.apache.org/jira/browse/SPARK-9333
> Project: Spark
> Issue Type: New Feature
> Components: SQL
> Affects Versions: 1.4.1
> Reporter: Kuldeep
>
> -Right now there is `registerTempTable` which registers table in sql/hive context. Then there is `save` which registers it to hivestore but materializes the df. Would it make sense to have a mid-path like `registerTable` which registers to hivestore and points to table in sql context without materializing it?-
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org