Need help with ThriftServer/Spark1.1.0

[Yana Kadiyska <ya...@gmail.com>]

Hi ladies and gents,

trying to get Thrift server up and running in an effort to replace Shark.

My first attempt to run sbin/start-thriftserver resulted in:

14/09/15 17:09:05 ERROR TThreadPoolServer: Error occurred during processing
of message.
java.lang.RuntimeException:
org.apache.thrift.transport.TTransportException: java.net.SocketException:
Connection reset
        at
org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:219)
        at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:189)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.thrift.transport.TTransportException:
java.net.SocketException: Connection reset
        at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:129)
        at
org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
        at
org.apache.thrift.transport.TSaslTransport.receiveSaslMessage(TSaslTransport.java:178)
        at
org.apache.thrift.transport.TSaslServerTransport.handleSaslStartMessage(TSaslServerTransport.java:125)
        at
org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:253)
        at
org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41)
        at
org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216)
        ... 4 more
Caused by: java.net.SocketException: Connection reset
        at java.net.SocketInputStream.read(SocketInputStream.java:196)
        at java.net.SocketInputStream.read(SocketInputStream.java:122)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
        at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)

After turing logging levels up it seemed like this error is related to SASL
and the SO advice was to turn it off
via: <property><name>hive.server2.authentication</name><value>NOSASL</value></property>

But I still have no luck:
(this is the full command that gets run)

java -cp
/spark-1.1.0-bin-cdh4/conf:/spark-1.1.0-bin-cdh4/lib/spark-assembly-1.1.0-hadoop2.0.0-mr1-cdh4.2.0.jar:/spark-1.1.0-bin-cdh4/lib/datanucleus-core-3.2.2.jar:/spark-1.1.0-bin-cdh4/lib/datanucleus-rdbms-3.2.1.jar:/a/shark/spark-1.1.0-bin-cdh4/lib/datanucleus-api-jdo-3.2.1.jar:/hadoop/share/hadoop/mapreduce1//conf
-XX:MaxPermSize=128m-Xms4012m -Xmx4012m org.apache.spark.deploy.SparkSubmit
--class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 --master
spark://master-ip:7077 spark-internal --hiveconf
hive.server.thrift.bind.host ip-to-bind

14/09/15 17:05:05 ERROR TThreadPoolServer:
Error occurred during processing of message.
java.lang.ClassCastException: org.apache.thrift.transport.TSocket cannot be
cast to org.apache.thrift.transport.TSaslServerTransport
        at
org.apache.hive.service.auth.TUGIContainingProcessor.process(TUGIContainingProcessor.java:53)
        at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)

Any idea what might be going on? I compiled w/ -Phive against the 1.1.0.
hive-site.conf is the conf file we used previously. SparkSQL does work for
me  but does not have  a lot of functionality I need.

Any help appreciated -- I do acklnowledge this is likely more of a Hive
question than spark...If there is a precompiled version of CDH4 that
includes thrift-server I'd be happy to try that too...

thanks again.