You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Roc Chu <ch...@gmail.com> on 2014/12/24 11:31:21 UTC

got ”org.apache.thrift.protocol.TProtocolException: Expected protocol id ffffff82 but got ffffff80“ from hive metastroe service when I use show tables command in spark-sql shell

this is my problem. I use mysql to store hive meta data. and i can get what
i want when I exec show tables in hive shell. but in the same machine. I
use spark-sql to execute same command (show tables), I got errors.
I look at the log of hive metastore find this errors

2014-12-24 05:04:59,874 ERROR [pool-3-thread-2]: server.TThreadPoolServer
(TThreadPoolServer.java:run(294)) - Thrift error occurred during processing
of message.
org.apache.thrift.protocol.TProtocolException: Expected protocol id
ffffff82 but got ffffff80
        at
org.apache.thrift.protocol.TCompactProtocol.readMessageBegin(TCompactProtocol.java:503)
        at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:75)
        at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
2014-12-24 05:05:01,883 ERROR [pool-3-thread-3]: server.TThreadPoolServer
(TThreadPoolServer.java:run(294)) - Thrift error occurred during processing
of message.
org.apache.thrift.protocol.TProtocolException: Expected protocol id
ffffff82 but got ffffff80
        at
org.apache.thrift.protocol.TCompactProtocol.readMessageBegin(TCompactProtocol.java:503)
        at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:75)
        at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

I compiled e spark from soucecode of spark-1.1.1
and my hive was compiled from apache-hive-0.15.0

I put the same hive-site.xml both hive conf and apark conf.

waiting for help
thanks

Re: got ”org.apache.thrift.protocol.TProtocolException: Expected protocol id ffffff82 but got ffffff80“ from hive metastroe service when I use show tables command in spark-sql shell

Posted by Cheng Lian <li...@gmail.com>.
Hi Roc,

Spark SQL 1.2.0 can only work with Hive 0.12.0 or Hive 0.13.1 
(controlled by compilation flags), versions prior 1.2.0 only works with 
Hive 0.12.0. So Hive 0.15.0-SNAPSHOT is not an option.

Would like to add that this is due to backwards compatibility issue of 
Hive metastore, AFAIK there's no popular system works with multiple 
versions of Hive metastore simultaneously.

Cheng

On 12/24/14 6:31 PM, Roc Chu wrote:
> this is my problem. I use mysql to store hive meta data. and i can get 
> what i want when I exec show tables in hive shell. but in the same 
> machine. I use spark-sql to execute same command (show tables), I got 
> errors.
> I look at the log of hive metastore find this errors
>
> 2014-12-24 05:04:59,874 ERROR [pool-3-thread-2]: 
> server.TThreadPoolServer (TThreadPoolServer.java:run(294)) - Thrift 
> error occurred during processing of message.
> org.apache.thrift.protocol.TProtocolException: Expected protocol id 
> ffffff82 but got ffffff80
>         at 
> org.apache.thrift.protocol.TCompactProtocol.readMessageBegin(TCompactProtocol.java:503)
>         at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:75)
>         at 
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> 2014-12-24 05:05:01,883 ERROR [pool-3-thread-3]: 
> server.TThreadPoolServer (TThreadPoolServer.java:run(294)) - Thrift 
> error occurred during processing of message.
> org.apache.thrift.protocol.TProtocolException: Expected protocol id 
> ffffff82 but got ffffff80
>         at 
> org.apache.thrift.protocol.TCompactProtocol.readMessageBegin(TCompactProtocol.java:503)
>         at 
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:75)
>         at 
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
>
> I compiled e spark from soucecode of spark-1.1.1
> and my hive was compiled from apache-hive-0.15.0
>
> I put the same hive-site.xml both hive conf and apark conf.
>
> waiting for help
> thanks
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org