You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Abhishek Tripathi <tr...@gmail.com> on 2015/05/28 19:22:00 UTC
Spark1.3.1 build issue with CDH5.4.0 getUnknownFields
Hi ,
I'm using CDH5.4.0 quick start VM and tried to build Spark with Hive
compatibility so that I can run Spark sql and access temp table remotely.
I used below command to build Spark, it was build successful but when I
tried to access Hive data from Spark sql, I get error.
Thanks,
Abhi
---------------------------------------------------
*mvn -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0-cdh5.4.0 -Phive
-Phive-thriftserver -DskipTests clean package*
[cloudera@quickstart spark-1.3.1]$export HADOOP_CONF_DIR=/etc/hive/conf/
[cloudera@quickstart spark-1.3.1]$ ./bin/spark-sql
SET spark.sql.hive.version=0.13.1
spark-sql> show tables;
sample_07 false
t1 false
Time taken: 3.901 seconds, Fetched 2 row(s)
spark-sql> select * from t1;
15/05/19 23:48:46 ERROR SparkSQLDriver: Failed in [select * from t1]
java.lang.VerifyError: class
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$SetOwnerRequestProto
overrides final method *getUnknownFields*
.()Lcom/google/protobuf/UnknownFieldSet;
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
Re: Spark1.3.1 build issue with CDH5.4.0 getUnknownFields
Posted by ayan guha <gu...@gmail.com>.
Probably a naive question: can you try the same in hive CLI and see if your
SQL is working? Looks like hive thing to me as spark is faithfully
delegating the query to hive.
On 29 May 2015 03:22, "Abhishek Tripathi" <tr...@gmail.com> wrote:
> Hi ,
> I'm using CDH5.4.0 quick start VM and tried to build Spark with Hive
> compatibility so that I can run Spark sql and access temp table remotely.
>
> I used below command to build Spark, it was build successful but when I
> tried to access Hive data from Spark sql, I get error.
>
> Thanks,
> Abhi
>
> ---------------------------------------------------
> *mvn -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0-cdh5.4.0 -Phive
> -Phive-thriftserver -DskipTests clean package*
>
> [cloudera@quickstart spark-1.3.1]$export HADOOP_CONF_DIR=/etc/hive/conf/
> [cloudera@quickstart spark-1.3.1]$ ./bin/spark-sql
> SET spark.sql.hive.version=0.13.1
> spark-sql> show tables;
> sample_07 false
> t1 false
> Time taken: 3.901 seconds, Fetched 2 row(s)
> spark-sql> select * from t1;
> 15/05/19 23:48:46 ERROR SparkSQLDriver: Failed in [select * from t1]
> java.lang.VerifyError: class
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$SetOwnerRequestProto
> overrides final method *getUnknownFields*
> .()Lcom/google/protobuf/UnknownFieldSet;
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>
>
>
>