Problems launching 1.2.0-SNAPSHOT cluster with Hive support on EC2

[curtkohler <c....@elsevier.com>]

I've developed a Spark application using the 1.2.0-SNAPSHOP branch that
leverages Spark Streaming and Hive and can run it locally with no problem (I
need some fixes in the 1.2.0 branch). I successfully launched my EC2 cluster
by specifying a git commit hash from the 1.2.0-SNAPSHOT branch as the
version (as documented in the scripts)  and was able to run non-Hive Spark
jobs against it.  However, when I try and deploy my Hive application to it,
I am getting an exception (java.lang.ArrayStoreException:
org.apache.spark.sql.execution.SparkStrategies$CommandStrategy?) creating my
HiveContext.  I suspect that after the spark-ec2 launch script causes the
source code clone of the git repo/commit point, the assembly isn't being
built with Hive support (e.g. a -Phive profile on the mvn build command as
described in building Spark with Maven in the documentation).


Is anyone aware of a way I can get Hive support added into my compiled
assembly on the master/slaves using the spark_ec scripts (or where it would
make sense to hack the existing scripts to get it working).


Thanks,

Curt



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Problems-launching-1-2-0-SNAPSHOT-cluster-with-Hive-support-on-EC2-tp19221.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org