[jira] [Created] (ZEPPELIN-120) Tutorial fails on Mesos master

["Yang Lei (JIRA)" <ji...@apache.org>]

Yang Lei created ZEPPELIN-120:
---------------------------------

             Summary: Tutorial fails on Mesos master
                 Key: ZEPPELIN-120
                 URL: https://issues.apache.org/jira/browse/ZEPPELIN-120
             Project: Zeppelin
          Issue Type: Bug
          Components: Interpreters
            Reporter: Yang Lei


I was able to run tutorial on spark local mode (w/o MASTER environment variable). 

However when running on Mesos , the %sql script failed with "java.lang.reflect.InvocationTargetException",  detailed exception as follows:

Exception in thread "Thread-83" org.apache.spark.SparkException: Executor Spark home `spark.mesos.executor.home` is not set!
	at org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend$$anonfun$2.apply(MesosSchedulerBackend.scala:74)
	at org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend$$anonfun$2.apply(MesosSchedulerBackend.scala:74)
	at scala.Option.getOrElse(Option.scala:120)
	at org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend.createExecutorInfo(MesosSchedulerBackend.scala:73)
	at org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend.createMesosTask(MesosSchedulerBackend.scala:270)
	at org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend$$anonfun$resourceOffers$1$$anonfun$apply$mcV$sp$2$$anonfun$apply$1.apply(MesosSchedulerBackend.scala:232)
	at org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend$$anonfun$resourceOffers$1$$anonfun$apply$mcV$sp$2$$anonfun$apply$1.apply(MesosSchedulerBackend.scala:226)
	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
	at org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend$$anonfun$resourceOffers$1$$anonfun$apply$mcV$sp$2.apply(MesosSchedulerBackend.scala:226)
	at org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend$$anonfun$resourceOffers$1$$anonfun$apply$mcV$sp$2.apply(MesosSchedulerBackend.scala:225)
	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
	at org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend$$anonfun$resourceOffers$1.apply$mcV$sp(MesosSchedulerBackend.scala:225)
	at org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend.inClassLoader(MesosSchedulerBackend.scala:172)
	at org.apache.spark.scheduler.cluster.mesos.MesosSchedulerBackend.resourceOffers(MesosSchedulerBackend.scala:188)

I defined the following in conf/zeppelin-env.sh

export MESOS_NATIVE_JAVA_LIBRARY=/usr/local/lib/libmesos.so
export ZEPPELIN_JAVA_OPTS="-Dspark.executor.uri=http://www.apache.org/dyn/closer.cgi/spark/spark-1.4.0/spark-1.4.0-bin-hadoop2.4.tgz"
export SPARK_EXECUTOR_URI=http://www.apache.org/dyn/closer.cgi/spark/spark-1.4.0/spark-1.4.0-bin-hadoop2.4.tgz

I built the zeppelin with :

mvn clean package -Pspark-1.4 -Dhadoop.version=2.2.0 -Phadoop-2.2 -DskipTests

Before the failure,I can see zeppelin framework created on Mesos.I can also see from the zeppelin application UI (:4040) that the configurations of spark.executor.ui and spark.master are correct.

After the failure, the framework is terminated.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)