Contineous errors trying to start spark-shell

[Mohamed Lrhazi <Mo...@georgetown.edu>]

Hello,

I am trying to just start spark-shell... it starts, the prompt appears,
then a never ending (literally) stream of these log lines proceeds....
What is it trying to do? Why is it failing?

To start it I do:

$ docker run -it ncssm/spark-base /spark/bin/spark-shell --master spark://
devzero.cs.georgetown.edu:7077




The log lines look like:


5/07/04 00:36:36 INFO Worker: Asked to launch executor
app-20150704003631-0004/45 for Spark shell
15/07/04 00:36:36 INFO ExecutorRunner: Launch command:
"/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java" "-cp"
"/spark/sbin/../conf/:/spark/lib/spark-assembly-1.4.0-hadoop2.6.0.jar:/spark/lib/datanucleus-api-jdo-3.2.6.jar:/spark/lib/datanucleus-rdbms-3.2.9.jar:/spark/lib/datanucleus-core-3.2.10.jar"
"-Xms512M" "-Xmx512M" "-Dspark.driver.port=50266"
"org.apache.spark.executor.CoarseGrainedExecutorBackend" "--driver-url"
"akka.tcp://sparkDriver@172.17.0.45:50266/user/CoarseGrainedScheduler"
"--executor-id" "45" "--hostname" "10.212.55.41" "--cores" "16" "--app-id"
"app-20150704003631-0004" "--worker-url" "akka.tcp://
sparkWorker@10.212.55.41:7078/user/Worker"
15/07/04 00:36:39 INFO Worker: Executor app-20150704003631-0004/45 finished
with state EXITED message Command exited with code 1 exitStatus 1


On an example worker node, I see a corresponding unending stream of errors:

15/07/04 00:36:31 INFO Worker: Asked to launch executor
app-20150704003631-0004/7 for Spark shell
15/07/04 00:36:31 INFO ExecutorRunner: Launch command:
"/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java" "-cp"
"/spark/sbin/../conf/:/spark/lib/spark-assembly-1.4.0-hadoop2.6.0.jar:/spark/lib/datanucleus-api-jdo-3.2.6.jar:/spark/lib/datanucleus-rdbms-3.2.9.jar:/spark/lib/datanucleus-core-3.2.10.jar"
"-Xms512M" "-Xmx512M" "-Dspark.driver.port=50266"
"org.apache.spark.executor.CoarseGrainedExecutorBackend" "--driver-url"
"akka.tcp://sparkDriver@172.17.0.45:50266/user/CoarseGrainedScheduler"
"--executor-id" "7" "--hostname" "10.212.55.41" "--cores" "16" "--app-id"
"app-20150704003631-0004" "--worker-url" "akka.tcp://
sparkWorker@10.212.55.41:7078/user/Worker"
15/07/04 00:36:36 INFO Worker: Executor app-20150704003631-0004/7 finished
with state EXITED message Command exited with code 1 exitStatus 1


Thanks,
Mohamed.