spark-submit fails to get jar from http source

[lbustelo <gi...@bustelos.com>]

I'm running a 1.0.0 standalone cluster based on amplab/dockerscripts with 3
workers. I'm testing out spark-submit and I'm getting errors using
*--deploy-mode cluster* and using an http:// url to my JAR. I'm getting the
following error back.

Sending launch command to spark://master:7077
Driver successfully submitted as driver-20140613191831-0009
... waiting before polling master for driver state
... polling master for driver state
State of driver-20140613191831-0009 is ERROR
Exception from cluster was: java.io.IOException: No FileSystem for scheme:
http

I verified that my jar URL is accessible from within the spark nodes
(workers and master). I also ran with the same URL and *--deploy-mode
client*, and things worked. 

Documentation at
http://spark.apache.org/docs/latest/submitting-applications.html in section
/Advanced Dependency Management/ suggest that this should work.

Is this a known issue, or are my expectations wrong?

PS - I find it very limiting that spark-submit would not take care of
uploading my jar to the cluster. This is a fundamental requirement that most
frameworks support (i.e Storm, Hadoop, etc.). I do not consider this to be a
requirement specific to the JobServer work, rather part of the master's api.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/spark-submit-fails-to-get-jar-from-http-source-tp7592.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.