You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Matthew Brandyberry (JIRA)" <ji...@apache.org> on 2015/06/26 00:32:05 UTC
[jira] [Comment Edited] (SPARK-6355) Spark standalone cluster does
not support local:/ url for jar file
[ https://issues.apache.org/jira/browse/SPARK-6355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602072#comment-14602072 ]
Matthew Brandyberry edited comment on SPARK-6355 at 6/25/15 10:31 PM:
----------------------------------------------------------------------
This appears to fix fixed in 1.4.0:
Using simply <filepath>
{noformat}
15/06/25 17:16:49 INFO Executor: Fetching http://9.41.180.152:41860/jars/simple-project-1.0-SNAPSHOT.jar with timestamp 1435270607309
15/06/25 17:16:50 INFO Utils: Fetching http://9.41.180.152:41860/jars/simple-project-1.0-SNAPSHOT.jar to /tmp/spark-76ae9294-c1e7-4cd5-b242-28ca93684f9e/executor-fdda27da-f4a8-447f-9bfc-8a26e33ded58/fetchFileTemp6323092224818161420.tmp
15/06/25 17:16:50 INFO Utils: Copying /tmp/spark-76ae9294-c1e7-4cd5-b242-28ca93684f9e/executor-fdda27da-f4a8-447f-9bfc-8a26e33ded58/8335963441435270607309_cache to /home/mbrandy/spark/work/app-20150625171647-0003/0/./simple-project-1.0-SNAPSHOT.jar
15/06/25 17:16:50 INFO Executor: Adding file:/home/mbrandy/spark/work/app-20150625171647-0003/0/./simple-project-1.0-SNAPSHOT.jar to class loader
{noformat}
Using local:<filepath>
{noformat}
15/06/25 17:17:10 INFO Executor: Fetching file:/home/mbrandy/simple-project-1.0-SNAPSHOT.jar with timestamp 1435270627993
15/06/25 17:17:10 INFO Utils: Copying /home/mbrandy/simple-project-1.0-SNAPSHOT.jar to /tmp/spark-76ae9294-c1e7-4cd5-b242-28ca93684f9e/executor-17d4c6f3-7c64-4fd8-8c3e-6530df306d3b/14876367751435270627993_cache
15/06/25 17:17:10 INFO Utils: Copying /tmp/spark-76ae9294-c1e7-4cd5-b242-28ca93684f9e/executor-17d4c6f3-7c64-4fd8-8c3e-6530df306d3b/14876367751435270627993_cache to /home/mbrandy/spark/work/app-20150625171708-0004/0/./simple-project-1.0-SNAPSHOT.jar
15/06/25 17:17:10 INFO Executor: Adding file:/home/mbrandy/spark/work/app-20150625171708-0004/0/./simple-project-1.0-SNAPSHOT.jar to class loader
{noformat}
was (Author: mtbrandy):
This appears to fix fixed in 1.4.0:
Using simply <filepath>
{noformat}
15/06/25 17:16:49 INFO Executor: Fetching http://9.41.180.152:41860/jars/simple-project-1.0-SNAPSHOT.jar with timestamp 1435270607309
15/06/25 17:16:50 INFO Utils: Fetching http://9.41.180.152:41860/jars/simple-project-1.0-SNAPSHOT.jar to /tmp/spark-76ae9294-c1e7-4cd5-b242-28ca93684f9e/executor-fdda27da-f4a8-447f-9bfc-8a26e33ded58/fetchFileTemp6323092224818161420.tmp
15/06/25 17:16:50 INFO Utils: Copying /tmp/spark-76ae9294-c1e7-4cd5-b242-28ca93684f9e/executor-fdda27da-f4a8-447f-9bfc-8a26e33ded58/8335963441435270607309_cache to /home/mbrandy/spark/work/app-20150625171647-0003/0/./simple-project-1.0-SNAPSHOT.jar
15/06/25 17:16:50 INFO Executor: Adding file:/home/mbrandy/spark/work/app-20150625171647-0003/0/./simple-project-1.0-SNAPSHOT.jar to class loader
{noformat}
Using local:<filepath>
15/06/25 17:17:10 INFO Executor: Fetching file:/home/mbrandy/simple-project-1.0-SNAPSHOT.jar with timestamp 1435270627993
15/06/25 17:17:10 INFO Utils: Copying /home/mbrandy/simple-project-1.0-SNAPSHOT.jar to /tmp/spark-76ae9294-c1e7-4cd5-b242-28ca93684f9e/executor-17d4c6f3-7c64-4fd8-8c3e-6530df306d3b/14876367751435270627993_cache
15/06/25 17:17:10 INFO Utils: Copying /tmp/spark-76ae9294-c1e7-4cd5-b242-28ca93684f9e/executor-17d4c6f3-7c64-4fd8-8c3e-6530df306d3b/14876367751435270627993_cache to /home/mbrandy/spark/work/app-20150625171708-0004/0/./simple-project-1.0-SNAPSHOT.jar
15/06/25 17:17:10 INFO Executor: Adding file:/home/mbrandy/spark/work/app-20150625171708-0004/0/./simple-project-1.0-SNAPSHOT.jar to class loader
> Spark standalone cluster does not support local:/ url for jar file
> ------------------------------------------------------------------
>
> Key: SPARK-6355
> URL: https://issues.apache.org/jira/browse/SPARK-6355
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.2.1, 1.3.0
> Reporter: Jesper Lundgren
> Priority: Minor
>
> Submitting a new spark application to a standalone cluster with local:/path will result in an exception.
> Driver successfully submitted as driver-20150316171157-0004
> ... waiting before polling master for driver state
> ... polling master for driver state
> State of driver-20150316171157-0004 is ERROR
> Exception from cluster was: java.io.IOException: No FileSystem for scheme: local
> java.io.IOException: No FileSystem for scheme: local
> at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584)
> at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
> at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
> at org.apache.spark.deploy.worker.DriverRunner.org$apache$spark$deploy$worker$DriverRunner$$downloadUserJar(DriverRunner.scala:141)
> at org.apache.spark.deploy.worker.DriverRunner$$anon$1.run(DriverRunner.scala:75)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org