You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Sean Owen (JIRA)" <ji...@apache.org> on 2014/11/22 23:07:12 UTC
[jira] [Commented] (SPARK-4556) binary distribution assembly can't
run in local mode
[ https://issues.apache.org/jira/browse/SPARK-4556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14222206#comment-14222206 ]
Sean Owen commented on SPARK-4556:
----------------------------------
Hm, but is that a bug? I think compute-classpath.sh is designed to support running from the project root in development, or running from the files as laid out in the release, at least judging from your comments and the script itself. I don't think the raw contents of the assembly JAR themselves are a runnable installation.
> binary distribution assembly can't run in local mode
> ----------------------------------------------------
>
> Key: SPARK-4556
> URL: https://issues.apache.org/jira/browse/SPARK-4556
> Project: Spark
> Issue Type: Bug
> Components: Build, Spark Shell
> Reporter: Sean Busbey
>
> After building the binary distribution assembly, the resultant tarball can't be used for local mode.
> {code}
> busbey2-MBA:spark busbey$ mvn -Pbigtop-dist -DskipTests=true package
> [INFO] Scanning for projects...
> ...SNIP...
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Spark Project Parent POM ........................... SUCCESS [ 32.227 s]
> [INFO] Spark Project Networking ........................... SUCCESS [ 31.402 s]
> [INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 8.864 s]
> [INFO] Spark Project Core ................................. SUCCESS [15:39 min]
> [INFO] Spark Project Bagel ................................ SUCCESS [ 29.470 s]
> [INFO] Spark Project GraphX ............................... SUCCESS [05:20 min]
> [INFO] Spark Project Streaming ............................ SUCCESS [11:02 min]
> [INFO] Spark Project Catalyst ............................. SUCCESS [11:26 min]
> [INFO] Spark Project SQL .................................. SUCCESS [11:33 min]
> [INFO] Spark Project ML Library ........................... SUCCESS [14:27 min]
> [INFO] Spark Project Tools ................................ SUCCESS [ 40.980 s]
> [INFO] Spark Project Hive ................................. SUCCESS [11:45 min]
> [INFO] Spark Project REPL ................................. SUCCESS [03:15 min]
> [INFO] Spark Project Assembly ............................. SUCCESS [04:22 min]
> [INFO] Spark Project External Twitter ..................... SUCCESS [ 43.567 s]
> [INFO] Spark Project External Flume Sink .................. SUCCESS [ 50.367 s]
> [INFO] Spark Project External Flume ....................... SUCCESS [01:41 min]
> [INFO] Spark Project External MQTT ........................ SUCCESS [ 40.973 s]
> [INFO] Spark Project External ZeroMQ ...................... SUCCESS [ 54.878 s]
> [INFO] Spark Project External Kafka ....................... SUCCESS [01:23 min]
> [INFO] Spark Project Examples ............................. SUCCESS [10:19 min]
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 01:47 h
> [INFO] Finished at: 2014-11-22T02:13:51-06:00
> [INFO] Final Memory: 79M/2759M
> [INFO] ------------------------------------------------------------------------
> busbey2-MBA:spark busbey$ cd assembly/target/
> busbey2-MBA:target busbey$ mkdir dist-temp
> busbey2-MBA:target busbey$ tar -C dist-temp -xzf spark-assembly_2.10-1.3.0-SNAPSHOT-dist.tar.gz
> busbey2-MBA:target busbey$ cd dist-temp/
> busbey2-MBA:dist-temp busbey$ ./bin/spark-shell
> ls: /Users/busbey/projects/spark/assembly/target/dist-temp/assembly/target/scala-2.10: No such file or directory
> Failed to find Spark assembly in /Users/busbey/projects/spark/assembly/target/dist-temp/assembly/target/scala-2.10
> You need to build Spark before running this program.
> {code}
> It looks like the classpath calculations in {{bin/compute_classpath.sh}} don't handle it.
> If I move all of the spark-*.jar files from the top level into the lib folder and touch the RELEASE file, then the spark shell launches in local mode normally.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org