You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by "Manikandan Narayanaswamy (JIRA)" <ji...@apache.org> on 2014/08/22 23:56:13 UTC
[jira] [Updated] (BIGTOP-1415) Remove redundant jars from spark
package.
[ https://issues.apache.org/jira/browse/BIGTOP-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manikandan Narayanaswamy updated BIGTOP-1415:
---------------------------------------------
Description:
But looking at the current parcel version of Spark, it packages a lot of things that are not necessary, and is missing some others that might be useful.
bagel/lib/spark-bagel_2.10-1.0.0-cdh5.1.0-SNAPSHOT.jar
core/lib/spark-core_2.10-1.0.0-cdh5.1.0-SNAPSHOT.jar
lib/*
mllib/lib/spark-mllib_2.10-1.0.0-cdh5.1.0-SNAPSHOT.jar
repl/lib/spark-repl_2.10-1.0.0-cdh5.1.0-SNAPSHOT.jar
streaming/lib/spark-streaming_2.10-1.0.0-cdh5.1.0-SNAPSHOT.jar
None of the above are necessary, since they are already packaged in assembly/lib/spark-assembly-1.0.0-cdh5.1.0-SNAPSHOT-hadoop2.3.0-cdh5.1.0-SNAPSHOT.jar, which is what everybody should be using.
examples/lib/spark-examples_2.10-1.0.0-cdh5.1.0-SNAPSHOT-javadoc.jar
examples/lib/spark-examples_2.10-1.0.0-cdh5.1.0-SNAPSHOT-sources.jar
Again not necessary, just because, well, anyone who needs that can use maven.
> Remove redundant jars from spark package.
> -----------------------------------------
>
> Key: BIGTOP-1415
> URL: https://issues.apache.org/jira/browse/BIGTOP-1415
> Project: Bigtop
> Issue Type: Bug
> Components: spark
> Affects Versions: 0.7.0
> Reporter: Manikandan Narayanaswamy
>
> But looking at the current parcel version of Spark, it packages a lot of things that are not necessary, and is missing some others that might be useful.
> bagel/lib/spark-bagel_2.10-1.0.0-cdh5.1.0-SNAPSHOT.jar
> core/lib/spark-core_2.10-1.0.0-cdh5.1.0-SNAPSHOT.jar
> lib/*
> mllib/lib/spark-mllib_2.10-1.0.0-cdh5.1.0-SNAPSHOT.jar
> repl/lib/spark-repl_2.10-1.0.0-cdh5.1.0-SNAPSHOT.jar
> streaming/lib/spark-streaming_2.10-1.0.0-cdh5.1.0-SNAPSHOT.jar
> None of the above are necessary, since they are already packaged in assembly/lib/spark-assembly-1.0.0-cdh5.1.0-SNAPSHOT-hadoop2.3.0-cdh5.1.0-SNAPSHOT.jar, which is what everybody should be using.
> examples/lib/spark-examples_2.10-1.0.0-cdh5.1.0-SNAPSHOT-javadoc.jar
> examples/lib/spark-examples_2.10-1.0.0-cdh5.1.0-SNAPSHOT-sources.jar
> Again not necessary, just because, well, anyone who needs that can use maven.
--
This message was sent by Atlassian JIRA
(v6.2#6252)