Re: [jira] [Created] (SPARK-1698) Improve spark integration

[Sean Owen <so...@cloudera.com>]

#1 and #2 are not relevant the issue of jar size. These can be problems in
general, but don't think there have been issues attributable to file
clashes. Shading has mechanisms to deal with this anyway.

#3 is a problem in general too, but is not specific to shading. Where
versions collide, build processes like Maven and shading must be used to
resolve them. But this happens regardless of whether you shade a fat jar.

#4 is a real problem specific to Java 6. It does seem like it will be
important to identify and remove more unnecessary dependencies to work
around it.

But shading per se is not the problem, and it is important to make a
packaged jar for the app. What are you proposing? Dependencies to be
removed?
On May 2, 2014 8:32 AM, "Guoqiang Li (JIRA)" <ji...@apache.org> wrote:

> Guoqiang Li created SPARK-1698:
> ----------------------------------
>
>              Summary: Improve spark integration
>                  Key: SPARK-1698
>                  URL: https://issues.apache.org/jira/browse/SPARK-1698
>              Project: Spark
>           Issue Type: Improvement
>           Components: Build, Deploy
>             Reporter: Guoqiang Li
>             Assignee: Guoqiang Li
>              Fix For: 1.0.0
>
>
> Use the shade plugin to create a big JAR with all the dependencies can
> cause a few problems
> 1. Missing jar's meta information
> 2. Some file is covered, eg: plugin.xml
> 3. Different versions of the jar may co-exist
> 4. Too big, java 6 does not support
>
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.2#6252)
>