[jira] [Assigned] (SPARK-36835) Spark 3.2.0 POMs are no longer "dependency reduced"

["Apache Spark (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-36835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Apache Spark reassigned SPARK-36835:
------------------------------------

    Assignee: Apache Spark

> Spark 3.2.0 POMs are no longer "dependency reduced"
> ---------------------------------------------------
>
>                 Key: SPARK-36835
>                 URL: https://issues.apache.org/jira/browse/SPARK-36835
>             Project: Spark
>          Issue Type: Bug
>          Components: Build
>    Affects Versions: 3.2.0
>            Reporter: Josh Rosen
>            Assignee: Apache Spark
>            Priority: Blocker
>
> It looks like Spark 3.2.0's POMs are no longer "dependency reduced". As a result, applications may pull in additional unnecessary dependencies when depending on Spark.
> Spark uses the Maven Shade plugin to create effective POMs and to bundle shaded versions of certain libraries with Spark (namely, Jetty, Guava, and JPPML). [By default|https://maven.apache.org/plugins/maven-shade-plugin/shade-mojo.html#createDependencyReducedPom], the Maven Shade plugin generates simplified POMs which remove dependencies on artifacts that have been shaded.
> SPARK-33212 / [b6f46ca29742029efea2790af7fdefbc2fcf52de|https://github.com/apache/spark/commit/b6f46ca29742029efea2790af7fdefbc2fcf52de] changed the configuration of the Maven Shade plugin, setting {{createDependencyReducedPom}} to {{false}}.
> As a result, the generated POMs now include compile-scope dependencies on the shaded libraries. For example, compare the {{org.eclipse.jetty}} dependencies in:
>  * Spark 3.1.2: [https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.12/3.1.2/spark-core_2.12-3.1.2.pom]
>  * Spark 3.2.0 RC2: [https://repository.apache.org/content/repositories/orgapachespark-1390/org/apache/spark/spark-core_2.12/3.2.0/spark-core_2.12-3.2.0.pom]
> I think we should revert back to generating "dependency reduced" POMs to ensure that Spark declares a proper set of dependencies and to avoid "unknown unknown" consequences of changing our generated POM format.
> /cc [~csun]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org