You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (JIRA)" <ji...@apache.org> on 2018/12/08 20:20:00 UTC
[jira] [Updated] (SPARK-26299) Shaded test jar in spark-streaming
cause deploy test jar twice
[ https://issues.apache.org/jira/browse/SPARK-26299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun updated SPARK-26299:
----------------------------------
Summary: Shaded test jar in spark-streaming cause deploy test jar twice (was: [MAVEN] Shaded test jar in spark-streaming cause deploy test jar twice)
> Shaded test jar in spark-streaming cause deploy test jar twice
> --------------------------------------------------------------
>
> Key: SPARK-26299
> URL: https://issues.apache.org/jira/browse/SPARK-26299
> Project: Spark
> Issue Type: Bug
> Components: Deploy
> Affects Versions: 2.3.2, 2.4.0
> Reporter: Liu, Linhong
> Priority: Major
>
> In spark root pom, there is a test-jar execution for all sub-projects including spark-streaming. This will attach an artifact: org.apache.spark:spark-streaming_2.11:*{color:#ff0000}test-jar{color}*:tests:2.3.2
> Also, in streaming pom, there is a shade test configuration, it will attach an artifact: org.apache.spark:spark-streaming_2.11:{color:#ff0000}*jar*{color}:tests:2.3.2
> But two artifacts actually point to the same file: spark-streaming_2.11-2.3.2-tests.jar
> So, when deploy spark to nexus using mvn deploy, maven will upload the test.jar twice since it belongs to 2 artifacts. Then deploy fails due to nexus don't allow overrides existing file for non-SNAPSHOT release.
> What's more, after checking the spark-streaming, shaded test jar is exactly same to original test jar. It seems we can just delete the related shade config.
> {code:java}
> <build>
> <outputDirectory>target/scala-${scala.binary.version}/classes</outputDirectory>
> <testOutputDirectory>target/scala-${scala.binary.version}/test-classes</testOutputDirectory>
> <plugins>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-shade-plugin</artifactId>
> <configuration>
> <shadeTestJar>true</shadeTestJar>
> </configuration>
> </plugin>
> </plugins>
> </build>{code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org