You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2020/06/27 02:04:00 UTC
[jira] [Resolved] (SPARK-32023) Generate spark streaming test jar
with maven plugin maven-jar-plugin
[ https://issues.apache.org/jira/browse/SPARK-32023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun resolved SPARK-32023.
-----------------------------------
Resolution: Cannot Reproduce
Please see [https://github.com/apache/spark/pull/28857#issuecomment-650014523]
> Generate spark streaming test jar with maven plugin maven-jar-plugin
> --------------------------------------------------------------------
>
> Key: SPARK-32023
> URL: https://issues.apache.org/jira/browse/SPARK-32023
> Project: Spark
> Issue Type: Bug
> Components: Structured Streaming
> Affects Versions: 3.0.0
> Reporter: Wan Kun
> Priority: Minor
>
> When I compile streaming module alone, the test jar generated incorrect.
> So we need to import maven plugin `maven-jar-plugin`
> {code:java}
> mvn clean package -DskipTests -Phive -Phive-thriftserver -Pyarn -DskipTests -pl streaming
> [kun.wan@nta-offline-gw1 streaming]$ ll target/ -l
> total 2668
> drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:46 analysis
> drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:46 antrun
> drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:46 maven-archiver
> drwxr-xr-x 3 kun.wan staff 4096 Jun 18 19:45 maven-shared-archive-resources
> -rw-r--r-- 1 kun.wan staff 1134932 Jun 18 19:46 original-spark-streaming_2.12-3.0.0.jar
> drwxr-xr-x 4 kun.wan staff 4096 Jun 18 19:46 scala-2.12
> -rw-r--r-- 1 kun.wan staff 1137992 Jun 18 19:46 spark-streaming_2.12-3.0.0.jar
> -rw-r--r-- 1 kun.wan staff 252535 Jun 18 19:46 spark-streaming_2.12-3.0.0-sources.jar
> -rw-r--r-- 1 kun.wan staff 22 Jun 18 19:46 spark-streaming_2.12-3.0.0-tests.jar
> -rw-r--r-- 1 kun.wan staff 170004 Jun 18 19:46 spark-streaming_2.12-3.0.0-test-sources.jar
> drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:46 tmp
> # After change maven plugin
> [kun.wan@nta-offline-gw1 streaming]$ ll target/ -l
> total 3500
> drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:52 analysis
> drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:51 antrun
> drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:52 maven-archiver
> drwxr-xr-x 3 kun.wan staff 4096 Jun 18 19:51 maven-shared-archive-resources
> -rw-r--r-- 1 kun.wan staff 1134959 Jun 18 19:52 original-spark-streaming_2.12-3.0.0.jar
> drwxr-xr-x 4 kun.wan staff 4096 Jun 18 19:51 scala-2.12
> -rw-r--r-- 1 kun.wan staff 1138019 Jun 18 19:52 spark-streaming_2.12-3.0.0.jar
> -rw-r--r-- 1 kun.wan staff 252558 Jun 18 19:52 spark-streaming_2.12-3.0.0-sources.jar
> -rw-r--r-- 1 kun.wan staff 853239 Jun 18 19:52 spark-streaming_2.12-3.0.0-tests.jar
> -rw-r--r-- 1 kun.wan staff 170027 Jun 18 19:52 spark-streaming_2.12-3.0.0-test-sources.jar
> drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:51 tmp
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org