You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/06/18 12:09:20 UTC
[GitHub] [spark] wankunde opened a new pull request #28857: Generate spark streaming test jar with maven plugin maven-jar-plugin
wankunde opened a new pull request #28857:
URL: https://github.com/apache/spark/pull/28857
### What changes were proposed in this pull request?
Generate spark streaming test jar with maven plugin maven-jar-plugin
### Why are the changes needed?
When I compile streaming module alone, the test jar generated incorrect.
So we need to import maven plugin `maven-jar-plugin`
Maven Command : `mvn clean package -DskipTests -Phive -Phive-thriftserver -Pyarn -DskipTests -pl streaming`
```
[kun.wan@nta-offline-gw1 streaming]$ ll target/ -l
total 2668
drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:46 analysis
drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:46 antrun
drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:46 maven-archiver
drwxr-xr-x 3 kun.wan staff 4096 Jun 18 19:45 maven-shared-archive-resources
-rw-r--r-- 1 kun.wan staff 1134932 Jun 18 19:46 original-spark-streaming_2.12-3.0.0.jar
drwxr-xr-x 4 kun.wan staff 4096 Jun 18 19:46 scala-2.12
-rw-r--r-- 1 kun.wan staff 1137992 Jun 18 19:46 spark-streaming_2.12-3.0.0.jar
-rw-r--r-- 1 kun.wan staff 252535 Jun 18 19:46 spark-streaming_2.12-3.0.0-sources.jar
-rw-r--r-- 1 kun.wan staff 22 Jun 18 19:46 spark-streaming_2.12-3.0.0-tests.jar
-rw-r--r-- 1 kun.wan staff 170004 Jun 18 19:46 spark-streaming_2.12-3.0.0-test-sources.jar
drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:46 tmp
[kun.wan@nta-offline-gw1 streaming]$ ll target/ -l
total 3500
drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:52 analysis
drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:51 antrun
drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:52 maven-archiver
drwxr-xr-x 3 kun.wan staff 4096 Jun 18 19:51 maven-shared-archive-resources
-rw-r--r-- 1 kun.wan staff 1134959 Jun 18 19:52 original-spark-streaming_2.12-3.0.0.jar
drwxr-xr-x 4 kun.wan staff 4096 Jun 18 19:51 scala-2.12
-rw-r--r-- 1 kun.wan staff 1138019 Jun 18 19:52 spark-streaming_2.12-3.0.0.jar
-rw-r--r-- 1 kun.wan staff 252558 Jun 18 19:52 spark-streaming_2.12-3.0.0-sources.jar
-rw-r--r-- 1 kun.wan staff 853239 Jun 18 19:52 spark-streaming_2.12-3.0.0-tests.jar
-rw-r--r-- 1 kun.wan staff 170027 Jun 18 19:52 spark-streaming_2.12-3.0.0-test-sources.jar
drwxr-xr-x 2 kun.wan staff 4096 Jun 18 19:51 tmp
```
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Jenkins Tested.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-646074812
What do you mean it's incorrect?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-647196445
Gentle ping, @wankunde .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-646706942
**[Test build #124291 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124291/testReport)** for PR 28857 at commit [`cbeaeb1`](https://github.com/apache/spark/commit/cbeaeb19112c028345b49f4870ccaca3744dc52c).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-645976374
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-646703995
Jenkins test this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-646706942
**[Test build #124291 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124291/testReport)** for PR 28857 at commit [`cbeaeb1`](https://github.com/apache/spark/commit/cbeaeb19112c028345b49f4870ccaca3744dc52c).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #28857:
URL: https://github.com/apache/spark/pull/28857
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28857: [Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-645974948
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28857: Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-645974948
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28857: [Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-645976374
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-646427158
Hm, I'm a little uneasy about it, as that shade config was put in a long time ago to work around Guava issues. It may or may not be necessary, but, what if you leave it in?
Also, do you actually need test jars?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-650473151
Oh, thank you for confirmation, @wankunde . Then, I close this PR and SPARK-32023.
In any way, thank you for your contribution on this issue. Please don't hesitate to report a weird situation like this. We can solve together! Thanks again.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wankunde commented on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
wankunde commented on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-650014523
Hi, @dongjoon-hyun , I'm sorry , the result of maven compile seems OK after I change a computer.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-646707416
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-646811126
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-646242041
It seems that he meant `22 bytes`, the size of tests jar.
```
-rw-r--r-- 1 kun.wan staff 22 Jun 18 19:46 spark-streaming_2.12-3.0.0-tests.jar
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wankunde commented on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
wankunde commented on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-646414088
Hi @srowen, I have compared the compilation results with and without shade plugin. Their result classes are the same. So I don't think we need to keep shade plugin anymore.
```
➜ tmp ls -al shade
total 11864
drwxr-xr-x 21 wankun staff 672 6 19 11:11 .
drwxr-xr-x 9 wankun staff 288 6 19 11:15 ..
-rw-r--r-- 1 wankun staff 30 6 19 11:11 .plxarc
drwxr-xr-x 4 wankun staff 128 6 19 11:11 analysis
drwxr-xr-x 3 wankun staff 96 6 19 11:11 antrun
-rw-r--r-- 1 wankun staff 2103 6 19 11:11 checkstyle-cachefile
-rw-r--r-- 1 wankun staff 8225 6 19 11:11 checkstyle-checker.xml
-rw-r--r-- 1 wankun staff 2133 6 19 11:11 checkstyle-output.xml
drwxr-xr-x 3 wankun staff 96 6 19 11:11 maven-archiver
drwxr-xr-x 3 wankun staff 96 6 19 11:11 maven-shared-archive-resources
-rw-r--r-- 1 wankun staff 849617 6 19 11:11 original-spark-streaming_2.12-3.0.0-tests.jar
-rw-r--r-- 1 wankun staff 1129042 6 19 11:11 original-spark-streaming_2.12-3.0.0.jar
drwxr-xr-x 4 wankun staff 128 6 19 11:11 scala-2.12
-rw-r--r-- 1 wankun staff 91 6 19 11:11 scalastyle-output.xml
drwxr-xr-x 3 wankun staff 96 6 19 11:11 site
-rw-r--r-- 1 wankun staff 1640857 6 19 11:11 spark-streaming_2.12-3.0.0-javadoc.jar
-rw-r--r-- 1 wankun staff 252544 6 19 11:11 spark-streaming_2.12-3.0.0-sources.jar
-rw-r--r-- 1 wankun staff 170013 6 19 11:11 spark-streaming_2.12-3.0.0-test-sources.jar
-rw-r--r-- 1 wankun staff 854204 6 19 11:11 spark-streaming_2.12-3.0.0-tests.jar
-rw-r--r-- 1 wankun staff 1137942 6 19 11:11 spark-streaming_2.12-3.0.0.jar
drwxr-xr-x 2 wankun staff 64 6 19 11:11 tmp
➜ tmp ls -al without_shade
total 10192
drwxr-xr-x 20 wankun staff 640 6 19 11:13 .
drwxr-xr-x 9 wankun staff 288 6 19 11:15 ..
-rw-r--r-- 1 wankun staff 30 6 19 11:13 .plxarc
drwxr-xr-x 4 wankun staff 128 6 19 11:13 analysis
drwxr-xr-x 3 wankun staff 96 6 19 11:13 antrun
-rw-r--r-- 1 wankun staff 2103 6 19 11:13 checkstyle-cachefile
-rw-r--r-- 1 wankun staff 8225 6 19 11:13 checkstyle-checker.xml
-rw-r--r-- 1 wankun staff 2133 6 19 11:13 checkstyle-output.xml
drwxr-xr-x 3 wankun staff 96 6 19 11:13 maven-archiver
drwxr-xr-x 3 wankun staff 96 6 19 11:13 maven-shared-archive-resources
-rw-r--r-- 1 wankun staff 1129002 6 19 11:13 original-spark-streaming_2.12-3.0.0.jar
drwxr-xr-x 4 wankun staff 128 6 19 11:13 scala-2.12
-rw-r--r-- 1 wankun staff 91 6 19 11:13 scalastyle-output.xml
drwxr-xr-x 3 wankun staff 96 6 19 11:13 site
-rw-r--r-- 1 wankun staff 1640857 6 19 11:13 spark-streaming_2.12-3.0.0-javadoc.jar
-rw-r--r-- 1 wankun staff 252504 6 19 11:13 spark-streaming_2.12-3.0.0-sources.jar
-rw-r--r-- 1 wankun staff 169973 6 19 11:13 spark-streaming_2.12-3.0.0-test-sources.jar
-rw-r--r-- 1 wankun staff 849577 6 19 11:13 spark-streaming_2.12-3.0.0-tests.jar
-rw-r--r-- 1 wankun staff 1137902 6 19 11:13 spark-streaming_2.12-3.0.0.jar
drwxr-xr-x 2 wankun staff 64 6 19 11:13 tmp
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gaborgsomogyi commented on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
gaborgsomogyi commented on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-647599602
@wankunde Just curious what's the result w/ `sbt`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-646704216
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-646707416
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-648977499
Gentle ping again, @wankunde .
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-646316112
Oh I see. Do you have to remove the shade plugin config though?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] wankunde commented on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
wankunde commented on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-646469772
Update PR, and leave the shade plugin in.
The test jar is need if I want to compile some other spark modules(eg. mllib ) individually.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-646898179
Hi, @wankunde . I tried the command in the PR description in `master` branch, but got a different result. Where did you get the result?
```
$ ls -al streaming/target/*tests.jar
fish: No matches for wildcard 'streaming/target/*tests.jar'. See `help expand`.
ls -al streaming/target/*tests.jar
$ mvn clean package -DskipTests -Phive -Phive-thriftserver -Pyarn -DskipTests -pl streaming
$ ls -al streaming/target/*tests.jar
-rw-r--r-- 1 dongjoon staff 849937 Jun 19 16:35 streaming/target/original-spark-streaming_2.12-3.1.0-SNAPSHOT-tests.jar
-rw-r--r-- 1 dongjoon staff 854537 Jun 19 16:35 streaming/target/spark-streaming_2.12-3.1.0-SNAPSHOT-tests.jar
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-646704216
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-646898179
Hi, @wankunde . I tried the command in the PR description in `master` branch, but got a different result. Where did you get the result?
```
$ ls -al streaming/target/*tests.jar
fish: No matches for wildcard 'streaming/target/*tests.jar'. See `help expand`.
ls -al streaming/target/*tests.jar
$ mvn clean package -DskipTests -Phive -Phive-thriftserver -Pyarn -DskipTests -pl streaming
$ ls -al streaming/target/*tests.jar
-rw-r--r-- 1 dongjoon staff 849937 Jun 19 16:35 streaming/target/original-spark-streaming_2.12-3.1.0-SNAPSHOT-tests.jar
-rw-r--r-- 1 dongjoon staff 854537 Jun 19 16:35 streaming/target/spark-streaming_2.12-3.1.0-SNAPSHOT-tests.jar
```
It was the same for `branch-3.0`. `tests.jar` seems to be created correctly without this patch.
```
$ ls -al target/*tests.jar
-rw-r--r-- 1 dongjoon staff 849533 Jun 19 16:41 target/original-spark-streaming_2.12-3.0.1-SNAPSHOT-tests.jar
-rw-r--r-- 1 dongjoon staff 854120 Jun 19 16:41 target/spark-streaming_2.12-3.0.1-SNAPSHOT-tests.jar
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-646811126
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #28857: [SPARK-32023][Streaming]Generate spark streaming test jar with maven plugin maven-jar-plugin
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #28857:
URL: https://github.com/apache/spark/pull/28857#issuecomment-646810345
**[Test build #124291 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124291/testReport)** for PR 28857 at commit [`cbeaeb1`](https://github.com/apache/spark/commit/cbeaeb19112c028345b49f4870ccaca3744dc52c).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org