You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/10/14 03:31:16 UTC
[GitHub] [spark] LuciferYang opened a new pull request, #38247: [SPARK-40795][BUILD] Exclude redundant jars from spark-protobuf-assembly jar
LuciferYang opened a new pull request, #38247:
URL: https://github.com/apache/spark/pull/38247
### What changes were proposed in this pull request?
This pr add `assemblyPackageScala` and `assemblyExcludedJars` to `SparkProtobuf` module sbt build options to exclude redundant jars from spark-protobuf-assembly jar.
### Why are the changes needed?
Exclude redundant jars from `spark-protobuf-assembly` jar
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- Pass GitHub Actions
- Manual test
run `build/sbt "protobuf/assembly"`
**Before**
```
[debug] Including from cache: spark-tags_2.12-3.4.0-SNAPSHOT.jar
[debug] Including from cache: unused-1.0.0.jar
[debug] Including from cache: spark-protobuf_2.12-3.4.0-SNAPSHOT.jar
[debug] Including from cache: scala-collection-compat_2.12-2.2.0.jar
[debug] Including from cache: pmml-model-1.4.8.jar
[debug] Including from cache: protobuf-java-3.21.1.jar
[debug] Including from cache: guava-14.0.1.jar
[debug] Including from cache: scala-library-2.12.17.jar
```
**After**
```
[debug] Including: spark-protobuf_2.12-3.4.0-SNAPSHOT.jar
[debug] Including from cache: protobuf-java-3.21.1.jar
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] pan3793 commented on pull request #38247: [SPARK-40795][BUILD] Exclude redundant jars from spark-protobuf-assembly jar
Posted by GitBox <gi...@apache.org>.
pan3793 commented on PR #38247:
URL: https://github.com/apache/spark/pull/38247#issuecomment-1278972733
Some issues exist on branch-3.3 as well, as Spark support use sbt to package docker image(for dev/testing), could we consider backport(maybe partially) to branch-3.3 as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] pan3793 commented on pull request #38247: [SPARK-40795][BUILD] Exclude redundant jars from spark-protobuf-assembly jar
Posted by GitBox <gi...@apache.org>.
pan3793 commented on PR #38247:
URL: https://github.com/apache/spark/pull/38247#issuecomment-1278974915
Sorry, I may miss read it, it seems only for the new added module.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #38247: [SPARK-40795][BUILD] Exclude redundant jars from spark-protobuf-assembly jar
Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #38247: [SPARK-40795][BUILD] Exclude redundant jars from spark-protobuf-assembly jar
URL: https://github.com/apache/spark/pull/38247
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #38247: [SPARK-40795][BUILD] Exclude redundant jars from spark-protobuf-assembly jar
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #38247:
URL: https://github.com/apache/spark/pull/38247#issuecomment-1278748001
thanks @HyukjinKwon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #38247: [SPARK-40795][BUILD] Exclude redundant jars from spark-protobuf-assembly jar
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #38247:
URL: https://github.com/apache/spark/pull/38247#issuecomment-1279034256
> Sorry, I may miss read here, it seems only for the new added module.
Yes, for other modules, sbt packaging has not kept consistent with maven's behavior
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org