You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "HyukjinKwon (via GitHub)" <gi...@apache.org> on 2023/10/13 08:14:24 UTC
[PR] [SPARK-45535][BUILD] Do not compile tests for snapshots and release build [spark]
HyukjinKwon opened a new pull request, #43365:
URL: https://github.com/apache/spark/pull/43365
### What changes were proposed in this pull request?
- `-DskipTests` compiles the tests, but skips running them
- `-Dmaven.test.skip=true` skips compiling the tests and does not run them
This PR switches from the former to the latter when we build the release because we do not need test classes.
Other places, we still need to better compile test classes because several tests in PySpark or SparkR need them, e.g., `org.apache.spark.sql.TestQueryExecutionListener` at `pyspark/sql/tests/test_dataframe.py` .
### Why are the changes needed?
- To remove unnecessary overhead for compiling tests.
- This can potentially fix the snapshot build being failed: https://github.com/apache/spark/actions/runs/6502277099
### Does this PR introduce _any_ user-facing change?
No, dev-only.
### How was this patch tested?
Manually tested.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45535][BUILD] Do not compile tests for snapshots and release build [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #43365:
URL: https://github.com/apache/spark/pull/43365#issuecomment-1761243584
Yup let's forget about this for now anyway, and see if your fix works
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45535][BUILD] Do not compile tests for snapshots and release build [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on PR #43365:
URL: https://github.com/apache/spark/pull/43365#issuecomment-1761131453
Ah, okie. We should really exclude them but seems like that's already included in the canonical releases too .. okay, let's don't do this for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45535][BUILD] Do not compile tests for snapshots and release build [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #43365:
URL: https://github.com/apache/spark/pull/43365#issuecomment-1761118085
IIRC, `-Dmaven.test.skip=true` will make the content of the `spark-xx-test.jar` empty when packaging and deploying.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45535][BUILD] Do not compile tests for snapshots and release build [spark]
Posted by "LuciferYang (via GitHub)" <gi...@apache.org>.
LuciferYang commented on PR #43365:
URL: https://github.com/apache/spark/pull/43365#issuecomment-1761180858
@HyukjinKwon Sorry, I manually checked, it seems like I remembered wrong. `-Dmaven.test.skip=true` will not package `spark-xxx-test.jar`, but here it's necessary to add `-Dmaven.test.skip=true` on top of `-DskipTests`, otherwise it will execute the scala tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [SPARK-45535][BUILD] Do not compile tests for snapshots and release build [spark]
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon closed pull request #43365: [SPARK-45535][BUILD] Do not compile tests for snapshots and release build
URL: https://github.com/apache/spark/pull/43365
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org