You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/10/12 02:54:54 UTC
[GitHub] [spark] srowen opened a new pull request #34250: Try 5g, not 6g, for test mem limit
srowen opened a new pull request #34250:
URL: https://github.com/apache/spark/pull/34250
### What changes were proposed in this pull request?
Use 5g of memory for tests, not 6g.
### Why are the changes needed?
6g might be flaky, per https://github.com/apache/spark/pull/34214
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Existing tests on Java 11
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #34250: [SPARK-36900][BUILD][TESTS][FOLLOWUP] Try 5g, not 6g, for test mem limit
Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #34250:
URL: https://github.com/apache/spark/pull/34250
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #34250: [SPARK-36900][BUILD][TESTS][FOLLOWUP] Try 5g, not 6g, for test mem limit
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #34250:
URL: https://github.com/apache/spark/pull/34250#issuecomment-941892104
That's fine, but I'm guessing it's not actually related to this change, but, who knows
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34250: [SPARK-36900][BUILD][TESTS][FOLLOWUP] Try 5g, not 6g, for test mem limit
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34250:
URL: https://github.com/apache/spark/pull/34250#issuecomment-941281523
Thank you for keeping trying this optimization, @srowen and @HyukjinKwon .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34250: [SPARK-36900][BUILD][TESTS][FOLLOWUP] Try 5g, not 6g, for test mem limit
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34250:
URL: https://github.com/apache/spark/pull/34250#issuecomment-941872717
Ah .. flakiness still persists. I could hack with calling GC for TPC-DS but seems like other tests are flaky too (e.g., https://github.com/apache/spark/runs/3871942012).
Probably we should find other ways around to 1. reduce memory usage with JDK 17 in tests or 2. add another environment variable (or respecting JAVA_OPTS) in these places we have ..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34250: [SPARK-36900][BUILD][TESTS][FOLLOWUP] Try 5g, not 6g, for test mem limit
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34250:
URL: https://github.com/apache/spark/pull/34250#issuecomment-941910237
I will follow up and bring this in back if this wasn't the reason. Thanks @srowen.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #34250: [SPARK-36900][BUILD][TESTS][FOLLOWUP] Try 5g, not 6g, for test mem limit
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #34250:
URL: https://github.com/apache/spark/pull/34250#issuecomment-941892104
That's fine, but I'm guessing it's not actually related to this change, but, who knows
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #34250: [SPARK-36900][BUILD][TESTS][FOLLOWUP] Try 5g, not 6g, for test mem limit
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #34250:
URL: https://github.com/apache/spark/pull/34250#issuecomment-941281523
Thank you for keeping trying this optimization, @srowen and @HyukjinKwon .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34250: [SPARK-36900][BUILD][TESTS][FOLLOWUP] Try 5g, not 6g, for test mem limit
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34250:
URL: https://github.com/apache/spark/pull/34250#issuecomment-940611879
im gonna just push this in. the tests already passed with both 4g and 6g. To actually test this change, we should monitor other builds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34250: [SPARK-36900][BUILD][TESTS][FOLLOWUP] Try 5g, not 6g, for test mem limit
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34250:
URL: https://github.com/apache/spark/pull/34250#issuecomment-940611944
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34250: [SPARK-36900][BUILD][TESTS][FOLLOWUP] Try 5g, not 6g, for test mem limit
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34250:
URL: https://github.com/apache/spark/pull/34250#issuecomment-941873364
Since we don't support JDK 17 yet (and don't run any tests in CI), let me revert this one first since this blocks other PRs for now 😢 .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #34250: [SPARK-36900][BUILD][TESTS][FOLLOWUP] Try 5g, not 6g, for test mem limit
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #34250:
URL: https://github.com/apache/spark/pull/34250#issuecomment-941872717
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org