You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/07/20 19:10:05 UTC
[GitHub] [spark] viirya opened a new pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings
viirya opened a new pull request #33447:
URL: https://github.com/apache/spark/pull/33447
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html
2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'.
4. Be sure to keep the PR description updated to reflect all changes.
5. Please write your PR title to summarize what this PR proposes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
7. If you want to add a new configuration, please read the guideline first for naming configurations in
'core/src/main/scala/org/apache/spark/internal/config/ConfigEntry.scala'.
8. If you want to add or modify an error type or message, please read the guideline first in
'core/src/main/resources/error/README.md'.
-->
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->
### Does this PR introduce _any_ user-facing change?
<!--
Note that it means *any* user-facing change including all aspects such as the documentation fix.
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
If possible, please also clarify if this is a user-facing change compared to the released Spark versions or within the unreleased branches such as master.
If no, write 'No'.
-->
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883727154
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45872/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883941090
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141361/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883830414
**[Test build #141372 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141372/testReport)** for PR 33447 at commit [`84d7451`](https://github.com/apache/spark/commit/84d7451b79f1f497a4c31cd268c346dccc149ce7).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884625506
**[Test build #141462 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141462/testReport)** for PR 33447 at commit [`394769d`](https://github.com/apache/spark/commit/394769df035db38bb91f1e110a4adfdaa778f419).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883629976
**[Test build #141355 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141355/testReport)** for PR 33447 at commit [`e9fba3c`](https://github.com/apache/spark/commit/e9fba3ca4630913cfb806322e3f41830f60235e5).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884583313
**[Test build #141427 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141427/testReport)** for PR 33447 at commit [`372a6ef`](https://github.com/apache/spark/commit/372a6ef9de95ac674ea770999e4f1814fe1bcb96).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884577645
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45966/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885408691
**[Test build #141518 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141518/testReport)** for PR 33447 at commit [`593f7c9`](https://github.com/apache/spark/commit/593f7c9d71e985ab0bda7c3da579ccdde1083a9f).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884592635
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45971/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885433617
> So after this change, PRs won't pass Jenkins tests unless we increase the time out of it?
Yea, this is also a concern.
Let me try two changes. 1. Only set lower mem for sbt in GITHUB_ACTIONS and leave Jenkins untouch. 2. Set higher MaxMetaspaceSize for Hive test in Jenkins too. (Currently I only set it for GA).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885747342
> @viirya could we backport this to branch-3.1 and branch-3.0 too please?
Ok. Let me backport it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884625506
**[Test build #141462 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141462/testReport)** for PR 33447 at commit [`394769d`](https://github.com/apache/spark/commit/394769df035db38bb91f1e110a4adfdaa778f419).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885591578
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885574259
**[Test build #141531 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141531/testReport)** for PR 33447 at commit [`dc48476`](https://github.com/apache/spark/commit/dc4847674bfeaab2e2a0c760e5675d506a134977).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun edited a comment on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun edited a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-891981642
Please don't say it's `Revert` in the PR title, @viirya ~ :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] c21 commented on a change in pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
c21 commented on a change in pull request #33447:
URL: https://github.com/apache/spark/pull/33447#discussion_r675462166
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala
##########
@@ -396,7 +396,9 @@ class SQLMetricsSuite extends SharedSparkSession with SQLMetricsTestUtils
}
}
- test("SPARK-32629: ShuffledHashJoin(full outer) metrics") {
+ // TODO (SPARK-36272): Reenable this after we figure out why the expected size doesn't
+ // match after we adjust building's memory settings.
+ ignore("SPARK-32629: ShuffledHashJoin(full outer) metrics") {
Review comment:
Thanks @viirya, I can also help look into this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884577363
**[Test build #141452 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141452/testReport)** for PR 33447 at commit [`476241a`](https://github.com/apache/spark/commit/476241a68d70e01adafdbce4f10164e857ffe4f6).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884775014
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45998/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885554275
**[Test build #141547 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141547/testReport)** for PR 33447 at commit [`424c9d0`](https://github.com/apache/spark/commit/424c9d0d55773faac3a9f1b9c395e5e4556fc869).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884577363
**[Test build #141452 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141452/testReport)** for PR 33447 at commit [`476241a`](https://github.com/apache/spark/commit/476241a68d70e01adafdbce4f10164e857ffe4f6).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884636560
**[Test build #141439 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141439/testReport)** for PR 33447 at commit [`21ea82b`](https://github.com/apache/spark/commit/21ea82b40b0a073ac38941410c289c3594088785).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884312750
**[Test build #141427 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141427/testReport)** for PR 33447 at commit [`372a6ef`](https://github.com/apache/spark/commit/372a6ef9de95ac674ea770999e4f1814fe1bcb96).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884429200
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45946/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884523704
Okay. Most tests are passed. Only Hive test has `java.lang.OutOfMemoryError: Metaspace`. Trying to increase MaxMetaspaceSize a bit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885265825
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46038/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885367302
**[Test build #141531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141531/testReport)** for PR 33447 at commit [`dc48476`](https://github.com/apache/spark/commit/dc4847674bfeaab2e2a0c760e5675d506a134977).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885135588
**[Test build #141515 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141515/testReport)** for PR 33447 at commit [`ab99396`](https://github.com/apache/spark/commit/ab9939608feddca31865deec18710122425fd5d1).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885412222
**[Test build #141520 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141520/testReport)** for PR 33447 at commit [`6488ca1`](https://github.com/apache/spark/commit/6488ca12a2469c767b5a11b44166c0af6eaf67ca).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885334239
**[Test build #141524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141524/testReport)** for PR 33447 at commit [`a3c3aa3`](https://github.com/apache/spark/commit/a3c3aa3688afdc996e8a29368563cca510cc7751).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #33447:
URL: https://github.com/apache/spark/pull/33447#discussion_r673522644
##########
File path: pom.xml
##########
@@ -2611,9 +2611,9 @@
</args>
<jvmArgs>
<jvmArg>-Xss128m</jvmArg>
- <jvmArg>-Xms4g</jvmArg>
- <jvmArg>-Xmx4g</jvmArg>
- <jvmArg>-XX:MaxMetaspaceSize=2g</jvmArg>
+ <jvmArg>-Xms6g</jvmArg>
+ <jvmArg>-Xmx6g</jvmArg>
+ <jvmArg>-XX:MaxMetaspaceSize=512m</jvmArg>
Review comment:
FYI, Under 1G, hive job failed with `OOM: Metaspace`. You can see the error message in the original PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883727806
Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45875/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884097466
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141370/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884676248
Created #33475 with only SERIAL_SBT_TESTS and parallelism change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #33447:
URL: https://github.com/apache/spark/pull/33447
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884211090
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141383/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884798320
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45998/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #33447:
URL: https://github.com/apache/spark/pull/33447#discussion_r673756931
##########
File path: build/sbt-launch-lib.bash
##########
@@ -17,7 +17,7 @@ declare -a java_args
declare -a scalac_args
declare -a sbt_commands
declare -a maven_profiles
-declare sbt_default_mem=4096
+declare sbt_default_mem=2048
Review comment:
After reducing sbt memory, seems better. Continue to tune it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885534078
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883803789
**[Test build #141366 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141366/testReport)** for PR 33447 at commit [`2f8ea89`](https://github.com/apache/spark/commit/2f8ea895ce61f8deb958d5d0a5b517ce6c368197).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885502071
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885180164
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46032/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885203368
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141515/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885256473
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46035/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884744138
**[Test build #141452 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141452/testReport)** for PR 33447 at commit [`476241a`](https://github.com/apache/spark/commit/476241a68d70e01adafdbce4f10164e857ffe4f6).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884082382
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45916/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884036682
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45911/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884608315
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141447/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883897114
**[Test build #141357 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141357/testReport)** for PR 33447 at commit [`ad6dbaa`](https://github.com/apache/spark/commit/ad6dbaa11bff4c363e01403dba8d88da6e0e1500).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885534078
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883942091
**[Test build #141393 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141393/testReport)** for PR 33447 at commit [`0d028af`](https://github.com/apache/spark/commit/0d028af87d2bbf9165a9324de52aa3336c096808).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884495206
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45958/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885396871
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46049/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya removed a comment on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885417386
And, maven's metaspace size config is not big enough for Hive test in Jenkins. Currently I configure it in `build_and_test.yml` for GA. Maybe I need to set it back to 2g in `pom.xml` to make Jenkins test happy too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883874672
**[Test build #141356 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141356/testReport)** for PR 33447 at commit [`4f98faa`](https://github.com/apache/spark/commit/4f98faa7a0969e5c187a106f28faf0416b98afc2).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884097466
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141370/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #33447:
URL: https://github.com/apache/spark/pull/33447#discussion_r673522273
##########
File path: pom.xml
##########
@@ -2611,9 +2611,9 @@
</args>
<jvmArgs>
<jvmArg>-Xss128m</jvmArg>
- <jvmArg>-Xms4g</jvmArg>
- <jvmArg>-Xmx4g</jvmArg>
- <jvmArg>-XX:MaxMetaspaceSize=2g</jvmArg>
+ <jvmArg>-Xms6g</jvmArg>
+ <jvmArg>-Xmx6g</jvmArg>
Review comment:
Hi, @viirya . Currently, GitHub Action kills our job because it consume more memory. The main return code is 137 . I'm not sure about this increament.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884656496
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45981/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883733461
**[Test build #141361 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141361/testReport)** for PR 33447 at commit [`0f890e3`](https://github.com/apache/spark/commit/0f890e3bbc8acd7aa057da88861c9cc8b9358183).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883839467
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45882/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #33447:
URL: https://github.com/apache/spark/pull/33447#discussion_r674168839
##########
File path: pom.xml
##########
@@ -2611,9 +2611,9 @@
</args>
<jvmArgs>
<jvmArg>-Xss128m</jvmArg>
- <jvmArg>-Xms4g</jvmArg>
- <jvmArg>-Xmx4g</jvmArg>
- <jvmArg>-XX:MaxMetaspaceSize=2g</jvmArg>
+ <jvmArg>-Xms6g</jvmArg>
+ <jvmArg>-Xmx6g</jvmArg>
+ <jvmArg>-XX:MaxMetaspaceSize=512m</jvmArg>
Review comment:
It will run with errors. Last time, Jenkins runs forever and was killed by timeout.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883859273
**[Test build #141355 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141355/testReport)** for PR 33447 at commit [`e9fba3c`](https://github.com/apache/spark/commit/e9fba3ca4630913cfb806322e3f41830f60235e5).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #33447:
URL: https://github.com/apache/spark/pull/33447#discussion_r675451735
##########
File path: dev/run-tests.py
##########
@@ -270,7 +270,12 @@ def exec_sbt(sbt_args=()):
"""Will call SBT in the current directory with the list of mvn_args passed
in and returns the subprocess for any further processing"""
- sbt_cmd = [os.path.join(SPARK_HOME, "build", "sbt")] + sbt_args
+ sbt_cmd = [os.path.join(SPARK_HOME, "build", "sbt")]
+
+ if "GITHUB_ACTIONS" in os.environ:
+ sbt_cmd = sbt_cmd + ['-mem 2300']
Review comment:
```suggestion
sbt_cmd = sbt_cmd + ['-mem', '2300']
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885341108
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46040/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885334239
**[Test build #141524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141524/testReport)** for PR 33447 at commit [`a3c3aa3`](https://github.com/apache/spark/commit/a3c3aa3688afdc996e8a29368563cca510cc7751).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883887799
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141356/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885704098
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141541/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885424374
**[Test build #141541 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141541/testReport)** for PR 33447 at commit [`6c6930f`](https://github.com/apache/spark/commit/6c6930fccdeafde4cc977a938311d1f2baa48d86).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883953754
**[Test build #141363 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141363/testReport)** for PR 33447 at commit [`818cd53`](https://github.com/apache/spark/commit/818cd5334d4a6fc22b625b34598b3aa6fc4b110e).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884641173
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141439/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya edited a comment on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya edited a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885433617
> So after this change, PRs won't pass Jenkins tests unless we increase the time out of it?
Yea, this is also a concern.
Let me try two changes. 1. Only set lower mem for sbt in GITHUB_ACTIONS and leave Jenkins untouch. 2. Set higher MaxMetaspaceSize for Hive test in Jenkins too. (Currently I only set it for GA. So Jenkins Hive test failed with metaspace oom now)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885515248
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46065/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885600470
**[Test build #141550 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141550/testReport)** for PR 33447 at commit [`b3009d5`](https://github.com/apache/spark/commit/b3009d5ed8d85372fa7affbfdbe47c264bfa1d82).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883922648
**[Test build #141361 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141361/testReport)** for PR 33447 at commit [`0f890e3`](https://github.com/apache/spark/commit/0f890e3bbc8acd7aa057da88861c9cc8b9358183).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883699726
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45871/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885417386
And, maven's metaspace size config is not big enough for Hive test in Jenkins. Currently I configure it in `build_and_test.yml` for GA. Maybe I need to set it back to 2g in `pom.xml` to make Jenkins test happy too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-891631725
I got some feedback from GitHub:
> Hi Hyukjin,
>
> Thanks for getting back to us.
>
> I will be taking over this ticket for now because Carlos will be out for the next few days.
>
> I have some feedback from our engineers regarding the issue you experienced, and a configuration change that was causing the memory issues has been rolled back.
>
> Can you confirm that you are no longer impacted with this issue?
>
> Looking forward to hearing back from you.
>
> Thanks,
> Faisal
Do you guys think we should try to revert and see how it goes? or leave it as is?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #33447:
URL: https://github.com/apache/spark/pull/33447#discussion_r675347057
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala
##########
@@ -396,7 +396,9 @@ class SQLMetricsSuite extends SharedSparkSession with SQLMetricsTestUtils
}
}
- test("SPARK-32629: ShuffledHashJoin(full outer) metrics") {
+ // We adjust building's memory settings and it seems affecting this test.
Review comment:
For completeness, I would file a JIRA for this one though. e.g.) TODO(SPARK-XXXXX): Reenable this blah blah
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885401003
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46049/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884798360
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45998/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883975180
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141363/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya edited a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya edited a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885359431
Seems all tests are passed except for `sql - other tests` which failed due to `SQLMetricsSuite.SPARK-32629: ShuffledHashJoin(full outer) metrics`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #33447:
URL: https://github.com/apache/spark/pull/33447#discussion_r674509703
##########
File path: pom.xml
##########
@@ -2662,7 +2662,7 @@
<include>**/*Suite.java</include>
</includes>
<reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
- <argLine>-ea -Xmx4g -Xss4m -XX:MaxMetaspaceSize=2g -XX:ReservedCodeCacheSize=${CodeCacheSize} -Dio.netty.tryReflectionSetAccessible=true</argLine>
+ <argLine>-ea -Xms1024m -Xmx3200m -Xss4m -XX:MaxMetaspaceSize=1200m -XX:ReservedCodeCacheSize=${CodeCacheSize} -Dio.netty.tryReflectionSetAccessible=true</argLine>
Review comment:
Shall we remove `-Xms1024m`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884031098
**[Test build #141366 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141366/testReport)** for PR 33447 at commit [`2f8ea89`](https://github.com/apache/spark/commit/2f8ea895ce61f8deb958d5d0a5b517ce6c368197).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885390894
Yeah, is it not related to this PR? I think the test failure in `SQLMetricsSuite.SPARK-32629` seems new to me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885365778
Can we skip/ignore the test `SPARK-32629` for now? Otherwise looks pretty good
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884432790
More test are succeeded. Let me tune it further.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884653772
Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45981/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883816993
Thanks for investigating this @viirya.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883825434
FWIW, I created a ticket at GitHub Actions:
<img width="987" alt="Screen Shot 2021-07-21 at 10 57 24 AM" src="https://user-images.githubusercontent.com/6477701/126418579-fd5f6b94-b01c-4c7a-b215-4f32cc1b7fb6.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885534733
**[Test build #141550 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141550/testReport)** for PR 33447 at commit [`b3009d5`](https://github.com/apache/spark/commit/b3009d5ed8d85372fa7affbfdbe47c264bfa1d82).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884462040
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45946/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884657570
Looks like we're almost there!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885442998
**[Test build #141543 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141543/testReport)** for PR 33447 at commit [`5323d69`](https://github.com/apache/spark/commit/5323d69e254d579a380603fc24fce3f045b596f2).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883975180
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141363/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884342673
Thank you so much for investing this, @viirya .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885390958
Actually, let me just merge this in first. let see if it fails consistently.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883700961
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45870/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #33447:
URL: https://github.com/apache/spark/pull/33447#discussion_r675680402
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala
##########
@@ -396,7 +396,9 @@ class SQLMetricsSuite extends SharedSparkSession with SQLMetricsTestUtils
}
}
- test("SPARK-32629: ShuffledHashJoin(full outer) metrics") {
+ // TODO (SPARK-36272): Reenable this after we figure out why the expected size doesn't
+ // match after we adjust building's memory settings.
+ ignore("SPARK-32629: ShuffledHashJoin(full outer) metrics") {
Review comment:
Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883803308
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45879/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885601279
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141550/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885273310
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46035/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883628957
GA continues to fail. Trying to change memory settings and see the effect.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884434178
**[Test build #141439 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141439/testReport)** for PR 33447 at commit [`21ea82b`](https://github.com/apache/spark/commit/21ea82b40b0a073ac38941410c289c3594088785).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885367302
**[Test build #141531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141531/testReport)** for PR 33447 at commit [`dc48476`](https://github.com/apache/spark/commit/dc4847674bfeaab2e2a0c760e5675d506a134977).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885704098
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141541/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883758802
**[Test build #141363 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141363/testReport)** for PR 33447 at commit [`818cd53`](https://github.com/apache/spark/commit/818cd5334d4a6fc22b625b34598b3aa6fc4b110e).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #33447:
URL: https://github.com/apache/spark/pull/33447#discussion_r674371358
##########
File path: pom.xml
##########
@@ -2611,9 +2611,9 @@
</args>
<jvmArgs>
<jvmArg>-Xss128m</jvmArg>
- <jvmArg>-Xms4g</jvmArg>
- <jvmArg>-Xmx4g</jvmArg>
- <jvmArg>-XX:MaxMetaspaceSize=2g</jvmArg>
+ <jvmArg>-Xms6g</jvmArg>
+ <jvmArg>-Xmx6g</jvmArg>
+ <jvmArg>-XX:MaxMetaspaceSize=512m</jvmArg>
Review comment:
Yea, 1g seems not enough too.. Trying to add a bit more.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883655878
**[Test build #141356 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141356/testReport)** for PR 33447 at commit [`4f98faa`](https://github.com/apache/spark/commit/4f98faa7a0969e5c187a106f28faf0416b98afc2).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885160208
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46032/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883942091
**[Test build #141393 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141393/testReport)** for PR 33447 at commit [`0d028af`](https://github.com/apache/spark/commit/0d028af87d2bbf9165a9324de52aa3336c096808).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884857602
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141462/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883707928
**[Test build #141360 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141360/testReport)** for PR 33447 at commit [`8aa69dc`](https://github.com/apache/spark/commit/8aa69dc8b35013d13af195537d974c11a5d99099).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883825743
**[Test build #141370 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141370/testReport)** for PR 33447 at commit [`aacc1dd`](https://github.com/apache/spark/commit/aacc1dd3cc2908a230c271b34addd5394889285b).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884608315
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141447/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884016496
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885537623
@viirya could we backport this to branch-3.1 and branch-3.0 too please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885313659
**[Test build #141522 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141522/testReport)** for PR 33447 at commit [`2176f82`](https://github.com/apache/spark/commit/2176f8295e42ca853475c53c9babe5f2f741a39d).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] c21 commented on a change in pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
c21 commented on a change in pull request #33447:
URL: https://github.com/apache/spark/pull/33447#discussion_r675498120
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala
##########
@@ -396,7 +396,9 @@ class SQLMetricsSuite extends SharedSparkSession with SQLMetricsTestUtils
}
}
- test("SPARK-32629: ShuffledHashJoin(full outer) metrics") {
+ // TODO (SPARK-36272): Reenable this after we figure out why the expected size doesn't
+ // match after we adjust building's memory settings.
+ ignore("SPARK-32629: ShuffledHashJoin(full outer) metrics") {
Review comment:
Filed https://github.com/apache/spark/pull/33494 now to log several data size metrics separately and keep debugging there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883892606
**[Test build #141383 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141383/testReport)** for PR 33447 at commit [`85e2c98`](https://github.com/apache/spark/commit/85e2c98afd341d2f7e6c27a9771f1dc70ebc24e2).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884591645
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45966/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884591953
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885487506
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46061/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885131972
All tests are passed except for `hive - other tests` which needs more Metaspace size. Let me rise Metaspace size. Hopefully it won't cause 137 on the container.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884211090
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141383/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #33447:
URL: https://github.com/apache/spark/pull/33447#discussion_r673526374
##########
File path: pom.xml
##########
@@ -2611,9 +2611,9 @@
</args>
<jvmArgs>
<jvmArg>-Xss128m</jvmArg>
- <jvmArg>-Xms4g</jvmArg>
- <jvmArg>-Xmx4g</jvmArg>
- <jvmArg>-XX:MaxMetaspaceSize=2g</jvmArg>
+ <jvmArg>-Xms6g</jvmArg>
+ <jvmArg>-Xmx6g</jvmArg>
+ <jvmArg>-XX:MaxMetaspaceSize=512m</jvmArg>
Review comment:
I once reduced it to 128m and see `OOM: Metaspace`. Currently, 512m is still running. If it fails, probably increasing it back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885273310
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46035/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885037999
**[Test build #141505 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141505/testReport)** for PR 33447 at commit [`05b5b3e`](https://github.com/apache/spark/commit/05b5b3e52929c610d9d07de7607af95c59dfbfb0).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883892606
**[Test build #141383 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141383/testReport)** for PR 33447 at commit [`85e2c98`](https://github.com/apache/spark/commit/85e2c98afd341d2f7e6c27a9771f1dc70ebc24e2).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885343648
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46042/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883757344
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45872/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883887799
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141356/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883866464
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883732727
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883803308
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45879/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884495206
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45958/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884719323
`hive - other tests` looks like not running? I'm going to commit an empty commit to trigger it again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya edited a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya edited a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884719323
`hive - other tests` looks like not running? I'm going to commit an empty commit to trigger another round.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885601279
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141550/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on a change in pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on a change in pull request #33447:
URL: https://github.com/apache/spark/pull/33447#discussion_r674506580
##########
File path: project/SparkBuild.scala
##########
@@ -1197,7 +1197,7 @@ object TestSettings {
(Global / concurrentRestrictions) := {
// The number of concurrent test groups is empirically chosen based on experience
// with Jenkins flakiness.
- if (sys.env.contains("SERIAL_SBT_TESTS")) (Global / concurrentRestrictions).value
+ if (sys.env.contains("SERIAL_SBT_TESTS")) Seq(Tags.limit(Tags.Test, 1))
Review comment:
Why do we make this change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885472389
**[Test build #141547 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141547/testReport)** for PR 33447 at commit [`424c9d0`](https://github.com/apache/spark/commit/424c9d0d55773faac3a9f1b9c395e5e4556fc869).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885425699
> This patch fails from timeout after a configured wait of 500m.
So after this change, PRs won't pass Jenkins tests unless we increase the time out of it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885470667
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46059/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885441703
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46059/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-891998593
@dongjoon-hyun Okay. Thanks for reminding. Created a new JIRA and raising memory and parallelism in #33623.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885457009
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46059/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885513532
**[Test build #141524 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141524/testReport)** for PR 33447 at commit [`a3c3aa3`](https://github.com/apache/spark/commit/a3c3aa3688afdc996e8a29368563cca510cc7751).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #33447:
URL: https://github.com/apache/spark/pull/33447#discussion_r675464935
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala
##########
@@ -396,7 +396,9 @@ class SQLMetricsSuite extends SharedSparkSession with SQLMetricsTestUtils
}
}
- test("SPARK-32629: ShuffledHashJoin(full outer) metrics") {
+ // TODO (SPARK-36272): Reenable this after we figure out why the expected size doesn't
+ // match after we adjust building's memory settings.
+ ignore("SPARK-32629: ShuffledHashJoin(full outer) metrics") {
Review comment:
🙏
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885241699
**[Test build #141520 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141520/testReport)** for PR 33447 at commit [`6488ca1`](https://github.com/apache/spark/commit/6488ca12a2469c767b5a11b44166c0af6eaf67ca).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883912475
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883846277
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45882/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885202345
**[Test build #141515 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141515/testReport)** for PR 33447 at commit [`ab99396`](https://github.com/apache/spark/commit/ab9939608feddca31865deec18710122425fd5d1).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885472389
**[Test build #141547 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141547/testReport)** for PR 33447 at commit [`424c9d0`](https://github.com/apache/spark/commit/424c9d0d55773faac3a9f1b9c395e5e4556fc869).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884273707
**[Test build #141393 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141393/testReport)** for PR 33447 at commit [`0d028af`](https://github.com/apache/spark/commit/0d028af87d2bbf9165a9324de52aa3336c096808).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #33447:
URL: https://github.com/apache/spark/pull/33447#discussion_r674509759
##########
File path: pom.xml
##########
@@ -2713,7 +2713,7 @@
<reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
<junitxml>.</junitxml>
<filereports>SparkTestSuite.txt</filereports>
- <argLine>-ea -Xmx4g -Xss4m -XX:MaxMetaspaceSize=2g -XX:ReservedCodeCacheSize=${CodeCacheSize} -Dio.netty.tryReflectionSetAccessible=true</argLine>
+ <argLine>-ea -Xms1024m -Xmx3200m -Xss4m -XX:MaxMetaspaceSize=1200m -XX:ReservedCodeCacheSize=${CodeCacheSize} -Dio.netty.tryReflectionSetAccessible=true</argLine>
Review comment:
Shall we remove `-Xms1024m`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883866464
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884461980
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45946/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883778276
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45877/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884113087
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45916/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883764794
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45877/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884665799
BTW, the optimized (minimized) values looks good to me. I'm just wondering if there is something which is effectively not required here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885420113
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884132568
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45916/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885467810
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46061/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884838567
**[Test build #141462 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141462/testReport)** for PR 33447 at commit [`394769d`](https://github.com/apache/spark/commit/394769df035db38bb91f1e110a4adfdaa778f419).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885335070
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46042/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884591954
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883629976
**[Test build #141355 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141355/testReport)** for PR 33447 at commit [`e9fba3c`](https://github.com/apache/spark/commit/e9fba3ca4630913cfb806322e3f41830f60235e5).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884758766
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141452/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya edited a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya edited a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885131972
All tests are passed except for `hive - other tests` which needs more Metaspace size:
```
[error] Uncaught exception when running org.apache.spark.sql.hive.execution.HiveTypeCoercionSuite: java.lang.OutOfMemoryError: Metaspace
```
Let me rise Metaspace size. Hopefully it won't cause 137 on the container.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885135588
**[Test build #141515 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141515/testReport)** for PR 33447 at commit [`ab99396`](https://github.com/apache/spark/commit/ab9939608feddca31865deec18710122425fd5d1).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883851141
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45888/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884592627
Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45971/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885496739
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46065/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883827110
Thank you @HyukjinKwon
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883941090
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141361/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885077040
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46022/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884016496
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885233710
**[Test build #141518 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141518/testReport)** for PR 33447 at commit [`593f7c9`](https://github.com/apache/spark/commit/593f7c9d71e985ab0bda7c3da579ccdde1083a9f).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885290198
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46038/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885534733
**[Test build #141550 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141550/testReport)** for PR 33447 at commit [`b3009d5`](https://github.com/apache/spark/commit/b3009d5ed8d85372fa7affbfdbe47c264bfa1d82).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884302190
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141393/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885195677
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46032/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883743567
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45872/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884527770
**[Test build #141447 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141447/testReport)** for PR 33447 at commit [`d0b02dd`](https://github.com/apache/spark/commit/d0b02dd1e7e35929ca5c649fcd55e133e1962d91).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on a change in pull request #33447:
URL: https://github.com/apache/spark/pull/33447#discussion_r674509888
##########
File path: project/SparkBuild.scala
##########
@@ -1120,9 +1120,9 @@ object TestSettings {
.map { case (k,v) => s"-D$k=$v" }.toSeq,
(Test / javaOptions) += "-ea",
// SPARK-29282 This is for consistency between JDK8 and JDK11.
- (Test / javaOptions) ++= "-Xmx4g -Xss4m -XX:MaxMetaspaceSize=2g -XX:+UseParallelGC -XX:-UseDynamicNumberOfGCThreads"
+ (Test / javaOptions) ++= "-Xms1024m -Xmx3200m -Xss4m -XX:MaxMetaspaceSize=1200m -XX:+UseParallelGC -XX:-UseDynamicNumberOfGCThreads -XX:ReservedCodeCacheSize=128m"
Review comment:
ditto for `-Xms1024m`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885416085
> Yeah, is it not related to this PR? I think the test failure in `SQLMetricsSuite.SPARK-32629` seems new to me.
This only touched memory settings of building. Sounds unlikely it will affect `SQLMetricsSuite`. But I saw it fails consistently in GA. Let me try locally to verify.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885203368
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141515/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885423347
`SQLMetricsSuite.SPARK-32629` can pass locally in master branch. The test looks like related to build side size of shuffle hash join. So probably memory setting can affect it.
Do you prefer just ignoring it for now and fix it later?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885563873
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141547/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gengliangwang commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
gengliangwang commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-891967066
@HyukjinKwon I think we can try reverting this one. Higher parallelism and memory settings can reduce the test time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884312750
**[Test build #141427 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141427/testReport)** for PR 33447 at commit [`372a6ef`](https://github.com/apache/spark/commit/372a6ef9de95ac674ea770999e4f1814fe1bcb96).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884462040
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45946/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885401003
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46049/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885363829
Use `EXCLUDED_TAGS`. Reverted some changes like `Xms` which may be unnecessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883757344
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45872/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-891981162
Okay to test it by reverting it. I suppose higher memory settings can make CI more smooth (faster).
I will submit a reverting PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883779573
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45877/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885385200
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46049/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885442998
**[Test build #141543 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141543/testReport)** for PR 33447 at commit [`5323d69`](https://github.com/apache/spark/commit/5323d69e254d579a380603fc24fce3f045b596f2).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885470667
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46059/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883701915
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45870/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya edited a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings
Posted by GitBox <gi...@apache.org>.
viirya edited a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883628957
GA continues to fail, e.g.,
```
Error: running /home/runner/work/spark-1/spark-1/build/sbt -Phadoop-3.2 -Phive-2.3 -Phive-thriftserver catalyst/test hive-thriftserver/test ; received return code 137
Error: Process completed with exit code 18.
```
Trying to change memory settings and see the effect.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884058271
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884434178
**[Test build #141439 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141439/testReport)** for PR 33447 at commit [`21ea82b`](https://github.com/apache/spark/commit/21ea82b40b0a073ac38941410c289c3594088785).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883824116
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45882/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884659563
Looks like `sql - other tests` was failed due to `org.apache.spark.sql.execution.metric.SQLMetricsSuite`, not memory issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883703479
**[Test build #141357 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141357/testReport)** for PR 33447 at commit [`ad6dbaa`](https://github.com/apache/spark/commit/ad6dbaa11bff4c363e01403dba8d88da6e0e1500).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883655878
**[Test build #141356 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141356/testReport)** for PR 33447 at commit [`4f98faa`](https://github.com/apache/spark/commit/4f98faa7a0969e5c187a106f28faf0416b98afc2).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885563873
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141547/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885502062
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883920330
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45899/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-891981642
Please don't say it's `Revert` commit, @viirya .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884598906
**[Test build #141447 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141447/testReport)** for PR 33447 at commit [`d0b02dd`](https://github.com/apache/spark/commit/d0b02dd1e7e35929ca5c649fcd55e133e1962d91).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885483072
**[Test build #141522 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141522/testReport)** for PR 33447 at commit [`2176f82`](https://github.com/apache/spark/commit/2176f8295e42ca853475c53c9babe5f2f741a39d).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885359431
Seems all tests are passed except for `sql - other tests` which failed due to ``SQLMetricsSuite.SPARK-32629: ShuffledHashJoin(full outer) metrics`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884007240
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45911/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883920309
Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45899/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885425378
I prefer whichever way you want to do :-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883718065
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45871/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883803789
**[Test build #141366 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141366/testReport)** for PR 33447 at commit [`2f8ea89`](https://github.com/apache/spark/commit/2f8ea895ce61f8deb958d5d0a5b517ce6c368197).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883988703
**[Test build #141398 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141398/testReport)** for PR 33447 at commit [`c47bbb6`](https://github.com/apache/spark/commit/c47bbb6a3af7cfc661638ec6f28b2c196cf3c54f).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885537448
Merged to master and branch-3.2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883758802
**[Test build #141363 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141363/testReport)** for PR 33447 at commit [`818cd53`](https://github.com/apache/spark/commit/818cd5334d4a6fc22b625b34598b3aa6fc4b110e).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885574821
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46069/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885069216
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141481/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #33447:
URL: https://github.com/apache/spark/pull/33447#discussion_r675270796
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -163,9 +164,10 @@ jobs:
- name: Run tests
env: ${{ fromJSON(needs.configure-jobs.outputs.envs) }}
run: |
- # Hive and SQL tests become flaky when running in parallel as it's too intensive.
- if [[ "$MODULES_TO_TEST" == "hive" ]] || [[ "$MODULES_TO_TEST" == "sql" ]]; then export SERIAL_SBT_TESTS=1; fi
- ./dev/run-tests --parallelism 2 --modules "$MODULES_TO_TEST" --included-tags "$INCLUDED_TAGS" --excluded-tags "$EXCLUDED_TAGS"
+ # Hive "other tests" test needs larger metaspace size based on experiment.
+ if [[ "$MODULES_TO_TEST" == "hive" ]] && [[ "$COMMENT_TO_TEST" == "- other tests" ]]; then export METASPACE_SIZE=2g; fi
Review comment:
I think you can check `EXCLUDED_TAGS` contains `SlowHiveTest` instead of adding new env `COMMENT_TO_TEST`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884074233
**[Test build #141370 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141370/testReport)** for PR 33447 at commit [`aacc1dd`](https://github.com/apache/spark/commit/aacc1dd3cc2908a230c271b34addd5394889285b).
* This patch **fails from timeout after a configured wait of `500m`**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884659011
Yea, seems so. :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884527770
**[Test build #141447 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141447/testReport)** for PR 33447 at commit [`d0b02dd`](https://github.com/apache/spark/commit/d0b02dd1e7e35929ca5c649fcd55e133e1962d91).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884592635
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45971/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883779227
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45879/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884478021
Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45958/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883846277
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45882/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
viirya commented on a change in pull request #33447:
URL: https://github.com/apache/spark/pull/33447#discussion_r675274056
##########
File path: .github/workflows/build_and_test.yml
##########
@@ -163,9 +164,10 @@ jobs:
- name: Run tests
env: ${{ fromJSON(needs.configure-jobs.outputs.envs) }}
run: |
- # Hive and SQL tests become flaky when running in parallel as it's too intensive.
- if [[ "$MODULES_TO_TEST" == "hive" ]] || [[ "$MODULES_TO_TEST" == "sql" ]]; then export SERIAL_SBT_TESTS=1; fi
- ./dev/run-tests --parallelism 2 --modules "$MODULES_TO_TEST" --included-tags "$INCLUDED_TAGS" --excluded-tags "$EXCLUDED_TAGS"
+ # Hive "other tests" test needs larger metaspace size based on experiment.
+ if [[ "$MODULES_TO_TEST" == "hive" ]] && [[ "$COMMENT_TO_TEST" == "- other tests" ]]; then export METASPACE_SIZE=2g; fi
Review comment:
Oh. Got it. Let me change it in next commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883920330
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45899/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884058271
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885424374
**[Test build #141541 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141541/testReport)** for PR 33447 at commit [`6c6930f`](https://github.com/apache/spark/commit/6c6930fccdeafde4cc977a938311d1f2baa48d86).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884857602
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141462/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884132568
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45916/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885195677
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46032/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885591578
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885391058
ohh can you file a JIRA please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885281563
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46038/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884641173
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141439/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885349721
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885420113
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #33447: [SPARK-36270][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-891981352
Wow, it's a nice feedback from GitHub, @HyukjinKwon .
Instead of a revert, why don't we try to reduce the limitation one by one?
- Renable the test case
- Increase parallelism
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-883912475
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-885241699
**[Test build #141520 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141520/testReport)** for PR 33447 at commit [`6488ca1`](https://github.com/apache/spark/commit/6488ca12a2469c767b5a11b44166c0af6eaf67ca).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #33447: [SPARK-xxxxx][BUILD] Change memory settings for enabling GA
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #33447:
URL: https://github.com/apache/spark/pull/33447#issuecomment-884656496
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45981/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org