You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/02/03 05:27:49 UTC
[GitHub] [spark] attilapiros opened a new pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
attilapiros opened a new pull request #31450:
URL: https://github.com/apache/spark/pull/31450
### What changes were proposed in this pull request?
This PR adds the following metrics to track executor remove reasons during dynamic allocation:
- `numberExecutorsGracefullyDecommissioned`: number of executors which reached the finished decommissioning state and shut itself down cleanly
- `numberExecutorsDecommissionUnfinished`: executors which requested to decommission but they stopped without reaching the finished decommissioning state
- `numberExecutorsKilledByDriver`: executors killed by the driver (requested to stop)
- `numberExecutorsExitedUnexpectedly`: executors exited without driver request
### Why are the changes needed?
For supporting monitoring of dynamic allocation better with these metrics.
### Does this PR introduce _any_ user-facing change?
Yes. The new metrics will be available for monitoring.
### How was this patch tested?
With unit and integration tests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-772311962
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39400/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-776398704
**[Test build #135087 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135087/testReport)** for PR 31450 at commit [`7eb3df5`](https://github.com/apache/spark/commit/7eb3df53aae53cd8c28c69d9bf5bff40abf64e70).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] attilapiros commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
attilapiros commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-778107152
jenkins retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-774424075
**[Test build #134952 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134952/testReport)** for PR 31450 at commit [`c03258e`](https://github.com/apache/spark/commit/c03258ebef8fdd1c3d83c1fe5b77732a2069aa53).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-776444351
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39669/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-774353694
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134943/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-781232151
I just saw this. I think we don't usually port improvement back but I guess it won't break anything and pretty safe? I am fine if you feel strongly on this. cc @dongjoon-hyun too FYI.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-776443862
**[Test build #135087 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135087/testReport)** for PR 31450 at commit [`7eb3df5`](https://github.com/apache/spark/commit/7eb3df53aae53cd8c28c69d9bf5bff40abf64e70).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-778123122
**[Test build #135123 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135123/testReport)** for PR 31450 at commit [`7eb3df5`](https://github.com/apache/spark/commit/7eb3df53aae53cd8c28c69d9bf5bff40abf64e70).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-774431778
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39535/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-774453715
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134952/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-772255230
**[Test build #134812 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134812/testReport)** for PR 31450 at commit [`5560ec6`](https://github.com/apache/spark/commit/5560ec6b32f329ed6f9ad614df8f8d5c06ab33a7).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-772295859
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39400/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-774445541
**[Test build #134952 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134952/testReport)** for PR 31450 at commit [`c03258e`](https://github.com/apache/spark/commit/c03258ebef8fdd1c3d83c1fe5b77732a2069aa53).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-772311962
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39400/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-772279654
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39400/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-774453715
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134952/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-778198992
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135123/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-776398704
**[Test build #135087 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135087/testReport)** for PR 31450 at commit [`7eb3df5`](https://github.com/apache/spark/commit/7eb3df53aae53cd8c28c69d9bf5bff40abf64e70).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] attilapiros commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
attilapiros commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-772371345
I think counters better fits for this type of metrics so I will change these gauges to counter.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-776413082
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39669/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-774306946
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39526/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-774345612
**[Test build #134943 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134943/testReport)** for PR 31450 at commit [`d37289d`](https://github.com/apache/spark/commit/d37289da58c289e3327c5e2d3ae611d4de647954).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] asfgit closed pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #31450:
URL: https://github.com/apache/spark/pull/31450
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-774427853
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39535/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-774353694
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134943/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-772254067
**[Test build #134812 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134812/testReport)** for PR 31450 at commit [`5560ec6`](https://github.com/apache/spark/commit/5560ec6b32f329ed6f9ad614df8f8d5c06ab33a7).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-778150043
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39704/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-772255241
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134812/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-772254067
**[Test build #134812 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134812/testReport)** for PR 31450 at commit [`5560ec6`](https://github.com/apache/spark/commit/5560ec6b32f329ed6f9ad614df8f8d5c06ab33a7).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-774440623
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39535/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] attilapiros commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
attilapiros commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-774276366
Locally the "Test decommissioning with dynamic allocation & shuffle cleanups" is passing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] holdenk commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
holdenk commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-780967400
Merged to the current dev branch. WDYT about back porting this to 3.1 @HyukjinKwon ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-778198992
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135123/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-772255241
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134812/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-774318333
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39526/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-778170405
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39704/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-774336116
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39526/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-778190442
**[Test build #135123 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135123/testReport)** for PR 31450 at commit [`7eb3df5`](https://github.com/apache/spark/commit/7eb3df53aae53cd8c28c69d9bf5bff40abf64e70).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-778170405
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39704/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] holdenk commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
holdenk commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-776432824
Wonderful, thanks for picking this up :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-774295061
**[Test build #134943 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134943/testReport)** for PR 31450 at commit [`d37289d`](https://github.com/apache/spark/commit/d37289da58c289e3327c5e2d3ae611d4de647954).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-780942849
Thanks @attilapiros for fixing this. cc @Ngone51 and @jiangxb1987 FYI since you guys are interested in this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-778137110
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39704/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] attilapiros edited a comment on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
attilapiros edited a comment on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-776439581
> Wonderful,...
Especially after it's been spiced up with my painting skills :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-774336116
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39526/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] attilapiros commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
attilapiros commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-776439581
> Wonderful, thanks for picking this up :)
Especially after it's been spiced up with my painting skills
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-774424075
**[Test build #134952 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134952/testReport)** for PR 31450 at commit [`c03258e`](https://github.com/apache/spark/commit/c03258ebef8fdd1c3d83c1fe5b77732a2069aa53).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-776451201
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] attilapiros commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
attilapiros commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-776422517
cc. @holdenk
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-774295061
**[Test build #134943 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134943/testReport)** for PR 31450 at commit [`d37289d`](https://github.com/apache/spark/commit/d37289da58c289e3327c5e2d3ae611d4de647954).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-778123122
**[Test build #135123 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135123/testReport)** for PR 31450 at commit [`7eb3df5`](https://github.com/apache/spark/commit/7eb3df53aae53cd8c28c69d9bf5bff40abf64e70).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-776451201
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] holdenk commented on pull request #31450: [SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
holdenk commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-780869480
LGTM. I'll merge this shortly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31450:
URL: https://github.com/apache/spark/pull/31450#issuecomment-774440623
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39535/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org