You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by zsxwing <gi...@git.apache.org> on 2018/02/21 22:42:06 UTC
[GitHub] spark pull request #20656: [SPARK-23475][WebUI]Skipped stages should be evic...
GitHub user zsxwing opened a pull request:
https://github.com/apache/spark/pull/20656
[SPARK-23475][WebUI]Skipped stages should be evicted before completed stages
## What changes were proposed in this pull request?
The root cause of missing completed stages is because `cleanupStages` will never remove skipped stages.
This PR changes the logic to always remove skipped stage first. This is safe since the job itself contains enough information to render skipped stages in the UI.
## How was this patch tested?
The new unit tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zsxwing/spark SPARK-23475
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20656.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20656
----
commit 82e7dda1042438392bdc9a294e8a01baebd1e24c
Author: Shixiong Zhu <zs...@...>
Date: 2018-02-21T22:39:38Z
skipped stages should be evicted before completed stages
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20656: [SPARK-23475][WebUI]Skipped stages should be evicted bef...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20656
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20656: [SPARK-23475][WebUI]Skipped stages should be evicted bef...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20656
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20656: [SPARK-23475][WebUI]Skipped stages should be evicted bef...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20656
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87602/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20656: [SPARK-23475][WebUI]Skipped stages should be evicted bef...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20656
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/998/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20656: [SPARK-23475][WebUI]Skipped stages should be evicted bef...
Posted by zsxwing <gi...@git.apache.org>.
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/20656
cc @vanzin
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #20656: [SPARK-23475][WebUI]Skipped stages should be evic...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/20656
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20656: [SPARK-23475][WebUI]Skipped stages should be evicted bef...
Posted by vanzin <gi...@git.apache.org>.
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/20656
LGTM pending tests.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20656: [SPARK-23475][WebUI]Skipped stages should be evicted bef...
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20656
Thanks! Merged to master/2.3
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20656: [SPARK-23475][WebUI]Skipped stages should be evicted bef...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20656
**[Test build #87602 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87602/testReport)** for PR 20656 at commit [`82e7dda`](https://github.com/apache/spark/commit/82e7dda1042438392bdc9a294e8a01baebd1e24c).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #20656: [SPARK-23475][WebUI]Skipped stages should be evicted bef...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20656
**[Test build #87602 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87602/testReport)** for PR 20656 at commit [`82e7dda`](https://github.com/apache/spark/commit/82e7dda1042438392bdc9a294e8a01baebd1e24c).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org