You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by sa...@apache.org on 2018/02/21 01:54:12 UTC
spark git commit: [SPARK-23470][UI] Use first attempt of last stage
to define job description.
Repository: spark
Updated Branches:
refs/heads/master 3e48f3b9e -> 2ba77ed9e
[SPARK-23470][UI] Use first attempt of last stage to define job description.
This is much faster than finding out what the last attempt is, and the
data should be the same.
There's room for improvement in this page (like only loading data for
the jobs being shown, instead of loading all available jobs and sorting
them), but this should bring performance on par with the 2.2 version.
Author: Marcelo Vanzin <va...@cloudera.com>
Closes #20644 from vanzin/SPARK-23470.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2ba77ed9
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/2ba77ed9
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/2ba77ed9
Branch: refs/heads/master
Commit: 2ba77ed9e51922303e3c3533e368b95788bd7de5
Parents: 3e48f3b
Author: Marcelo Vanzin <va...@cloudera.com>
Authored: Tue Feb 20 17:54:06 2018 -0800
Committer: Sameer Agarwal <sa...@apache.org>
Committed: Tue Feb 20 17:54:06 2018 -0800
----------------------------------------------------------------------
core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/2ba77ed9/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala b/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
index a9265d4..ac83de1 100644
--- a/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
+++ b/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
@@ -1048,7 +1048,7 @@ private[ui] object ApiHelper {
}
def lastStageNameAndDescription(store: AppStatusStore, job: JobData): (String, String) = {
- val stage = store.asOption(store.lastStageAttempt(job.stageIds.max))
+ val stage = store.asOption(store.stageAttempt(job.stageIds.max, 0))
(stage.map(_.name).getOrElse(""), stage.flatMap(_.description).getOrElse(job.name))
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org