You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by ka...@apache.org on 2015/05/29 07:09:54 UTC
spark git commit: [SPARK-7932] Fix misleading scheduler delay
visualization
Repository: spark
Updated Branches:
refs/heads/master 834e69952 -> 04ddcd4db
[SPARK-7932] Fix misleading scheduler delay visualization
The existing code rounds down to the nearest percent when computing the proportion
of a task's time that was spent on each phase of execution, and then computes
the scheduler delay proportion as 100 - sum(all other proportions). As a result,
a few extra percent can end up in the scheduler delay. This commit eliminates
the rounding so that the time visualizations correspond properly to the real times.
sarutak If you could take a look at this, that would be great! Not sure if there's a good
reason to round here that I missed.
cc shivaram
Author: Kay Ousterhout <ka...@gmail.com>
Closes #6484 from kayousterhout/SPARK-7932 and squashes the following commits:
1723cc4 [Kay Ousterhout] [SPARK-7932] Fix misleading scheduler delay visualization
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/04ddcd4d
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/04ddcd4d
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/04ddcd4d
Branch: refs/heads/master
Commit: 04ddcd4db7801abefa9c9effe5d88413b29d713b
Parents: 834e699
Author: Kay Ousterhout <ka...@gmail.com>
Authored: Thu May 28 22:09:49 2015 -0700
Committer: Kay Ousterhout <ka...@gmail.com>
Committed: Thu May 28 22:09:49 2015 -0700
----------------------------------------------------------------------
core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/04ddcd4d/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala b/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
index 31e2e7f..b83a49f 100644
--- a/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
+++ b/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
@@ -527,7 +527,7 @@ private[ui] class StagePage(parent: StagesTab) extends WebUIPage("stage") {
minLaunchTime = launchTime.min(minLaunchTime)
maxFinishTime = finishTime.max(maxFinishTime)
- def toProportion(time: Long) = (time.toDouble / totalExecutionTime * 100).toLong
+ def toProportion(time: Long) = time.toDouble / totalExecutionTime * 100
val metricsOpt = taskUIData.taskMetrics
val shuffleReadTime =
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org