You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by an...@apache.org on 2015/08/12 08:23:21 UTC
spark git commit: [SPARK-9829] [WEBUI] Display the update value for
peak execution memory
Repository: spark
Updated Branches:
refs/heads/master a807fcbe5 -> 4e3f4b934
[SPARK-9829] [WEBUI] Display the update value for peak execution memory
The peak execution memory is not correct because it shows the sum of finished tasks' values when a task finishes.
This PR fixes it by using the update value rather than the accumulator value.
Author: zsxwing <zs...@gmail.com>
Closes #8121 from zsxwing/SPARK-9829.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/4e3f4b93
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/4e3f4b93
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/4e3f4b93
Branch: refs/heads/master
Commit: 4e3f4b934f74e8c7c06f4940d6381343f9fd4918
Parents: a807fcb
Author: zsxwing <zs...@gmail.com>
Authored: Tue Aug 11 23:23:17 2015 -0700
Committer: Andrew Or <an...@databricks.com>
Committed: Tue Aug 11 23:23:17 2015 -0700
----------------------------------------------------------------------
core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/4e3f4b93/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala b/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
index 0c94204..fb4556b 100644
--- a/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
+++ b/core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
@@ -860,7 +860,7 @@ private[ui] class TaskDataSource(
}
val peakExecutionMemoryUsed = taskInternalAccumulables
.find { acc => acc.name == InternalAccumulator.PEAK_EXECUTION_MEMORY }
- .map { acc => acc.value.toLong }
+ .map { acc => acc.update.getOrElse("0").toLong }
.getOrElse(0L)
val maybeInput = metrics.flatMap(_.inputMetrics)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org