You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (Jira)" <ji...@apache.org> on 2020/01/12 04:10:00 UTC
[jira] [Resolved] (SPARK-30458) The Executor Computing Time in Time
Line of Stage Page is Wrong
[ https://issues.apache.org/jira/browse/SPARK-30458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun resolved SPARK-30458.
-----------------------------------
Fix Version/s: 3.0.0
2.4.5
Resolution: Fixed
Issue resolved by pull request 27135
[https://github.com/apache/spark/pull/27135]
> The Executor Computing Time in Time Line of Stage Page is Wrong
> ---------------------------------------------------------------
>
> Key: SPARK-30458
> URL: https://issues.apache.org/jira/browse/SPARK-30458
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 2.0.2, 2.1.3, 2.2.3, 2.3.4, 2.4.4, 3.0.0
> Reporter: SongXun
> Assignee: SongXun
> Priority: Minor
> Fix For: 2.4.5, 3.0.0
>
> Attachments: Snipaste_2020-01-08_19-04-33.png
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> The Executor Computing Time in Time Line of Stage Page is Wrong. It includes the Scheduler Delay Time, while the Proportion excludes the Scheduler Delay
> val executorOverhead = serializationTime + deserializationTime
> val executorRunTime = if (taskInfo.duration.isDefined) {
> totalExecutionTime - executorOverhead - gettingResultTime
> } else {
> metricsOpt.map(_.executorRunTime).getOrElse(
> totalExecutionTime - executorOverhead - gettingResultTime)
> }
> val executorComputingTime = executorRunTime - shuffleReadTime - shuffleWriteTime
> val executorComputingTimeProportion =
> math.max(100 - schedulerDelayProportion - shuffleReadTimeProportion -
> shuffleWriteTimeProportion - serializationTimeProportion -
> deserializationTimeProportion - gettingResultTimeProportion, 0)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org