You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Patrick Wendell (JIRA)" <ji...@apache.org> on 2014/09/19 01:42:34 UTC

[jira] [Updated] (SPARK-3574) Shuffle finish time always reported as -1

     [ https://issues.apache.org/jira/browse/SPARK-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Patrick Wendell updated SPARK-3574:
-----------------------------------
    Component/s: Spark Core

> Shuffle finish time always reported as -1
> -----------------------------------------
>
>                 Key: SPARK-3574
>                 URL: https://issues.apache.org/jira/browse/SPARK-3574
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 1.1.0
>            Reporter: Kay Ousterhout
>
> shuffleFinishTime is always reported as -1.  I think the way we fix this should be to set the shuffleFinishTime in each ShuffleWriteMetrics as the shuffles finish, but when aggregating the metrics, only report shuffleFinishTime as something other than -1 when *all* of the shuffles have completed.
> [~sandyr], it looks like this was introduced in your recent patch to incrementally report metrics.  Any chance you can fix this?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org