You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Sandy Ryza (JIRA)" <ji...@apache.org> on 2014/09/21 19:42:34 UTC

[jira] [Commented] (SPARK-3577) Add task metric to report spill time

    [ https://issues.apache.org/jira/browse/SPARK-3577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14142524#comment-14142524 ] 

Sandy Ryza commented on SPARK-3577:
-----------------------------------

No problem.  Yeah, I agree that a spill time metric would be useful.

> Add task metric to report spill time
> ------------------------------------
>
>                 Key: SPARK-3577
>                 URL: https://issues.apache.org/jira/browse/SPARK-3577
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 1.1.0
>            Reporter: Kay Ousterhout
>            Priority: Minor
>
> The ExternalSorter passes its own ShuffleWriteMetrics into ExternalSorter.  The write time recorded in those metrics is never used.  We should probably add task metrics to report this spill time, since for shuffles, this would have previously been reported as part of shuffle write time (with the original hash-based sorter).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org