[jira] [Resolved] (SPARK-14628) Remove all the Options in TaskMetrics

["Reynold Xin (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-14628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Reynold Xin resolved SPARK-14628.
---------------------------------
       Resolution: Fixed
         Assignee: Wenchen Fan
    Fix Version/s: 2.0.0

> Remove all the Options in TaskMetrics
> -------------------------------------
>
>                 Key: SPARK-14628
>                 URL: https://issues.apache.org/jira/browse/SPARK-14628
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Spark Core
>            Reporter: Reynold Xin
>            Assignee: Wenchen Fan
>             Fix For: 2.0.0
>
>
> Part of the reason why TaskMetrics and its callers are complicated are due to the optional metrics we collect, including input, output, shuffle read, and shuffle write. Given their values are zero, I think we can always track them. It is usually very obvious whether a task is supposed to read any data or not. By always tracking them, we can remove a lot of map, foreach, flatMap, getOrElse(0L) calls throughout Spark.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org