You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Kostas Sakellis (JIRA)" <ji...@apache.org> on 2015/01/13 20:59:35 UTC

[jira] [Created] (SPARK-5225) Support coalesed Input Metrics from different sources

Kostas Sakellis created SPARK-5225:
--------------------------------------

             Summary: Support coalesed Input Metrics from different sources
                 Key: SPARK-5225
                 URL: https://issues.apache.org/jira/browse/SPARK-5225
             Project: Spark
          Issue Type: Bug
            Reporter: Kostas Sakellis


Currently, If task reads data from more than one block and it is from different read methods we ignore the second read method bytes. For example:
                 CoalescedRDD
                            | 
                        Task1 
                  /         |            \
                /           |              \           
      hadoop       hadoop      cached

if Task1 starts reading from the hadoop blocks first, then the input metrics for Task1 will only contain input metrics from the hadoop blocks and ignre the input metrics from cached blocks. We need to change the way we collect input metrics so that it is not a single value but rather a collection of input metrics for a task. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org