[jira] [Created] (SPARK-6584) Provide ExecutorPrefixTaskLocation to support the rdd which can be aware of partition's executor location.

["SaintBacchus (JIRA)" <ji...@apache.org>]

SaintBacchus created SPARK-6584:
-----------------------------------

             Summary: Provide ExecutorPrefixTaskLocation to support the rdd which can be aware of partition's executor  location.
                 Key: SPARK-6584
                 URL: https://issues.apache.org/jira/browse/SPARK-6584
             Project: Spark
          Issue Type: Sub-task
    Affects Versions: 1.4.0
            Reporter: SaintBacchus


The function *RDD.getPreferredLocations* can only be set the host awareness prefer locations.
If some *RDD* wants to be scheduled by executor(such as BlockRDD), spark can do nothing for this.
So  I want to provide *ExecutorPrefixTaskLocation* to support the rdd which can be aware of partition's executor location. This mechanism can avoid data transfor in the case of many executor in the same host.
I think it's very useful especially for *SparkStreaming* since the *Receriver* save data into the *BlockManger* and then become a BlockRDD



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org