You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Zhihui (JIRA)" <ji...@apache.org> on 2014/06/19 10:34:24 UTC

[jira] [Commented] (SPARK-2193) Improve tasks‘ preferred locality by sorting tasks partial ordering

    [ https://issues.apache.org/jira/browse/SPARK-2193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14037121#comment-14037121 ] 

Zhihui commented on SPARK-2193:
-------------------------------

PR 1131 https://github.com/apache/spark/pull/1131

> Improve tasks‘ preferred locality by sorting tasks partial ordering
> -------------------------------------------------------------------
>
>                 Key: SPARK-2193
>                 URL: https://issues.apache.org/jira/browse/SPARK-2193
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 1.1.0
>            Reporter: Zhihui
>         Attachments: Improve Tasks Preferred Locality.pptx
>
>
> Now, the last executor(s) maybe not get it’s preferred task(s), although these tasks have build in pendingTasksForHosts map. Because executers pick up tasks sequential, their preferred task(s) maybe picked up by other executors.
> This appearance can be eliminated by sorting tasks partial ordering. Executor pick up task by host’s order of task’s preferredLocation, that mean, executor firstly pick up all tasks which task.preferredLocations.1 = executor.hostName, then secondly…  



--
This message was sent by Atlassian JIRA
(v6.2#6252)