[jira] [Resolved] (SPARK-1524) TaskSetManager'd better not schedule tasks which has no preferred executorId using PROCESS_LOCAL in the first search process

["Sean Owen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-1524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Owen resolved SPARK-1524.
------------------------------
    Resolution: Won't Fix

> TaskSetManager'd better not schedule tasks which has no preferred executorId using PROCESS_LOCAL in the first search process
> ----------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-1524
>                 URL: https://issues.apache.org/jira/browse/SPARK-1524
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>            Reporter: YanTang Zhai
>            Priority: Minor
>
> ShuffleMapTask is constructed with TaskLocation which has only host not (host, executorID) pair in DAGScheduler.
> When TaskSetManager schedules ShuffleMapTask which has no preferred executorId using specific execId host and PROCESS_LOCAL locality level, no tasks match the given locality constraint in the first search process.
> We also find that the host used by Scheduler is hostname while the host used by TaskLocation is IP in our cluster. The tow hosts do not match, that makes pendingTasksForHost HashMap empty and the finding task process against our expectation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org