[jira] [Resolved] (SPARK-6920) Be more explicit about references to "executor" and "task" in Spark on Mesos

["Hyukjin Kwon (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon resolved SPARK-6920.
---------------------------------
    Resolution: Incomplete

> Be more explicit about references to "executor" and "task" in Spark on Mesos
> ----------------------------------------------------------------------------
>
>                 Key: SPARK-6920
>                 URL: https://issues.apache.org/jira/browse/SPARK-6920
>             Project: Spark
>          Issue Type: Bug
>          Components: Documentation, Mesos
>    Affects Versions: 1.0.0
>            Reporter: Andrew Or
>            Priority: Major
>              Labels: bulk-closed
>
> In both Spark and Mesos, the terms "executor" and "task" mean different things depending on the context. In the past, this has caused a great deal of confusion, to both the user and the developer of Spark.
> The consequences are real. For instance, the fine-grained `MesosSchedulerBackend` code incorrectly uses `spark.tasks.cpus` as the number of cores to grant the Mesos executor [0]. This is a result of conflating the Mesos executor with the Spark executor, and the confusion is reflected in the comment, where the "executor" here refers to the Mesos executor, not the Spark executor.
> {code}
> // If the executor doesn't exist yet, subtract CPU for executor
> {code}
> We really need to be explicit about our references to these terms such that this part of the Spark code can progress as quickly as other modules.
> [0] https://github.com/apache/spark/blob/6de282e2de3cb69f9b746d03fde581429248824a/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerBackend.scala#L238



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org