[jira] [Updated] (SPARK-2269) Clean up and add unit tests for resourceOffers in MesosSchedulerBackend

["Andrew Or (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-2269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrew Or updated SPARK-2269:
-----------------------------
    Affects Version/s: 1.1.0

> Clean up and add unit tests for resourceOffers in MesosSchedulerBackend
> -----------------------------------------------------------------------
>
>                 Key: SPARK-2269
>                 URL: https://issues.apache.org/jira/browse/SPARK-2269
>             Project: Spark
>          Issue Type: Bug
>          Components: Mesos
>    Affects Versions: 1.1.0
>            Reporter: Patrick Wendell
>            Assignee: Tim Chen
>
> This function could be simplified a bit. We could re-write it without offerableIndices or creating the mesosTasks array as large as the offer list. There is a lot of logic around making sure you get the correct index into mesosTasks and offers, really we should just build mesosTasks directly from the offers we get back. To associate the tasks we are launching with the offers we can just create a hashMap from the slaveId to the original offer.
> The basic logic of the function is that you take the mesos offers, convert them to spark offers, then convert the results back.
> One reason I think it might be designed as it is now is to deal with the case where Mesos gives multiple offers for a single slave. I checked directly with the Mesos team and they said this won't ever happen, you'll get at most one offer per mesos slave within a set of offers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org