You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by Michael Park <mc...@gmail.com> on 2015/09/05 01:19:43 UTC
Re: Review Request 35702: Added /reserve HTTP endpoint to the master.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35702/
-----------------------------------------------------------
(Updated Sept. 4, 2015, 11:19 p.m.)
Review request for mesos, Adam B, Benjamin Hindman, Ben Mahler, Jie Yu, Joris Van Remoortere, and Vinod Kone.
Changes
-------
Rebased. NNFR.
Bugs: MESOS-2600
https://issues.apache.org/jira/browse/MESOS-2600
Repository: mesos
Description
-------
This involved a lot more challenges than I anticipated, I've captured the various approaches and limitations and deal-breakers of those approaches here: [Master Endpoint Implementation Challenges](https://docs.google.com/document/d/1cwVz4aKiCYP9Y4MOwHYZkyaiuEv7fArCye-vPvB2lAI/edit#)
Key points:
* This is a stop-gap solution until we shift the offer creation/management logic from the master to the allocator.
* `updateAvailable` and `updateSlave` are kept separate because
(1) `updateAvailable` is allowed to fail whereas `updateSlave` must not.
(2) `updateAvailable` returns a `Future` whereas `updateSlave` does not.
(3) `updateAvailable` never leaves the allocator in an over-allocated state and must not, whereas `updateSlave` does, and can.
* The algorithm:
* Initially, the master pessimistically assume that what seems like "available" resources will be gone.
This is due to the race between the allocator scheduling an `allocate` call to itself vs master's
`allocator->updateAvailable` invocation.
As such, we first try to satisfy the request only with the offered resources.
* We greedily rescind one offer at a time until we've rescinded sufficiently many offers.
IMPORTANT: We perform `recoverResources(..., Filters())` which has a default `refuse_sec` of 5 seconds,
rather than `recoverResources(..., None())` so that we can virtually always win the race against `allocate`.
In the rare case that we do lose, no disaster occurs. We simply fail to satisfy the request.
* If we still don't have enough resources after resciding all offers, be semi-optimistic and forward the
request to the allocator since there may be available resources to satisfy the request.
* If the allocator returns a failure, report the error to the user with `Conflict`.
This approach is clearly not ideal, since we would prefer to rescind as little offers as possible.
Diffs (updated)
-----
src/master/http.cpp 94e97a2898106579434e8cdec04b7b0e130a810e
src/master/master.hpp e1331851c19e3372a4a525dcfd7ba2a01c3e97a6
src/master/master.cpp 5589eca4317b597de509f3387cfc349083b361ac
src/master/validation.hpp 43b8d84556e7f0a891dddf6185bbce7ca50b360a
src/master/validation.cpp ffb7bf07b8a40d6e14f922eabcf46045462498b5
Diff: https://reviews.apache.org/r/35702/diff/
Testing
-------
`make check`
Thanks,
Michael Park
Re: Review Request 35702: Added /reserve HTTP endpoint to the master.
Posted by Guangya Liu <gy...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35702/#review97881
-----------------------------------------------------------
Ship it!
Ship It!
- Guangya Liu
On 九月 4, 2015, 11:19 p.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35702/
> -----------------------------------------------------------
>
> (Updated 九月 4, 2015, 11:19 p.m.)
>
>
> Review request for mesos, Adam B, Benjamin Hindman, Ben Mahler, Jie Yu, Joris Van Remoortere, and Vinod Kone.
>
>
> Bugs: MESOS-2600
> https://issues.apache.org/jira/browse/MESOS-2600
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This involved a lot more challenges than I anticipated, I've captured the various approaches and limitations and deal-breakers of those approaches here: [Master Endpoint Implementation Challenges](https://docs.google.com/document/d/1cwVz4aKiCYP9Y4MOwHYZkyaiuEv7fArCye-vPvB2lAI/edit#)
>
> Key points:
>
> * This is a stop-gap solution until we shift the offer creation/management logic from the master to the allocator.
> * `updateAvailable` and `updateSlave` are kept separate because
> (1) `updateAvailable` is allowed to fail whereas `updateSlave` must not.
> (2) `updateAvailable` returns a `Future` whereas `updateSlave` does not.
> (3) `updateAvailable` never leaves the allocator in an over-allocated state and must not, whereas `updateSlave` does, and can.
> * The algorithm:
> * Initially, the master pessimistically assume that what seems like "available" resources will be gone.
> This is due to the race between the allocator scheduling an `allocate` call to itself vs master's
> `allocator->updateAvailable` invocation.
> As such, we first try to satisfy the request only with the offered resources.
> * We greedily rescind one offer at a time until we've rescinded sufficiently many offers.
> IMPORTANT: We perform `recoverResources(..., Filters())` which has a default `refuse_sec` of 5 seconds,
> rather than `recoverResources(..., None())` so that we can virtually always win the race against `allocate`.
> In the rare case that we do lose, no disaster occurs. We simply fail to satisfy the request.
> * If we still don't have enough resources after resciding all offers, be semi-optimistic and forward the
> request to the allocator since there may be available resources to satisfy the request.
> * If the allocator returns a failure, report the error to the user with `Conflict`.
>
> This approach is clearly not ideal, since we would prefer to rescind as little offers as possible.
>
>
> Diffs
> -----
>
> src/master/http.cpp 94e97a2898106579434e8cdec04b7b0e130a810e
> src/master/master.hpp e1331851c19e3372a4a525dcfd7ba2a01c3e97a6
> src/master/master.cpp 5589eca4317b597de509f3387cfc349083b361ac
> src/master/validation.hpp 43b8d84556e7f0a891dddf6185bbce7ca50b360a
> src/master/validation.cpp ffb7bf07b8a40d6e14f922eabcf46045462498b5
>
> Diff: https://reviews.apache.org/r/35702/diff/
>
>
> Testing
> -------
>
> `make check`
>
>
> Thanks,
>
> Michael Park
>
>