You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Ben Mahler <be...@gmail.com> on 2014/06/05 19:35:25 UTC
Review Request 22268: Added "implicit" reconciliation.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22268/
-----------------------------------------------------------
Review request for mesos, Benjamin Hindman, Niklas Nielsen, and Vinod Kone.
Bugs: MESOS-1379
https://issues.apache.org/jira/browse/MESOS-1379
Repository: mesos-git
Description
-------
Currently frameworks must ask to reconcile a set of tasks explicitly, this means that the tasks must be known to the framework.
In some cases, the framework may lose the information about a particular task (bugs, data loss, etc). When this occurs, we do not provide a mechanism for the state to be reconciled.
That is, the non-terminal tasks known to the Master / Slave that are unknown to the framework will remain running. Status updates from these tasks are the only way that frameworks will be informed of these tasks. This patch introduces the notion of an "implicit" reconciliation request where the framework does not ask about any explicit tasks. In this case, the master replies with all tasks currently known.
Diffs
-----
include/mesos/scheduler.hpp a17db5919f8c434a918bf5f782ef5e02c5a9be9c
include/mesos/scheduler/scheduler.proto 4deda5598d9c242f97659d06309294f24f7cfd8a
src/master/master.cpp 91dc1fd85ed83913ab5c7542a96a9d71160b4f94
src/messages/messages.proto 6f6e570b1018d8c54d4b8d811470086832792ec4
src/tests/reconciliation_tests.cpp 677d18e1a4e1683809d21eb96bab48d6e3868207
Diff: https://reviews.apache.org/r/22268/diff/
Testing
-------
Added two tests for implicit reconciliation.
Thanks,
Ben Mahler
Re: Review Request 22268: Added "implicit" reconciliation.
Posted by Ben Mahler <be...@gmail.com>.
> On June 6, 2014, 8:02 p.m., Vinod Kone wrote:
> >
I'll also update the documentation for the Java API, filed MESOS-1461 to get reconciliation added to the Python API.
> On June 6, 2014, 8:02 p.m., Vinod Kone wrote:
> > include/mesos/scheduler.hpp, line 314
> > <https://reviews.apache.org/r/22268/diff/1/?file=604057#file604057line314>
> >
> > s/task/non-terminal task/ ?
> >
> > or is the plan to send all unacknowledged tasks in the future once acks go through the master?
Good point, I updated the documentation to imply that the framework should only be asking about non-terminal tasks.
- Ben
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22268/#review44956
-----------------------------------------------------------
On June 5, 2014, 5:35 p.m., Ben Mahler wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22268/
> -----------------------------------------------------------
>
> (Updated June 5, 2014, 5:35 p.m.)
>
>
> Review request for mesos, Benjamin Hindman, Niklas Nielsen, and Vinod Kone.
>
>
> Bugs: MESOS-1379
> https://issues.apache.org/jira/browse/MESOS-1379
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Currently frameworks must ask to reconcile a set of tasks explicitly, this means that the tasks must be known to the framework.
>
> In some cases, the framework may lose the information about a particular task (bugs, data loss, etc). When this occurs, we do not provide a mechanism for the state to be reconciled.
>
> That is, the non-terminal tasks known to the Master / Slave that are unknown to the framework will remain running. Status updates from these tasks are the only way that frameworks will be informed of these tasks. This patch introduces the notion of an "implicit" reconciliation request where the framework does not ask about any explicit tasks. In this case, the master replies with all tasks currently known.
>
>
> Diffs
> -----
>
> include/mesos/scheduler.hpp a17db5919f8c434a918bf5f782ef5e02c5a9be9c
> include/mesos/scheduler/scheduler.proto 4deda5598d9c242f97659d06309294f24f7cfd8a
> src/master/master.cpp 91dc1fd85ed83913ab5c7542a96a9d71160b4f94
> src/messages/messages.proto 6f6e570b1018d8c54d4b8d811470086832792ec4
> src/tests/reconciliation_tests.cpp 677d18e1a4e1683809d21eb96bab48d6e3868207
>
> Diff: https://reviews.apache.org/r/22268/diff/
>
>
> Testing
> -------
>
> Added two tests for implicit reconciliation.
>
>
> Thanks,
>
> Ben Mahler
>
>
Re: Review Request 22268: Added "implicit" reconciliation.
Posted by Vinod Kone <vi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22268/#review44956
-----------------------------------------------------------
Ship it!
include/mesos/scheduler.hpp
<https://reviews.apache.org/r/22268/#comment79553>
s/task/non-terminal task/ ?
or is the plan to send all unacknowledged tasks in the future once acks go through the master?
include/mesos/scheduler/scheduler.proto
<https://reviews.apache.org/r/22268/#comment79554>
s/task/non-terminal task/
src/messages/messages.proto
<https://reviews.apache.org/r/22268/#comment79552>
s/task/non-terminal task/
src/tests/reconciliation_tests.cpp
<https://reviews.apache.org/r/22268/#comment79556>
s/ImplicitNonTerminal/ImplicitNonTerminalTask/ ? :)
src/tests/reconciliation_tests.cpp
<https://reviews.apache.org/r/22268/#comment79558>
EXPECT_TRUE(update.get().has_slave_id());
src/tests/reconciliation_tests.cpp
<https://reviews.apache.org/r/22268/#comment79557>
s/ImplicitTerminal/ImplicitTerminalTask/
src/tests/reconciliation_tests.cpp
<https://reviews.apache.org/r/22268/#comment79559>
EXPECT_TRUE(update.get().has_slave_id())
- Vinod Kone
On June 5, 2014, 5:35 p.m., Ben Mahler wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22268/
> -----------------------------------------------------------
>
> (Updated June 5, 2014, 5:35 p.m.)
>
>
> Review request for mesos, Benjamin Hindman, Niklas Nielsen, and Vinod Kone.
>
>
> Bugs: MESOS-1379
> https://issues.apache.org/jira/browse/MESOS-1379
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Currently frameworks must ask to reconcile a set of tasks explicitly, this means that the tasks must be known to the framework.
>
> In some cases, the framework may lose the information about a particular task (bugs, data loss, etc). When this occurs, we do not provide a mechanism for the state to be reconciled.
>
> That is, the non-terminal tasks known to the Master / Slave that are unknown to the framework will remain running. Status updates from these tasks are the only way that frameworks will be informed of these tasks. This patch introduces the notion of an "implicit" reconciliation request where the framework does not ask about any explicit tasks. In this case, the master replies with all tasks currently known.
>
>
> Diffs
> -----
>
> include/mesos/scheduler.hpp a17db5919f8c434a918bf5f782ef5e02c5a9be9c
> include/mesos/scheduler/scheduler.proto 4deda5598d9c242f97659d06309294f24f7cfd8a
> src/master/master.cpp 91dc1fd85ed83913ab5c7542a96a9d71160b4f94
> src/messages/messages.proto 6f6e570b1018d8c54d4b8d811470086832792ec4
> src/tests/reconciliation_tests.cpp 677d18e1a4e1683809d21eb96bab48d6e3868207
>
> Diff: https://reviews.apache.org/r/22268/diff/
>
>
> Testing
> -------
>
> Added two tests for implicit reconciliation.
>
>
> Thanks,
>
> Ben Mahler
>
>
Re: Review Request 22268: Added "implicit" reconciliation.
Posted by Ben Mahler <be...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22268/
-----------------------------------------------------------
(Updated June 6, 2014, 10:27 p.m.)
Review request for mesos, Benjamin Hindman and Vinod Kone.
Changes
-------
Vinod's review and Java documentation updates.
Bugs: MESOS-1379
https://issues.apache.org/jira/browse/MESOS-1379
Repository: mesos-git
Description
-------
Currently frameworks must ask to reconcile a set of tasks explicitly, this means that the tasks must be known to the framework.
In some cases, the framework may lose the information about a particular task (bugs, data loss, etc). When this occurs, we do not provide a mechanism for the state to be reconciled.
That is, the non-terminal tasks known to the Master / Slave that are unknown to the framework will remain running. Status updates from these tasks are the only way that frameworks will be informed of these tasks. This patch introduces the notion of an "implicit" reconciliation request where the framework does not ask about any explicit tasks. In this case, the master replies with all tasks currently known.
Diffs (updated)
-----
include/mesos/scheduler.hpp a17db5919f8c434a918bf5f782ef5e02c5a9be9c
include/mesos/scheduler/scheduler.proto 4deda5598d9c242f97659d06309294f24f7cfd8a
src/java/src/org/apache/mesos/SchedulerDriver.java ae2d915d9afb3d35d42a24b56115a386ea960154
src/master/master.cpp 89f426c14de365369b900864f1983b1f9260953f
src/messages/messages.proto 6f6e570b1018d8c54d4b8d811470086832792ec4
src/python/src/mesos.py a2fc0e92a021f96214bfcfe9cf9703ed6290803b
src/tests/reconciliation_tests.cpp 677d18e1a4e1683809d21eb96bab48d6e3868207
Diff: https://reviews.apache.org/r/22268/diff/
Testing
-------
Added two tests for implicit reconciliation.
Thanks,
Ben Mahler
Re: Review Request 22268: Added "implicit" reconciliation.
Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22268/#review44958
-----------------------------------------------------------
Patch looks great!
Reviews applied: [22267, 22268]
All tests passed.
- Mesos ReviewBot
On June 5, 2014, 5:35 p.m., Ben Mahler wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22268/
> -----------------------------------------------------------
>
> (Updated June 5, 2014, 5:35 p.m.)
>
>
> Review request for mesos, Benjamin Hindman, Niklas Nielsen, and Vinod Kone.
>
>
> Bugs: MESOS-1379
> https://issues.apache.org/jira/browse/MESOS-1379
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> Currently frameworks must ask to reconcile a set of tasks explicitly, this means that the tasks must be known to the framework.
>
> In some cases, the framework may lose the information about a particular task (bugs, data loss, etc). When this occurs, we do not provide a mechanism for the state to be reconciled.
>
> That is, the non-terminal tasks known to the Master / Slave that are unknown to the framework will remain running. Status updates from these tasks are the only way that frameworks will be informed of these tasks. This patch introduces the notion of an "implicit" reconciliation request where the framework does not ask about any explicit tasks. In this case, the master replies with all tasks currently known.
>
>
> Diffs
> -----
>
> include/mesos/scheduler.hpp a17db5919f8c434a918bf5f782ef5e02c5a9be9c
> include/mesos/scheduler/scheduler.proto 4deda5598d9c242f97659d06309294f24f7cfd8a
> src/master/master.cpp 91dc1fd85ed83913ab5c7542a96a9d71160b4f94
> src/messages/messages.proto 6f6e570b1018d8c54d4b8d811470086832792ec4
> src/tests/reconciliation_tests.cpp 677d18e1a4e1683809d21eb96bab48d6e3868207
>
> Diff: https://reviews.apache.org/r/22268/diff/
>
>
> Testing
> -------
>
> Added two tests for implicit reconciliation.
>
>
> Thanks,
>
> Ben Mahler
>
>