You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by Klaus Ma <kl...@gmail.com> on 2016/01/16 06:46:24 UTC
Re: Review Request 41305: MESOS-1718: move getExecutorInfo from slave
to master.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41305/
-----------------------------------------------------------
(Updated Jan. 16, 2016, 1:46 p.m.)
Review request for mesos, Ben Mahler, Guangya Liu, Artem Harutyunyan, Ian Downes, Joris Van Remoortere, Joseph Wu, and Vinod Kone.
Changes
-------
Add period to Summary.
Summary (updated)
-----------------
MESOS-1718: move getExecutorInfo from slave to master.
Bugs: MESOS-1718
https://issues.apache.org/jira/browse/MESOS-1718
Repository: mesos
Description
-------
MESOS-1718: move getExecutorInfo from slave to master
Diffs
-----
src/master/constants.hpp ebab341e58035d4b579828add752c1ee37efeb95
src/master/master.hpp f02d165874fa8023675e545793de699aeecae29b
src/master/master.cpp 863a11c82d322f56db1ccf25b73a4ffff1304a46
src/slave/constants.hpp bcbb1401aa8f9f04c4f9256bb4f560e18d8994e0
src/slave/slave.hpp b7586ce42bfac9d9885a3eb8d82deb94680c236c
src/slave/slave.cpp 759c8d5b1bfb5ad723aa423e1487998ed62bbc3a
Diff: https://reviews.apache.org/r/41305/diff/
Testing
-------
Thanks,
Klaus Ma
Re: Review Request 41305: MESOS-1718: move getExecutorInfo from slave
to master.
Posted by Klaus Ma <kl...@gmail.com>.
> On Jan. 22, 2016, 5:37 a.m., Joseph Wu wrote:
> > src/master/master.cpp, lines 3770-3779
> > <https://reviews.apache.org/r/41305/diff/3/?file=1200898#file1200898line3770>
> >
> > It should still be invalid to supply both a CommandInfo and ExecutorInfo in the same TaskInfo.
The check was removed at https://reviews.apache.org/r/41306
- Klaus
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41305/#review115703
-----------------------------------------------------------
On Jan. 19, 2016, 10:55 a.m., Klaus Ma wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41305/
> -----------------------------------------------------------
>
> (Updated Jan. 19, 2016, 10:55 a.m.)
>
>
> Review request for mesos, Ben Mahler, Guangya Liu, Artem Harutyunyan, Ian Downes, Joris Van Remoortere, Joseph Wu, and Vinod Kone.
>
>
> Bugs: MESOS-1718
> https://issues.apache.org/jira/browse/MESOS-1718
>
>
> Repository: mesos
>
>
> Description
> -------
>
> MESOS-1718: move getExecutorInfo from slave to master
>
>
> Diffs
> -----
>
> src/master/constants.hpp 2c3299bc1aaa0888f7e47a71965c56ada8ecc21f
> src/master/master.hpp 3a7e18232323a1c051bcc97915484b1195fffe58
> src/master/master.cpp 9ee56277c8a472be9e683d5db505becfb5f7c422
> src/slave/constants.hpp bcbb1401aa8f9f04c4f9256bb4f560e18d8994e0
> src/slave/slave.hpp b7586ce42bfac9d9885a3eb8d82deb94680c236c
> src/slave/slave.cpp bb501810d7bb1261ebbbdd147c49948e5a2f8665
>
> Diff: https://reviews.apache.org/r/41305/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Klaus Ma
>
>
Re: Review Request 41305: MESOS-1718: move getExecutorInfo from slave
to master.
Posted by Klaus Ma <kl...@gmail.com>.
> On Jan. 22, 2016, 5:37 a.m., Joseph Wu wrote:
> > General questions:
> >
> > - Besides removing the possibility for overcommit, does this change give us anything else?
> > - What happens if a master sends this modified `TaskInfo` to an agent that does not have this chain of patches?
> > - Have you considered completely converting a `CommandInfo` to `ExecutorInfo`? If we do that, it may be possible to remove all the other places with special logic for command tasks.
>
> Klaus Ma wrote:
> * It's more for evicting executor by master, there's several EPICs about sharing resources between frameworks; and all of them need the feature to let resources owner get resouces back: MESOS-4303 (reclaim/preemption in Mesos), "Optimsitic Offer Phase 2", "Borrow/Lend policy on Quota" (draft idea in my mind)
>
> * Good point! If did not get required info, e.g. `launch_dir`, we should keep the same behaviour as before: let salve trace the executor info.
>
> * Yes, I'm used to add `taskCommand` into `ExecutorInfo`; but when launch docker executor, it need task info which is different with normal executor. In this patch, I'd like to focus on moving the executor info into master.
@Joseph, any more comments?
- Klaus
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41305/#review115703
-----------------------------------------------------------
On Feb. 18, 2016, 7:45 a.m., Klaus Ma wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41305/
> -----------------------------------------------------------
>
> (Updated Feb. 18, 2016, 7:45 a.m.)
>
>
> Review request for mesos, Ben Mahler and Joseph Wu.
>
>
> Bugs: MESOS-1718
> https://issues.apache.org/jira/browse/MESOS-1718
>
>
> Repository: mesos
>
>
> Description
> -------
>
> MESOS-1718: move getExecutorInfo from slave to master
>
>
> Diffs
> -----
>
> src/master/constants.hpp 2c3299bc1aaa0888f7e47a71965c56ada8ecc21f
> src/master/master.hpp 2f2ad2ada508e1923bf995ab124367a3b082b572
> src/master/master.cpp e1ca81dab85a7ab1391eca0d6bd995548bf79c22
> src/slave/constants.hpp bcbb1401aa8f9f04c4f9256bb4f560e18d8994e0
> src/slave/slave.hpp ced835dec797bcc5640422468487a4289a737e38
> src/slave/slave.cpp f0be0d5bf4b853952caf06e2e262c2903d79ead3
>
> Diff: https://reviews.apache.org/r/41305/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Klaus Ma
>
>
Re: Review Request 41305: MESOS-1718: move getExecutorInfo from slave
to master.
Posted by Klaus Ma <kl...@gmail.com>.
> On Jan. 22, 2016, 5:37 a.m., Joseph Wu wrote:
> > src/master/master.cpp, line 2860
> > <https://reviews.apache.org/r/41305/diff/3/?file=1200898#file1200898line2860>
> >
> > `launcher_dir` is an optional field.
> >
> > Same for the other fields you added (`sandbox_dir`, `switch_user`, `executor_rootfs).
@Joseph, do you mean we need to check whether the value is set? I think it's also related with compatibility with old slave, we should also handle it when old slave did not report `slaveInfo`, right?
- Klaus
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41305/#review115703
-----------------------------------------------------------
On Feb. 12, 2016, 11:02 p.m., Klaus Ma wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41305/
> -----------------------------------------------------------
>
> (Updated Feb. 12, 2016, 11:02 p.m.)
>
>
> Review request for mesos, Ben Mahler and Joseph Wu.
>
>
> Bugs: MESOS-1718
> https://issues.apache.org/jira/browse/MESOS-1718
>
>
> Repository: mesos
>
>
> Description
> -------
>
> MESOS-1718: move getExecutorInfo from slave to master
>
>
> Diffs
> -----
>
> src/master/constants.hpp 2c3299bc1aaa0888f7e47a71965c56ada8ecc21f
> src/master/master.hpp 3a7e18232323a1c051bcc97915484b1195fffe58
> src/master/master.cpp 9ee56277c8a472be9e683d5db505becfb5f7c422
> src/slave/constants.hpp bcbb1401aa8f9f04c4f9256bb4f560e18d8994e0
> src/slave/slave.hpp b7586ce42bfac9d9885a3eb8d82deb94680c236c
> src/slave/slave.cpp bb501810d7bb1261ebbbdd147c49948e5a2f8665
>
> Diff: https://reviews.apache.org/r/41305/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Klaus Ma
>
>
Re: Review Request 41305: MESOS-1718: move getExecutorInfo from slave
to master.
Posted by Klaus Ma <kl...@gmail.com>.
> On Jan. 22, 2016, 5:37 a.m., Joseph Wu wrote:
> > General questions:
> >
> > - Besides removing the possibility for overcommit, does this change give us anything else?
> > - What happens if a master sends this modified `TaskInfo` to an agent that does not have this chain of patches?
> > - Have you considered completely converting a `CommandInfo` to `ExecutorInfo`? If we do that, it may be possible to remove all the other places with special logic for command tasks.
* It's more for evicting executor by master, there's several EPICs about sharing resources between frameworks; and all of them need the feature to let resources owner get resouces back: MESOS-4303 (reclaim/preemption in Mesos), "Optimsitic Offer Phase 2", "Borrow/Lend policy on Quota" (draft idea in my mind)
* Good point! If did not get required info, e.g. `launch_dir`, we should keep the same behaviour as before: let salve trace the executor info.
* Yes, I'm used to add `taskCommand` into `ExecutorInfo`; but when launch docker executor, it need task info which is different with normal executor. In this patch, I'd like to focus on moving the executor info into master.
- Klaus
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41305/#review115703
-----------------------------------------------------------
On Jan. 19, 2016, 10:55 a.m., Klaus Ma wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41305/
> -----------------------------------------------------------
>
> (Updated Jan. 19, 2016, 10:55 a.m.)
>
>
> Review request for mesos, Ben Mahler, Guangya Liu, Artem Harutyunyan, Ian Downes, Joris Van Remoortere, Joseph Wu, and Vinod Kone.
>
>
> Bugs: MESOS-1718
> https://issues.apache.org/jira/browse/MESOS-1718
>
>
> Repository: mesos
>
>
> Description
> -------
>
> MESOS-1718: move getExecutorInfo from slave to master
>
>
> Diffs
> -----
>
> src/master/constants.hpp 2c3299bc1aaa0888f7e47a71965c56ada8ecc21f
> src/master/master.hpp 3a7e18232323a1c051bcc97915484b1195fffe58
> src/master/master.cpp 9ee56277c8a472be9e683d5db505becfb5f7c422
> src/slave/constants.hpp bcbb1401aa8f9f04c4f9256bb4f560e18d8994e0
> src/slave/slave.hpp b7586ce42bfac9d9885a3eb8d82deb94680c236c
> src/slave/slave.cpp bb501810d7bb1261ebbbdd147c49948e5a2f8665
>
> Diff: https://reviews.apache.org/r/41305/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Klaus Ma
>
>
Re: Review Request 41305: MESOS-1718: move getExecutorInfo from slave
to master.
Posted by Joseph Wu <jo...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41305/#review115703
-----------------------------------------------------------
General questions:
- Besides removing the possibility for overcommit, does this change give us anything else?
- What happens if a master sends this modified `TaskInfo` to an agent that does not have this chain of patches?
- Have you considered completely converting a `CommandInfo` to `ExecutorInfo`? If we do that, it may be possible to remove all the other places with special logic for command tasks.
src/master/master.cpp (line 2860)
<https://reviews.apache.org/r/41305/#comment176725>
`launcher_dir` is an optional field.
Same for the other fields you added (`sandbox_dir`, `switch_user`, `executor_rootfs).
src/master/master.cpp (lines 3770 - 3779)
<https://reviews.apache.org/r/41305/#comment176721>
It should still be invalid to supply both a CommandInfo and ExecutorInfo in the same TaskInfo.
- Joseph Wu
On Jan. 18, 2016, 6:55 p.m., Klaus Ma wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41305/
> -----------------------------------------------------------
>
> (Updated Jan. 18, 2016, 6:55 p.m.)
>
>
> Review request for mesos, Ben Mahler, Guangya Liu, Artem Harutyunyan, Ian Downes, Joris Van Remoortere, Joseph Wu, and Vinod Kone.
>
>
> Bugs: MESOS-1718
> https://issues.apache.org/jira/browse/MESOS-1718
>
>
> Repository: mesos
>
>
> Description
> -------
>
> MESOS-1718: move getExecutorInfo from slave to master
>
>
> Diffs
> -----
>
> src/master/constants.hpp 2c3299bc1aaa0888f7e47a71965c56ada8ecc21f
> src/master/master.hpp 3a7e18232323a1c051bcc97915484b1195fffe58
> src/master/master.cpp 9ee56277c8a472be9e683d5db505becfb5f7c422
> src/slave/constants.hpp bcbb1401aa8f9f04c4f9256bb4f560e18d8994e0
> src/slave/slave.hpp b7586ce42bfac9d9885a3eb8d82deb94680c236c
> src/slave/slave.cpp bb501810d7bb1261ebbbdd147c49948e5a2f8665
>
> Diff: https://reviews.apache.org/r/41305/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Klaus Ma
>
>
Re: Review Request 41305: MESOS-1718: move getExecutorInfo from slave
to master.
Posted by Klaus Ma <kl...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41305/
-----------------------------------------------------------
(Updated Jan. 19, 2016, 10:55 a.m.)
Review request for mesos, Ben Mahler, Guangya Liu, Artem Harutyunyan, Ian Downes, Joris Van Remoortere, Joseph Wu, and Vinod Kone.
Changes
-------
rebase for apply conflict
Bugs: MESOS-1718
https://issues.apache.org/jira/browse/MESOS-1718
Repository: mesos
Description
-------
MESOS-1718: move getExecutorInfo from slave to master
Diffs (updated)
-----
src/master/constants.hpp 2c3299bc1aaa0888f7e47a71965c56ada8ecc21f
src/master/master.hpp 3a7e18232323a1c051bcc97915484b1195fffe58
src/master/master.cpp 9ee56277c8a472be9e683d5db505becfb5f7c422
src/slave/constants.hpp bcbb1401aa8f9f04c4f9256bb4f560e18d8994e0
src/slave/slave.hpp b7586ce42bfac9d9885a3eb8d82deb94680c236c
src/slave/slave.cpp bb501810d7bb1261ebbbdd147c49948e5a2f8665
Diff: https://reviews.apache.org/r/41305/diff/
Testing
-------
Thanks,
Klaus Ma