You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by haosdent huang <ha...@gmail.com> on 2015/05/18 19:05:40 UTC
Review Request 34362: Include ExecutorInfos in master/state.json
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34362/
-----------------------------------------------------------
Review request for mesos and Adam B.
Bugs: MESOS-2743
https://issues.apache.org/jira/browse/MESOS-2743
Repository: mesos
Description
-------
Include ExecutorInfos in master/state.json
Diffs
-----
src/master/http.cpp 7fa47a9fa2f5667631169876964dccf7fdcc172e
Diff: https://reviews.apache.org/r/34362/diff/
Testing
-------
make check
Thanks,
haosdent huang
Re: Review Request 34362: Include ExecutorInfos in master/state.json
Posted by Mesos ReviewBot <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34362/#review84198
-----------------------------------------------------------
Patch looks great!
Reviews applied: [34362]
All tests passed.
- Mesos ReviewBot
On May 18, 2015, 5:41 p.m., haosdent huang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34362/
> -----------------------------------------------------------
>
> (Updated May 18, 2015, 5:41 p.m.)
>
>
> Review request for mesos and Adam B.
>
>
> Bugs: MESOS-2743
> https://issues.apache.org/jira/browse/MESOS-2743
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Include ExecutorInfos in master/state.json
>
>
> Diffs
> -----
>
> src/common/http.hpp afce7fea334c7bfa57efc48c413bf906a2ebde32
> src/common/http.cpp 2ac7fba7a3aac913540f1b09768777393b79284a
> src/master/http.cpp 7fa47a9fa2f5667631169876964dccf7fdcc172e
> src/slave/http.cpp b5e77b09435e50db5231a2b32faf76dab91dae54
>
> Diff: https://reviews.apache.org/r/34362/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> haosdent huang
>
>
Re: Review Request 34362: Include ExecutorInfos in master/state.json
Posted by haosdent huang <ha...@gmail.com>.
> On June 1, 2015, 8:53 a.m., Adam B wrote:
> > Looks good, but it's missing a few performance tricks we've used in other json model() functions, to reduce the json construction time, especially for large clusters. We should do some quick benchmarking on a cluster with thousands of (simulated) executors to see how much overhead this adds to state.json.
@adam-mesos, thank you very much for you explain and your help. I would try to simulate thousands of executors and upload the test result later. Maybe need after a few days.
- haosdent
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34362/#review85959
-----------------------------------------------------------
On June 1, 2015, 5:06 p.m., haosdent huang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34362/
> -----------------------------------------------------------
>
> (Updated June 1, 2015, 5:06 p.m.)
>
>
> Review request for mesos and Adam B.
>
>
> Bugs: MESOS-2743
> https://issues.apache.org/jira/browse/MESOS-2743
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Include ExecutorInfos in master/state.json
>
>
> Diffs
> -----
>
> src/common/http.hpp afce7fea334c7bfa57efc48c413bf906a2ebde32
> src/common/http.cpp 2ac7fba7a3aac913540f1b09768777393b79284a
> src/master/http.cpp 01dbdad9720a9cd30505683ae1117970cda918a0
> src/slave/http.cpp b5e77b09435e50db5231a2b32faf76dab91dae54
>
> Diff: https://reviews.apache.org/r/34362/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> haosdent huang
>
>
Re: Review Request 34362: Include ExecutorInfos in master/state.json
Posted by Adam B <ad...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34362/#review85959
-----------------------------------------------------------
Looks good, but it's missing a few performance tricks we've used in other json model() functions, to reduce the json construction time, especially for large clusters. We should do some quick benchmarking on a cluster with thousands of (simulated) executors to see how much overhead this adds to state.json.
src/master/http.cpp
<https://reviews.apache.org/r/34362/#comment137800>
Can you reserve the size of the executors array like we've done for tasks/offers (See MESOS-2353)? I suppose you'll have to loop through all the slaves to count each of their executor lists, but that's probably still less expensive than growing the executors array each time.
src/master/http.cpp
<https://reviews.apache.org/r/34362/#comment137802>
Nit: Please insert blank line between these two like in other similar blocks.
src/master/http.cpp
<https://reviews.apache.org/r/34362/#comment137801>
std::move(executors);
- Adam B
On May 18, 2015, 10:41 a.m., haosdent huang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34362/
> -----------------------------------------------------------
>
> (Updated May 18, 2015, 10:41 a.m.)
>
>
> Review request for mesos and Adam B.
>
>
> Bugs: MESOS-2743
> https://issues.apache.org/jira/browse/MESOS-2743
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Include ExecutorInfos in master/state.json
>
>
> Diffs
> -----
>
> src/common/http.hpp afce7fea334c7bfa57efc48c413bf906a2ebde32
> src/common/http.cpp 2ac7fba7a3aac913540f1b09768777393b79284a
> src/master/http.cpp 7fa47a9fa2f5667631169876964dccf7fdcc172e
> src/slave/http.cpp b5e77b09435e50db5231a2b32faf76dab91dae54
>
> Diff: https://reviews.apache.org/r/34362/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> haosdent huang
>
>
Re: Review Request 34362: Include ExecutorInfos in master/state.json
Posted by Alexander Rojas <al...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34362/#review84501
-----------------------------------------------------------
Ship it!
Ship It!
- Alexander Rojas
On May 18, 2015, 7:41 p.m., haosdent huang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34362/
> -----------------------------------------------------------
>
> (Updated May 18, 2015, 7:41 p.m.)
>
>
> Review request for mesos and Adam B.
>
>
> Bugs: MESOS-2743
> https://issues.apache.org/jira/browse/MESOS-2743
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Include ExecutorInfos in master/state.json
>
>
> Diffs
> -----
>
> src/common/http.hpp afce7fea334c7bfa57efc48c413bf906a2ebde32
> src/common/http.cpp 2ac7fba7a3aac913540f1b09768777393b79284a
> src/master/http.cpp 7fa47a9fa2f5667631169876964dccf7fdcc172e
> src/slave/http.cpp b5e77b09435e50db5231a2b32faf76dab91dae54
>
> Diff: https://reviews.apache.org/r/34362/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> haosdent huang
>
>
Re: Review Request 34362: Include ExecutorInfos in master/state.json
Posted by Adam B <ad...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34362/#review88564
-----------------------------------------------------------
Ship it!
Looks great! I'll fix the preexisting typo, and commit this soon.
src/common/http.cpp (line 159)
<https://reviews.apache.org/r/34362/#comment141143>
Found a typo comment that got copied around: s/in/it/?
- Adam B
On June 7, 2015, 2:29 a.m., haosdent huang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34362/
> -----------------------------------------------------------
>
> (Updated June 7, 2015, 2:29 a.m.)
>
>
> Review request for mesos and Adam B.
>
>
> Bugs: MESOS-2743
> https://issues.apache.org/jira/browse/MESOS-2743
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Include ExecutorInfos in master/state.json
>
>
> Diffs
> -----
>
> src/common/http.hpp afce7fea334c7bfa57efc48c413bf906a2ebde32
> src/common/http.cpp 2ac7fba7a3aac913540f1b09768777393b79284a
> src/master/http.cpp 01dbdad9720a9cd30505683ae1117970cda918a0
> src/slave/http.cpp b5e77b09435e50db5231a2b32faf76dab91dae54
>
> Diff: https://reviews.apache.org/r/34362/diff/
>
>
> Testing
> -------
>
> make check
>
> I start 1500 executor in 30 slaves. In executor, it just sleep. And the test result is
>
> time curl -s -o /dev/null localhost:5050/master/state.json
>
> real 0m0.651s
> user 0m0.001s
> sys 0m0.004s
>
>
> Thanks,
>
> haosdent huang
>
>
Re: Review Request 34362: Include ExecutorInfos in master/state.json
Posted by haosdent huang <ha...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34362/
-----------------------------------------------------------
(Updated June 7, 2015, 9:29 a.m.)
Review request for mesos and Adam B.
Bugs: MESOS-2743
https://issues.apache.org/jira/browse/MESOS-2743
Repository: mesos
Description
-------
Include ExecutorInfos in master/state.json
Diffs
-----
src/common/http.hpp afce7fea334c7bfa57efc48c413bf906a2ebde32
src/common/http.cpp 2ac7fba7a3aac913540f1b09768777393b79284a
src/master/http.cpp 01dbdad9720a9cd30505683ae1117970cda918a0
src/slave/http.cpp b5e77b09435e50db5231a2b32faf76dab91dae54
Diff: https://reviews.apache.org/r/34362/diff/
Testing (updated)
-------
make check
I start 1500 executor in 30 slaves. In executor, it just sleep. And the test result is
time curl -s -o /dev/null localhost:5050/master/state.json
real 0m0.651s
user 0m0.001s
sys 0m0.004s
Thanks,
haosdent huang
Re: Review Request 34362: Include ExecutorInfos in master/state.json
Posted by Mesos ReviewBot <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34362/#review86045
-----------------------------------------------------------
Patch looks great!
Reviews applied: [34362]
All tests passed.
- Mesos ReviewBot
On June 1, 2015, 5:13 p.m., haosdent huang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34362/
> -----------------------------------------------------------
>
> (Updated June 1, 2015, 5:13 p.m.)
>
>
> Review request for mesos and Adam B.
>
>
> Bugs: MESOS-2743
> https://issues.apache.org/jira/browse/MESOS-2743
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Include ExecutorInfos in master/state.json
>
>
> Diffs
> -----
>
> src/common/http.hpp afce7fea334c7bfa57efc48c413bf906a2ebde32
> src/common/http.cpp 2ac7fba7a3aac913540f1b09768777393b79284a
> src/master/http.cpp 01dbdad9720a9cd30505683ae1117970cda918a0
> src/slave/http.cpp b5e77b09435e50db5231a2b32faf76dab91dae54
>
> Diff: https://reviews.apache.org/r/34362/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> haosdent huang
>
>
Re: Review Request 34362: Include ExecutorInfos in master/state.json
Posted by haosdent huang <ha...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34362/
-----------------------------------------------------------
(Updated June 1, 2015, 5:13 p.m.)
Review request for mesos and Adam B.
Bugs: MESOS-2743
https://issues.apache.org/jira/browse/MESOS-2743
Repository: mesos
Description
-------
Include ExecutorInfos in master/state.json
Diffs (updated)
-----
src/common/http.hpp afce7fea334c7bfa57efc48c413bf906a2ebde32
src/common/http.cpp 2ac7fba7a3aac913540f1b09768777393b79284a
src/master/http.cpp 01dbdad9720a9cd30505683ae1117970cda918a0
src/slave/http.cpp b5e77b09435e50db5231a2b32faf76dab91dae54
Diff: https://reviews.apache.org/r/34362/diff/
Testing
-------
make check
Thanks,
haosdent huang
Re: Review Request 34362: Include ExecutorInfos in master/state.json
Posted by haosdent huang <ha...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34362/
-----------------------------------------------------------
(Updated June 1, 2015, 5:06 p.m.)
Review request for mesos and Adam B.
Bugs: MESOS-2743
https://issues.apache.org/jira/browse/MESOS-2743
Repository: mesos
Description
-------
Include ExecutorInfos in master/state.json
Diffs (updated)
-----
src/common/http.hpp afce7fea334c7bfa57efc48c413bf906a2ebde32
src/common/http.cpp 2ac7fba7a3aac913540f1b09768777393b79284a
src/master/http.cpp 01dbdad9720a9cd30505683ae1117970cda918a0
src/slave/http.cpp b5e77b09435e50db5231a2b32faf76dab91dae54
Diff: https://reviews.apache.org/r/34362/diff/
Testing
-------
make check
Thanks,
haosdent huang
Re: Review Request 34362: Include ExecutorInfos in master/state.json
Posted by haosdent huang <ha...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34362/
-----------------------------------------------------------
(Updated May 18, 2015, 5:41 p.m.)
Review request for mesos and Adam B.
Bugs: MESOS-2743
https://issues.apache.org/jira/browse/MESOS-2743
Repository: mesos
Description
-------
Include ExecutorInfos in master/state.json
Diffs (updated)
-----
src/common/http.hpp afce7fea334c7bfa57efc48c413bf906a2ebde32
src/common/http.cpp 2ac7fba7a3aac913540f1b09768777393b79284a
src/master/http.cpp 7fa47a9fa2f5667631169876964dccf7fdcc172e
src/slave/http.cpp b5e77b09435e50db5231a2b32faf76dab91dae54
Diff: https://reviews.apache.org/r/34362/diff/
Testing
-------
make check
Thanks,
haosdent huang