You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Ben Mahler <be...@gmail.com> on 2013/11/05 03:55:25 UTC
Review Request 15227: Improved performance of the Registrar.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15227/
-----------------------------------------------------------
Review request for mesos, Benjamin Hindman and Vinod Kone.
Repository: mesos-git
Description
-------
This drastically improves the Registrar performance, bringing it to an acceptable level.
Approximate Before vs After:
Admitting 10,000 slaves: 19 minutes vs 8 seconds
Readmitting 10,000 slaves: 41 minutes vs 7 seconds
Recovering 10,000 slaves: ~2 seconds vs ~2 seconds
Removing 10,0000 slaves: >1hr vs 9seconds
The improvements here are:
1. Eliminated the strange queueing behavior that caused additional update cycles (this was because we always deferred through recover(), even if we can proceed).
2. Eliminated multiple copy operations of registry::Slaves within Mutations.
Diffs
-----
src/master/registrar.cpp 42fe30ededd004e380abc835c224649e02999f11
Diff: https://reviews.apache.org/r/15227/diff/
Testing
-------
make check with the performance test
Thanks,
Ben Mahler
Re: Review Request 15227: Improved performance of the Registrar.
Posted by Ben Mahler <be...@gmail.com>.
> On Feb. 18, 2014, 7:01 p.m., Vinod Kone wrote:
> >
Ah sorry, please disregard this review as it will need significant updating!
- Ben
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15227/#review34732
-----------------------------------------------------------
On Nov. 5, 2013, 2:55 a.m., Ben Mahler wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15227/
> -----------------------------------------------------------
>
> (Updated Nov. 5, 2013, 2:55 a.m.)
>
>
> Review request for mesos, Benjamin Hindman and Vinod Kone.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> This drastically improves the Registrar performance, bringing it to an acceptable level.
>
> Approximate Before vs After:
> Admitting 10,000 slaves: 19 minutes vs 8 seconds
> Readmitting 10,000 slaves: 41 minutes vs 7 seconds
> Recovering 10,000 slaves: ~2 seconds vs ~2 seconds
> Removing 10,0000 slaves: >1hr vs 9seconds
>
> The improvements here are:
> 1. Eliminated the strange queueing behavior that caused additional update cycles (this was because we always deferred through recover(), even if we can proceed).
> 2. Eliminated multiple copy operations of registry::Slaves within Mutations.
>
>
> Diffs
> -----
>
> src/master/registrar.cpp 42fe30ededd004e380abc835c224649e02999f11
>
> Diff: https://reviews.apache.org/r/15227/diff/
>
>
> Testing
> -------
>
> make check with the performance test
>
>
> Thanks,
>
> Ben Mahler
>
>
Re: Review Request 15227: Improved performance of the Registrar.
Posted by Vinod Kone <vi...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15227/#review34732
-----------------------------------------------------------
Ship it!
src/master/registrar.cpp
<https://reviews.apache.org/r/15227/#comment64974>
CHECK_NOTNULL(slaves);
src/master/registrar.cpp
<https://reviews.apache.org/r/15227/#comment64975>
CHECK_NONTULL
src/master/registrar.cpp
<https://reviews.apache.org/r/15227/#comment64977>
CHECK_NONTULL
src/master/registrar.cpp
<https://reviews.apache.org/r/15227/#comment64981>
I'm surprised you had to do this optimization. FWICT, recover() should return immediately if recovery is done. Is not doing a defer() gives that much of a benefit?
I prefer the previous cleaner implementation if there are no significant benefits.
- Vinod Kone
On Nov. 5, 2013, 2:55 a.m., Ben Mahler wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15227/
> -----------------------------------------------------------
>
> (Updated Nov. 5, 2013, 2:55 a.m.)
>
>
> Review request for mesos, Benjamin Hindman and Vinod Kone.
>
>
> Repository: mesos-git
>
>
> Description
> -------
>
> This drastically improves the Registrar performance, bringing it to an acceptable level.
>
> Approximate Before vs After:
> Admitting 10,000 slaves: 19 minutes vs 8 seconds
> Readmitting 10,000 slaves: 41 minutes vs 7 seconds
> Recovering 10,000 slaves: ~2 seconds vs ~2 seconds
> Removing 10,0000 slaves: >1hr vs 9seconds
>
> The improvements here are:
> 1. Eliminated the strange queueing behavior that caused additional update cycles (this was because we always deferred through recover(), even if we can proceed).
> 2. Eliminated multiple copy operations of registry::Slaves within Mutations.
>
>
> Diffs
> -----
>
> src/master/registrar.cpp 42fe30ededd004e380abc835c224649e02999f11
>
> Diff: https://reviews.apache.org/r/15227/diff/
>
>
> Testing
> -------
>
> make check with the performance test
>
>
> Thanks,
>
> Ben Mahler
>
>