You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by Andrei Sekretenko <as...@mesosphere.io> on 2019/10/24 17:05:13 UTC
Review Request 71672: Modified Sorter::addSlave()/removeSlave() to
accept scalar quantities.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71672/
-----------------------------------------------------------
Review request for mesos, Benjamin Mahler and Meng Zhu.
Bugs: MESOS-10015
https://issues.apache.org/jira/browse/MESOS-10015
Repository: mesos
Description
-------
This patch moves conversion of agent `Resources` into `ScalarQuantities`
out of the Sorter. This results in performance improvement of
allocator's `updateAllocation()` method.
Results of `*BENCHMARK_WithReservationParam.UpdateAllocation*`:
Master + previous patch:
Agent resources size: 200 (50 frameworks)
Made 20 reserve and unreserve operations in 471.781183ms
Agent resources size: 400 (100 frameworks)
Made 20 reserve and unreserve operations in 1.022879058secs
Agent resources size: 800 (200 frameworks)
Made 20 reserve and unreserve operations in 2.622324521secs
...
Agent resources size: 6400 (1600 frameworks)
Made 20 reserve and unreserve operations in 2.04261335795mins
Master + previous patch + this patch:
Agent resources size: 200 (50 frameworks)
Made 20 reserve and unreserve operations in 463.223084ms
Agent resources size: 400 (100 frameworks)
Made 20 reserve and unreserve operations in 930.097972ms
Agent resources size: 800 (200 frameworks)
Made 20 reserve and unreserve operations in 2.160506847secs
...
Agent resources size: 6400 (1600 frameworks)
Made 20 reserve and unreserve operations in 1.50729369885mins
Diffs
-----
src/master/allocator/mesos/hierarchical.cpp 21010de363f25c516bb031e4ae48888e53621128
src/master/allocator/mesos/sorter/drf/sorter.hpp 3f6c7413f1b76f3fa86388360983763c8b76079f
src/master/allocator/mesos/sorter/drf/sorter.cpp ef79083b710fba628b4a7e93f903883899f8a71b
src/master/allocator/mesos/sorter/random/sorter.hpp a3097be98d175d2b47714eb8b70b1ce8c5c2bba8
src/master/allocator/mesos/sorter/random/sorter.cpp 86aeb1b8136eaffd2d52d3b603636b01383a9024
src/master/allocator/mesos/sorter/sorter.hpp 6b6b4a1811ba36e0212de17b9a6e63a6f8678a7f
src/tests/sorter_tests.cpp d7fdee8f2cab4c930230750f0bd1a55eb08f89bb
Diff: https://reviews.apache.org/r/71672/diff/1/
Testing
-------
Thanks,
Andrei Sekretenko
Re: Review Request 71672: Modified Sorter::addSlave()/removeSlave() to
accept scalar quantities.
Posted by Andrei Sekretenko <as...@mesosphere.io>.
> On Oct. 24, 2019, 11:54 p.m., Meng Zhu wrote:
> > src/master/allocator/mesos/sorter/sorter.hpp
> > Lines 132-137 (original), 132-137 (patched)
> > <https://reviews.apache.org/r/71672/diff/1/?file=2170237#file2170237line132>
> >
> > I find the comments to be a little bit convoluted, don't think we need to mention shared, how about:
> >
> > ```
> > // NOTE: Sorter provides no direct method to modify resource quantities on an
> > // agent. Updating is done by first `removeSlave()` and then `addSlave()` with
> > // new resource quantities.
> > ```
Tried to simplify this - see https://reviews.apache.org/r/71646/
- Andrei
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71672/#review218403
-----------------------------------------------------------
On Oct. 24, 2019, 5:05 p.m., Andrei Sekretenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71672/
> -----------------------------------------------------------
>
> (Updated Oct. 24, 2019, 5:05 p.m.)
>
>
> Review request for mesos, Benjamin Mahler and Meng Zhu.
>
>
> Bugs: MESOS-10015
> https://issues.apache.org/jira/browse/MESOS-10015
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch moves conversion of agent `Resources` into `ScalarQuantities`
> out of the Sorter. This results in performance improvement of
> allocator's `updateAllocation()` method.
>
> Results of `*BENCHMARK_WithReservationParam.UpdateAllocation*`:
>
> Master + previous patch:
> Agent resources size: 200 (50 frameworks)
> Made 20 reserve and unreserve operations in 471.781183ms
> Agent resources size: 400 (100 frameworks)
> Made 20 reserve and unreserve operations in 1.022879058secs
> Agent resources size: 800 (200 frameworks)
> Made 20 reserve and unreserve operations in 2.622324521secs
> ...
> Agent resources size: 6400 (1600 frameworks)
> Made 20 reserve and unreserve operations in 2.04261335795mins
>
> Master + previous patch + this patch:
> Agent resources size: 200 (50 frameworks)
> Made 20 reserve and unreserve operations in 463.223084ms
> Agent resources size: 400 (100 frameworks)
> Made 20 reserve and unreserve operations in 930.097972ms
> Agent resources size: 800 (200 frameworks)
> Made 20 reserve and unreserve operations in 2.160506847secs
> ...
> Agent resources size: 6400 (1600 frameworks)
> Made 20 reserve and unreserve operations in 1.50729369885mins
>
>
> Diffs
> -----
>
> src/master/allocator/mesos/hierarchical.cpp 21010de363f25c516bb031e4ae48888e53621128
> src/master/allocator/mesos/sorter/drf/sorter.hpp 3f6c7413f1b76f3fa86388360983763c8b76079f
> src/master/allocator/mesos/sorter/drf/sorter.cpp ef79083b710fba628b4a7e93f903883899f8a71b
> src/master/allocator/mesos/sorter/random/sorter.hpp a3097be98d175d2b47714eb8b70b1ce8c5c2bba8
> src/master/allocator/mesos/sorter/random/sorter.cpp 86aeb1b8136eaffd2d52d3b603636b01383a9024
> src/master/allocator/mesos/sorter/sorter.hpp 6b6b4a1811ba36e0212de17b9a6e63a6f8678a7f
> src/tests/sorter_tests.cpp d7fdee8f2cab4c930230750f0bd1a55eb08f89bb
>
>
> Diff: https://reviews.apache.org/r/71672/diff/1/
>
>
> Testing
> -------
>
> make check
>
> ran `*BENCHMARK_WithReservationParam.UpdateAllocation*`
>
>
> Thanks,
>
> Andrei Sekretenko
>
>
Re: Review Request 71672: Modified Sorter::addSlave()/removeSlave() to
accept scalar quantities.
Posted by Meng Zhu <mz...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/71672/#review218403
-----------------------------------------------------------
Fix it, then Ship it!
src/master/allocator/mesos/sorter/sorter.hpp
Lines 132-137 (original), 132-137 (patched)
<https://reviews.apache.org/r/71672/#comment306114>
I find the comments to be a little bit convoluted, don't think we need to mention shared, how about:
```
// NOTE: Sorter provides no direct method to modify resource quantities on an
// agent. Updating is done by first `removeSlave()` and then `addSlave()` with
// new resource quantities.
```
- Meng Zhu
On Oct. 24, 2019, 10:05 a.m., Andrei Sekretenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/71672/
> -----------------------------------------------------------
>
> (Updated Oct. 24, 2019, 10:05 a.m.)
>
>
> Review request for mesos, Benjamin Mahler and Meng Zhu.
>
>
> Bugs: MESOS-10015
> https://issues.apache.org/jira/browse/MESOS-10015
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch moves conversion of agent `Resources` into `ScalarQuantities`
> out of the Sorter. This results in performance improvement of
> allocator's `updateAllocation()` method.
>
> Results of `*BENCHMARK_WithReservationParam.UpdateAllocation*`:
>
> Master + previous patch:
> Agent resources size: 200 (50 frameworks)
> Made 20 reserve and unreserve operations in 471.781183ms
> Agent resources size: 400 (100 frameworks)
> Made 20 reserve and unreserve operations in 1.022879058secs
> Agent resources size: 800 (200 frameworks)
> Made 20 reserve and unreserve operations in 2.622324521secs
> ...
> Agent resources size: 6400 (1600 frameworks)
> Made 20 reserve and unreserve operations in 2.04261335795mins
>
> Master + previous patch + this patch:
> Agent resources size: 200 (50 frameworks)
> Made 20 reserve and unreserve operations in 463.223084ms
> Agent resources size: 400 (100 frameworks)
> Made 20 reserve and unreserve operations in 930.097972ms
> Agent resources size: 800 (200 frameworks)
> Made 20 reserve and unreserve operations in 2.160506847secs
> ...
> Agent resources size: 6400 (1600 frameworks)
> Made 20 reserve and unreserve operations in 1.50729369885mins
>
>
> Diffs
> -----
>
> src/master/allocator/mesos/hierarchical.cpp 21010de363f25c516bb031e4ae48888e53621128
> src/master/allocator/mesos/sorter/drf/sorter.hpp 3f6c7413f1b76f3fa86388360983763c8b76079f
> src/master/allocator/mesos/sorter/drf/sorter.cpp ef79083b710fba628b4a7e93f903883899f8a71b
> src/master/allocator/mesos/sorter/random/sorter.hpp a3097be98d175d2b47714eb8b70b1ce8c5c2bba8
> src/master/allocator/mesos/sorter/random/sorter.cpp 86aeb1b8136eaffd2d52d3b603636b01383a9024
> src/master/allocator/mesos/sorter/sorter.hpp 6b6b4a1811ba36e0212de17b9a6e63a6f8678a7f
> src/tests/sorter_tests.cpp d7fdee8f2cab4c930230750f0bd1a55eb08f89bb
>
>
> Diff: https://reviews.apache.org/r/71672/diff/1/
>
>
> Testing
> -------
>
> make check
>
> ran `*BENCHMARK_WithReservationParam.UpdateAllocation*`
>
>
> Thanks,
>
> Andrei Sekretenko
>
>