You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by Andrei Sekretenko <as...@mesosphere.io> on 2019/05/07 13:58:24 UTC
Re: Review Request 70591: Added `struct RoleInfo` to track role
reservations and framework IDs.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70591/
-----------------------------------------------------------
(Updated May 7, 2019, 1:58 p.m.)
Review request for mesos, Benjamin Mahler and Meng Zhu.
Summary (updated)
-----------------
Added `struct RoleInfo` to track role reservations and framework IDs.
Bugs: MESOS-9701
https://issues.apache.org/jira/browse/MESOS-9701
Repository: mesos
Description (updated)
-------
This patch introduces `struct RoleInfo` which contains the framework IDs and
reservatons tracked under a role. Two separate hashmaps used for this purpose
previously are replaced by a single hashmap, thus the need to keep the
two consistent with each other is removed. This simplifies the role tracking
logic in the allocator. The patch introduces minimal to no performance impact.
Diffs (updated)
-----
src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
src/master/allocator/mesos/hierarchical.cpp 64a076ddd29711437d539a06bb0470755828cc87
Diff: https://reviews.apache.org/r/70591/diff/2/
Changes: https://reviews.apache.org/r/70591/diff/1-2/
Testing
-------
make check
Benchmarking: 5 runs of `BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5` with the optimized build.
Performance impact in this benchmark seems to be negligible.
BEFORE:
Added 3000 agents in 51.553929ms
Added 3000 frameworks in 15.174748344secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.400805171secs
Made 0 allocation in 12.5850238secs
Added 3000 agents in 55.739336ms
Added 3000 frameworks in 14.730404769secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.563439682secs
Made 0 allocation in 13.063555055secs
Added 3000 agents in 54.414733ms
Added 3000 frameworks in 15.10136842secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 13.501664915secs
Made 0 allocation in 12.89034382secs
Added 3000 agents in 52.58252ms
Added 3000 frameworks in 14.048350298secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.299952145secs
Made 0 allocation in 11.888248811secs
Added 3000 agents in 52.821439ms
Added 3000 frameworks in 15.344450583secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.634000025secs
Made 0 allocation in 12.427171541secs
AFTER:
Added 3000 agents in 69.716648ms
Added 3000 frameworks in 15.249001979secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.860494226secs
Made 0 allocation in 12.228866329secs
Added 3000 agents in 52.639388ms
Added 3000 frameworks in 15.207895482secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.504777266secs
Made 0 allocation in 12.70388062secs
Added 3000 agents in 56.865794ms
Added 3000 frameworks in 15.284003915secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 13.86859815secs
Made 0 allocation in 12.538958231secs
Added 3000 agents in 56.028013ms
Added 3000 frameworks in 13.892577869secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.341724418secs
Made 0 allocation in 12.23022189secs
Added 3000 agents in 52.368219ms
Added 3000 frameworks in 13.978581104secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.701682501secs
Made 0 allocation in 12.141360313secs
Thanks,
Andrei Sekretenko
Re: Review Request 70591: Added `struct RoleInfo` to track role
reservations and framework IDs.
Posted by Andrei Sekretenko <as...@mesosphere.io>.
> On May 9, 2019, 8:55 p.m., Meng Zhu wrote:
> > src/master/allocator/mesos/hierarchical.hpp
> > Lines 111-114 (patched)
> > <https://reviews.apache.org/r/70591/diff/2/?file=2143582#file2143582line111>
> >
> > I find `roleInfo` as a OK name, espcially it is a value indexed by a role name.
> >
> > It seems odd to me that we leave `TODO` for such a task here (it's obvious we can either do it now or, due the lack of good choices, unlikely to happen in the future).
Removed this TODO. Indeed, we do not have good choices at this point, and they will not appear out of nowhere.
The renaming might become logical after some other refactoring or functional change, but this is another story.
> On May 9, 2019, 8:55 p.m., Meng Zhu wrote:
> > src/master/allocator/mesos/hierarchical.cpp
> > Lines 2609-2623 (original), 2614-2624 (patched)
> > <https://reviews.apache.org/r/70591/diff/2/?file=2143583#file2143583line2617>
> >
> > Thanks for the refactoring! Let's pull this refactor out into a separate patch in front this one.
Moved this refactoring into https://reviews.apache.org/r/70626/
- Andrei
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70591/#review215166
-----------------------------------------------------------
On May 10, 2019, 4:31 p.m., Andrei Sekretenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70591/
> -----------------------------------------------------------
>
> (Updated May 10, 2019, 4:31 p.m.)
>
>
> Review request for mesos, Benjamin Mahler and Meng Zhu.
>
>
> Bugs: MESOS-9701
> https://issues.apache.org/jira/browse/MESOS-9701
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch introduces struct RoleInfo which contains the framework IDs
> and reservatons tracked under a role. Two separate hashmaps used for
> this purpose previously are replaced by a single hashmap, thus the need
> to keep the two consistent with each other is removed. This simplifies
> the role tracking logic in the allocator. The patch introduces minimal
> to no performance impact.
>
>
> Diffs
> -----
>
> src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
> src/master/allocator/mesos/hierarchical.cpp 64a076ddd29711437d539a06bb0470755828cc87
>
>
> Diff: https://reviews.apache.org/r/70591/diff/3/
>
>
> Testing
> -------
>
> make check
>
> Benchmarking: 5 runs of `BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5` with the optimized build.
>
> Performance impact in this benchmark seems to be negligible.
>
> BEFORE:
> Added 3000 agents in 51.553929ms
> Added 3000 frameworks in 15.174748344secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.400805171secs
> Made 0 allocation in 12.5850238secs
>
> Added 3000 agents in 55.739336ms
> Added 3000 frameworks in 14.730404769secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.563439682secs
> Made 0 allocation in 13.063555055secs
>
> Added 3000 agents in 54.414733ms
> Added 3000 frameworks in 15.10136842secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.501664915secs
> Made 0 allocation in 12.89034382secs
>
> Added 3000 agents in 52.58252ms
> Added 3000 frameworks in 14.048350298secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.299952145secs
> Made 0 allocation in 11.888248811secs
>
> Added 3000 agents in 52.821439ms
> Added 3000 frameworks in 15.344450583secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.634000025secs
> Made 0 allocation in 12.427171541secs
>
>
> AFTER:
>
> Added 3000 agents in 69.716648ms
> Added 3000 frameworks in 15.249001979secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.860494226secs
> Made 0 allocation in 12.228866329secs
>
> Added 3000 agents in 52.639388ms
> Added 3000 frameworks in 15.207895482secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.504777266secs
> Made 0 allocation in 12.70388062secs
>
> Added 3000 agents in 56.865794ms
> Added 3000 frameworks in 15.284003915secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.86859815secs
> Made 0 allocation in 12.538958231secs
>
> Added 3000 agents in 56.028013ms
> Added 3000 frameworks in 13.892577869secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.341724418secs
> Made 0 allocation in 12.23022189secs
>
> Added 3000 agents in 52.368219ms
> Added 3000 frameworks in 13.978581104secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.701682501secs
> Made 0 allocation in 12.141360313secs
>
>
> Thanks,
>
> Andrei Sekretenko
>
>
Re: Review Request 70591: Added `struct RoleInfo` to track role
reservations and framework IDs.
Posted by Meng Zhu <mz...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70591/#review215166
-----------------------------------------------------------
src/master/allocator/mesos/hierarchical.hpp
Lines 111-114 (patched)
<https://reviews.apache.org/r/70591/#comment301698>
I find `roleInfo` as a OK name, espcially it is a value indexed by a role name.
It seems odd to me that we leave `TODO` for such a task here (it's obvious we can either do it now or, due the lack of good choices, unlikely to happen in the future).
src/master/allocator/mesos/hierarchical.hpp
Lines 115 (patched)
<https://reviews.apache.org/r/70591/#comment301697>
add space before `{`:
struct RoleInfo {
src/master/allocator/mesos/hierarchical.hpp
Lines 118 (patched)
<https://reviews.apache.org/r/70591/#comment301699>
New line above this.
src/master/allocator/mesos/hierarchical.cpp
Lines 2547-2572 (original), 2549-2577 (patched)
<https://reviews.apache.org/r/70591/#comment301700>
There are a lot `roles.at(role)` here, let's keep it in a variable?
```
RoleInfo& roleInfo = roles.at(role);
....
```
src/master/allocator/mesos/hierarchical.cpp
Lines 2609-2623 (original), 2614-2624 (patched)
<https://reviews.apache.org/r/70591/#comment301701>
Thanks for the refactoring! Let's pull this refactor out into a separate patch in front this one.
- Meng Zhu
On May 7, 2019, 6:58 a.m., Andrei Sekretenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70591/
> -----------------------------------------------------------
>
> (Updated May 7, 2019, 6:58 a.m.)
>
>
> Review request for mesos, Benjamin Mahler and Meng Zhu.
>
>
> Bugs: MESOS-9701
> https://issues.apache.org/jira/browse/MESOS-9701
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch introduces `struct RoleInfo` which contains the framework IDs and
> reservatons tracked under a role. Two separate hashmaps used for this purpose
> previously are replaced by a single hashmap, thus the need to keep the
> two consistent with each other is removed. This simplifies the role tracking
> logic in the allocator. The patch introduces minimal to no performance impact.
>
>
> Diffs
> -----
>
> src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
> src/master/allocator/mesos/hierarchical.cpp 64a076ddd29711437d539a06bb0470755828cc87
>
>
> Diff: https://reviews.apache.org/r/70591/diff/2/
>
>
> Testing
> -------
>
> make check
>
> Benchmarking: 5 runs of `BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5` with the optimized build.
>
> Performance impact in this benchmark seems to be negligible.
>
> BEFORE:
> Added 3000 agents in 51.553929ms
> Added 3000 frameworks in 15.174748344secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.400805171secs
> Made 0 allocation in 12.5850238secs
>
> Added 3000 agents in 55.739336ms
> Added 3000 frameworks in 14.730404769secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.563439682secs
> Made 0 allocation in 13.063555055secs
>
> Added 3000 agents in 54.414733ms
> Added 3000 frameworks in 15.10136842secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.501664915secs
> Made 0 allocation in 12.89034382secs
>
> Added 3000 agents in 52.58252ms
> Added 3000 frameworks in 14.048350298secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.299952145secs
> Made 0 allocation in 11.888248811secs
>
> Added 3000 agents in 52.821439ms
> Added 3000 frameworks in 15.344450583secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.634000025secs
> Made 0 allocation in 12.427171541secs
>
>
> AFTER:
>
> Added 3000 agents in 69.716648ms
> Added 3000 frameworks in 15.249001979secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.860494226secs
> Made 0 allocation in 12.228866329secs
>
> Added 3000 agents in 52.639388ms
> Added 3000 frameworks in 15.207895482secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.504777266secs
> Made 0 allocation in 12.70388062secs
>
> Added 3000 agents in 56.865794ms
> Added 3000 frameworks in 15.284003915secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.86859815secs
> Made 0 allocation in 12.538958231secs
>
> Added 3000 agents in 56.028013ms
> Added 3000 frameworks in 13.892577869secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.341724418secs
> Made 0 allocation in 12.23022189secs
>
> Added 3000 agents in 52.368219ms
> Added 3000 frameworks in 13.978581104secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.701682501secs
> Made 0 allocation in 12.141360313secs
>
>
> Thanks,
>
> Andrei Sekretenko
>
>
Re: Review Request 70591: Added `struct RoleInfo` to track role
reservations and framework IDs.
Posted by Mesos Reviewbot <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70591/#review215091
-----------------------------------------------------------
Patch looks great!
Reviews applied: [70591]
Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' CONFIGURATION='--verbose --disable-libtool-wrappers --disable-parallel-test-execution' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; ./support/docker-build.sh
- Mesos Reviewbot
On May 7, 2019, 1:58 p.m., Andrei Sekretenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70591/
> -----------------------------------------------------------
>
> (Updated May 7, 2019, 1:58 p.m.)
>
>
> Review request for mesos, Benjamin Mahler and Meng Zhu.
>
>
> Bugs: MESOS-9701
> https://issues.apache.org/jira/browse/MESOS-9701
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch introduces `struct RoleInfo` which contains the framework IDs and
> reservatons tracked under a role. Two separate hashmaps used for this purpose
> previously are replaced by a single hashmap, thus the need to keep the
> two consistent with each other is removed. This simplifies the role tracking
> logic in the allocator. The patch introduces minimal to no performance impact.
>
>
> Diffs
> -----
>
> src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
> src/master/allocator/mesos/hierarchical.cpp 64a076ddd29711437d539a06bb0470755828cc87
>
>
> Diff: https://reviews.apache.org/r/70591/diff/2/
>
>
> Testing
> -------
>
> make check
>
> Benchmarking: 5 runs of `BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5` with the optimized build.
>
> Performance impact in this benchmark seems to be negligible.
>
> BEFORE:
> Added 3000 agents in 51.553929ms
> Added 3000 frameworks in 15.174748344secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.400805171secs
> Made 0 allocation in 12.5850238secs
>
> Added 3000 agents in 55.739336ms
> Added 3000 frameworks in 14.730404769secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.563439682secs
> Made 0 allocation in 13.063555055secs
>
> Added 3000 agents in 54.414733ms
> Added 3000 frameworks in 15.10136842secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.501664915secs
> Made 0 allocation in 12.89034382secs
>
> Added 3000 agents in 52.58252ms
> Added 3000 frameworks in 14.048350298secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.299952145secs
> Made 0 allocation in 11.888248811secs
>
> Added 3000 agents in 52.821439ms
> Added 3000 frameworks in 15.344450583secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.634000025secs
> Made 0 allocation in 12.427171541secs
>
>
> AFTER:
>
> Added 3000 agents in 69.716648ms
> Added 3000 frameworks in 15.249001979secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.860494226secs
> Made 0 allocation in 12.228866329secs
>
> Added 3000 agents in 52.639388ms
> Added 3000 frameworks in 15.207895482secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.504777266secs
> Made 0 allocation in 12.70388062secs
>
> Added 3000 agents in 56.865794ms
> Added 3000 frameworks in 15.284003915secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.86859815secs
> Made 0 allocation in 12.538958231secs
>
> Added 3000 agents in 56.028013ms
> Added 3000 frameworks in 13.892577869secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.341724418secs
> Made 0 allocation in 12.23022189secs
>
> Added 3000 agents in 52.368219ms
> Added 3000 frameworks in 13.978581104secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.701682501secs
> Made 0 allocation in 12.141360313secs
>
>
> Thanks,
>
> Andrei Sekretenko
>
>
Re: Review Request 70591: Added `struct RoleInfo` to track role
reservations and framework IDs.
Posted by Meng Zhu <mz...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70591/#review215353
-----------------------------------------------------------
Much nicer, thanks!
src/master/allocator/mesos/hierarchical.hpp
Lines 114-116 (patched)
<https://reviews.apache.org/r/70591/#comment302012>
// Aggregated reserved scalar resource quantities on all agents
// tied to this role, if any. Note, non-scalar resources such as
// ports are excluded.
src/master/allocator/mesos/hierarchical.cpp
Lines 2578 (patched)
<https://reviews.apache.org/r/70591/#comment302013>
New line not necessary?
src/master/allocator/mesos/hierarchical.cpp
Lines 2643-2652 (original), 2646-2653 (patched)
<https://reviews.apache.org/r/70591/#comment302015>
Let's use a reference to make the code more consise:
```
CHECK(roles.contains(r));
RoleInfo& roleInfo = roles.at(r);
CHECK(roleInfo.reservationScalarQuantities.contains(quantities));
roleInfo.reservationScalarQuantities -= quantities;
if (roleInfo.isEmpty()) {
roles.erase(r);
}
```
- Meng Zhu
On May 16, 2019, 7:16 a.m., Andrei Sekretenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70591/
> -----------------------------------------------------------
>
> (Updated May 16, 2019, 7:16 a.m.)
>
>
> Review request for mesos, Benjamin Mahler and Meng Zhu.
>
>
> Bugs: MESOS-9701
> https://issues.apache.org/jira/browse/MESOS-9701
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch introduces struct RoleInfo which contains the framework IDs
> and reservatons tracked under a role. Two separate hashmaps used for
> this purpose previously are replaced by a single hashmap, thus the need
> to keep the two consistent with each other is removed. This simplifies
> the role tracking logic in the allocator. The patch introduces minimal
> to no performance impact.
>
>
> Diffs
> -----
>
> src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
> src/master/allocator/mesos/hierarchical.cpp 875cfcd091f5f58afb89e752da5100a75828dd67
>
>
> Diff: https://reviews.apache.org/r/70591/diff/4/
>
>
> Testing
> -------
>
> make check
>
> Benchmarking: 5 runs of `BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5` with the optimized build.
>
> Performance impact in this benchmark seems to be negligible.
>
> BEFORE:
> Added 3000 agents in 51.553929ms
> Added 3000 frameworks in 15.174748344secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.400805171secs
> Made 0 allocation in 12.5850238secs
>
> Added 3000 agents in 55.739336ms
> Added 3000 frameworks in 14.730404769secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.563439682secs
> Made 0 allocation in 13.063555055secs
>
> Added 3000 agents in 54.414733ms
> Added 3000 frameworks in 15.10136842secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.501664915secs
> Made 0 allocation in 12.89034382secs
>
> Added 3000 agents in 52.58252ms
> Added 3000 frameworks in 14.048350298secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.299952145secs
> Made 0 allocation in 11.888248811secs
>
> Added 3000 agents in 52.821439ms
> Added 3000 frameworks in 15.344450583secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.634000025secs
> Made 0 allocation in 12.427171541secs
>
>
> AFTER:
>
> Added 3000 agents in 69.716648ms
> Added 3000 frameworks in 15.249001979secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.860494226secs
> Made 0 allocation in 12.228866329secs
>
> Added 3000 agents in 52.639388ms
> Added 3000 frameworks in 15.207895482secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.504777266secs
> Made 0 allocation in 12.70388062secs
>
> Added 3000 agents in 56.865794ms
> Added 3000 frameworks in 15.284003915secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.86859815secs
> Made 0 allocation in 12.538958231secs
>
> Added 3000 agents in 56.028013ms
> Added 3000 frameworks in 13.892577869secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.341724418secs
> Made 0 allocation in 12.23022189secs
>
> Added 3000 agents in 52.368219ms
> Added 3000 frameworks in 13.978581104secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.701682501secs
> Made 0 allocation in 12.141360313secs
>
>
> Thanks,
>
> Andrei Sekretenko
>
>
Re: Review Request 70591: Added `struct Role` to track role
reservations and framework IDs.
Posted by Meng Zhu <mz...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70591/#review215457
-----------------------------------------------------------
Ship it!
Ship It!
- Meng Zhu
On May 22, 2019, 6:20 a.m., Andrei Sekretenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70591/
> -----------------------------------------------------------
>
> (Updated May 22, 2019, 6:20 a.m.)
>
>
> Review request for mesos, Benjamin Mahler and Meng Zhu.
>
>
> Bugs: MESOS-9701
> https://issues.apache.org/jira/browse/MESOS-9701
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch introduces `struct Role` which contains the framework IDs
> and reservatons tracked under a role. Two separate hashmaps used for
> this purpose previously are replaced by a single hashmap, thus the need
> to keep the two consistent with each other is removed. This simplifies
> the role tracking logic in the allocator. The patch introduces minimal
> to no performance impact.
>
>
> Diffs
> -----
>
> src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
> src/master/allocator/mesos/hierarchical.cpp 6df9a3b4ef470e57862815a862494cdb5e26d3e3
>
>
> Diff: https://reviews.apache.org/r/70591/diff/6/
>
>
> Testing
> -------
>
> make check
>
> Benchmarking: 5 runs of `BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5` with the optimized build.
>
> Performance impact in this benchmark seems to be negligible.
>
> BEFORE:
> Added 3000 agents in 51.553929ms
> Added 3000 frameworks in 15.174748344secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.400805171secs
> Made 0 allocation in 12.5850238secs
>
> Added 3000 agents in 55.739336ms
> Added 3000 frameworks in 14.730404769secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.563439682secs
> Made 0 allocation in 13.063555055secs
>
> Added 3000 agents in 54.414733ms
> Added 3000 frameworks in 15.10136842secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.501664915secs
> Made 0 allocation in 12.89034382secs
>
> Added 3000 agents in 52.58252ms
> Added 3000 frameworks in 14.048350298secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.299952145secs
> Made 0 allocation in 11.888248811secs
>
> Added 3000 agents in 52.821439ms
> Added 3000 frameworks in 15.344450583secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.634000025secs
> Made 0 allocation in 12.427171541secs
>
>
> AFTER:
>
> Added 3000 agents in 69.716648ms
> Added 3000 frameworks in 15.249001979secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.860494226secs
> Made 0 allocation in 12.228866329secs
>
> Added 3000 agents in 52.639388ms
> Added 3000 frameworks in 15.207895482secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.504777266secs
> Made 0 allocation in 12.70388062secs
>
> Added 3000 agents in 56.865794ms
> Added 3000 frameworks in 15.284003915secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.86859815secs
> Made 0 allocation in 12.538958231secs
>
> Added 3000 agents in 56.028013ms
> Added 3000 frameworks in 13.892577869secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.341724418secs
> Made 0 allocation in 12.23022189secs
>
> Added 3000 agents in 52.368219ms
> Added 3000 frameworks in 13.978581104secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.701682501secs
> Made 0 allocation in 12.141360313secs
>
>
> Thanks,
>
> Andrei Sekretenko
>
>
Re: Review Request 70591: Added `struct Role` to track role
reservations and framework IDs.
Posted by Andrei Sekretenko <as...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70591/
-----------------------------------------------------------
(Updated May 22, 2019, 1:20 p.m.)
Review request for mesos, Benjamin Mahler and Meng Zhu.
Changes
-------
Renamed `RoleInfo` into `Role`, avoiding non-const-reference alias `RoleInfo& roleInfo` and replacing `const RoleInfo& roleInfo` with `const Role& r`.
Summary (updated)
-----------------
Added `struct Role` to track role reservations and framework IDs.
Bugs: MESOS-9701
https://issues.apache.org/jira/browse/MESOS-9701
Repository: mesos
Description (updated)
-------
This patch introduces `struct Role` which contains the framework IDs
and reservatons tracked under a role. Two separate hashmaps used for
this purpose previously are replaced by a single hashmap, thus the need
to keep the two consistent with each other is removed. This simplifies
the role tracking logic in the allocator. The patch introduces minimal
to no performance impact.
Diffs (updated)
-----
src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
src/master/allocator/mesos/hierarchical.cpp 6df9a3b4ef470e57862815a862494cdb5e26d3e3
Diff: https://reviews.apache.org/r/70591/diff/6/
Changes: https://reviews.apache.org/r/70591/diff/5-6/
Testing
-------
make check
Benchmarking: 5 runs of `BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5` with the optimized build.
Performance impact in this benchmark seems to be negligible.
BEFORE:
Added 3000 agents in 51.553929ms
Added 3000 frameworks in 15.174748344secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.400805171secs
Made 0 allocation in 12.5850238secs
Added 3000 agents in 55.739336ms
Added 3000 frameworks in 14.730404769secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.563439682secs
Made 0 allocation in 13.063555055secs
Added 3000 agents in 54.414733ms
Added 3000 frameworks in 15.10136842secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 13.501664915secs
Made 0 allocation in 12.89034382secs
Added 3000 agents in 52.58252ms
Added 3000 frameworks in 14.048350298secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.299952145secs
Made 0 allocation in 11.888248811secs
Added 3000 agents in 52.821439ms
Added 3000 frameworks in 15.344450583secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.634000025secs
Made 0 allocation in 12.427171541secs
AFTER:
Added 3000 agents in 69.716648ms
Added 3000 frameworks in 15.249001979secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.860494226secs
Made 0 allocation in 12.228866329secs
Added 3000 agents in 52.639388ms
Added 3000 frameworks in 15.207895482secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.504777266secs
Made 0 allocation in 12.70388062secs
Added 3000 agents in 56.865794ms
Added 3000 frameworks in 15.284003915secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 13.86859815secs
Made 0 allocation in 12.538958231secs
Added 3000 agents in 56.028013ms
Added 3000 frameworks in 13.892577869secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.341724418secs
Made 0 allocation in 12.23022189secs
Added 3000 agents in 52.368219ms
Added 3000 frameworks in 13.978581104secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.701682501secs
Made 0 allocation in 12.141360313secs
Thanks,
Andrei Sekretenko
Re: Review Request 70591: Added `struct RoleInfo` to track role
reservations and framework IDs.
Posted by Meng Zhu <mz...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70591/#review215400
-----------------------------------------------------------
Ship it!
- Meng Zhu
On May 20, 2019, 9:26 a.m., Andrei Sekretenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70591/
> -----------------------------------------------------------
>
> (Updated May 20, 2019, 9:26 a.m.)
>
>
> Review request for mesos, Benjamin Mahler and Meng Zhu.
>
>
> Bugs: MESOS-9701
> https://issues.apache.org/jira/browse/MESOS-9701
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch introduces struct RoleInfo which contains the framework IDs
> and reservatons tracked under a role. Two separate hashmaps used for
> this purpose previously are replaced by a single hashmap, thus the need
> to keep the two consistent with each other is removed. This simplifies
> the role tracking logic in the allocator. The patch introduces minimal
> to no performance impact.
>
>
> Diffs
> -----
>
> src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
> src/master/allocator/mesos/hierarchical.cpp 6df9a3b4ef470e57862815a862494cdb5e26d3e3
>
>
> Diff: https://reviews.apache.org/r/70591/diff/5/
>
>
> Testing
> -------
>
> make check
>
> Benchmarking: 5 runs of `BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5` with the optimized build.
>
> Performance impact in this benchmark seems to be negligible.
>
> BEFORE:
> Added 3000 agents in 51.553929ms
> Added 3000 frameworks in 15.174748344secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.400805171secs
> Made 0 allocation in 12.5850238secs
>
> Added 3000 agents in 55.739336ms
> Added 3000 frameworks in 14.730404769secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.563439682secs
> Made 0 allocation in 13.063555055secs
>
> Added 3000 agents in 54.414733ms
> Added 3000 frameworks in 15.10136842secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.501664915secs
> Made 0 allocation in 12.89034382secs
>
> Added 3000 agents in 52.58252ms
> Added 3000 frameworks in 14.048350298secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.299952145secs
> Made 0 allocation in 11.888248811secs
>
> Added 3000 agents in 52.821439ms
> Added 3000 frameworks in 15.344450583secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.634000025secs
> Made 0 allocation in 12.427171541secs
>
>
> AFTER:
>
> Added 3000 agents in 69.716648ms
> Added 3000 frameworks in 15.249001979secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.860494226secs
> Made 0 allocation in 12.228866329secs
>
> Added 3000 agents in 52.639388ms
> Added 3000 frameworks in 15.207895482secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.504777266secs
> Made 0 allocation in 12.70388062secs
>
> Added 3000 agents in 56.865794ms
> Added 3000 frameworks in 15.284003915secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.86859815secs
> Made 0 allocation in 12.538958231secs
>
> Added 3000 agents in 56.028013ms
> Added 3000 frameworks in 13.892577869secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.341724418secs
> Made 0 allocation in 12.23022189secs
>
> Added 3000 agents in 52.368219ms
> Added 3000 frameworks in 13.978581104secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.701682501secs
> Made 0 allocation in 12.141360313secs
>
>
> Thanks,
>
> Andrei Sekretenko
>
>
Re: Review Request 70591: Added `struct RoleInfo` to track role
reservations and framework IDs.
Posted by Andrei Sekretenko <as...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70591/
-----------------------------------------------------------
(Updated May 20, 2019, 4:26 p.m.)
Review request for mesos, Benjamin Mahler and Meng Zhu.
Bugs: MESOS-9701
https://issues.apache.org/jira/browse/MESOS-9701
Repository: mesos
Description
-------
This patch introduces struct RoleInfo which contains the framework IDs
and reservatons tracked under a role. Two separate hashmaps used for
this purpose previously are replaced by a single hashmap, thus the need
to keep the two consistent with each other is removed. This simplifies
the role tracking logic in the allocator. The patch introduces minimal
to no performance impact.
Diffs (updated)
-----
src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
src/master/allocator/mesos/hierarchical.cpp 6df9a3b4ef470e57862815a862494cdb5e26d3e3
Diff: https://reviews.apache.org/r/70591/diff/5/
Changes: https://reviews.apache.org/r/70591/diff/4-5/
Testing
-------
make check
Benchmarking: 5 runs of `BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5` with the optimized build.
Performance impact in this benchmark seems to be negligible.
BEFORE:
Added 3000 agents in 51.553929ms
Added 3000 frameworks in 15.174748344secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.400805171secs
Made 0 allocation in 12.5850238secs
Added 3000 agents in 55.739336ms
Added 3000 frameworks in 14.730404769secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.563439682secs
Made 0 allocation in 13.063555055secs
Added 3000 agents in 54.414733ms
Added 3000 frameworks in 15.10136842secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 13.501664915secs
Made 0 allocation in 12.89034382secs
Added 3000 agents in 52.58252ms
Added 3000 frameworks in 14.048350298secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.299952145secs
Made 0 allocation in 11.888248811secs
Added 3000 agents in 52.821439ms
Added 3000 frameworks in 15.344450583secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.634000025secs
Made 0 allocation in 12.427171541secs
AFTER:
Added 3000 agents in 69.716648ms
Added 3000 frameworks in 15.249001979secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.860494226secs
Made 0 allocation in 12.228866329secs
Added 3000 agents in 52.639388ms
Added 3000 frameworks in 15.207895482secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.504777266secs
Made 0 allocation in 12.70388062secs
Added 3000 agents in 56.865794ms
Added 3000 frameworks in 15.284003915secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 13.86859815secs
Made 0 allocation in 12.538958231secs
Added 3000 agents in 56.028013ms
Added 3000 frameworks in 13.892577869secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.341724418secs
Made 0 allocation in 12.23022189secs
Added 3000 agents in 52.368219ms
Added 3000 frameworks in 13.978581104secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.701682501secs
Made 0 allocation in 12.141360313secs
Thanks,
Andrei Sekretenko
Re: Review Request 70591: Added `struct RoleInfo` to track role
reservations and framework IDs.
Posted by Andrei Sekretenko <as...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70591/
-----------------------------------------------------------
(Updated May 16, 2019, 2:16 p.m.)
Review request for mesos, Benjamin Mahler and Meng Zhu.
Bugs: MESOS-9701
https://issues.apache.org/jira/browse/MESOS-9701
Repository: mesos
Description
-------
This patch introduces struct RoleInfo which contains the framework IDs
and reservatons tracked under a role. Two separate hashmaps used for
this purpose previously are replaced by a single hashmap, thus the need
to keep the two consistent with each other is removed. This simplifies
the role tracking logic in the allocator. The patch introduces minimal
to no performance impact.
Diffs (updated)
-----
src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
src/master/allocator/mesos/hierarchical.cpp 875cfcd091f5f58afb89e752da5100a75828dd67
Diff: https://reviews.apache.org/r/70591/diff/4/
Changes: https://reviews.apache.org/r/70591/diff/3-4/
Testing
-------
make check
Benchmarking: 5 runs of `BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5` with the optimized build.
Performance impact in this benchmark seems to be negligible.
BEFORE:
Added 3000 agents in 51.553929ms
Added 3000 frameworks in 15.174748344secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.400805171secs
Made 0 allocation in 12.5850238secs
Added 3000 agents in 55.739336ms
Added 3000 frameworks in 14.730404769secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.563439682secs
Made 0 allocation in 13.063555055secs
Added 3000 agents in 54.414733ms
Added 3000 frameworks in 15.10136842secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 13.501664915secs
Made 0 allocation in 12.89034382secs
Added 3000 agents in 52.58252ms
Added 3000 frameworks in 14.048350298secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.299952145secs
Made 0 allocation in 11.888248811secs
Added 3000 agents in 52.821439ms
Added 3000 frameworks in 15.344450583secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.634000025secs
Made 0 allocation in 12.427171541secs
AFTER:
Added 3000 agents in 69.716648ms
Added 3000 frameworks in 15.249001979secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.860494226secs
Made 0 allocation in 12.228866329secs
Added 3000 agents in 52.639388ms
Added 3000 frameworks in 15.207895482secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.504777266secs
Made 0 allocation in 12.70388062secs
Added 3000 agents in 56.865794ms
Added 3000 frameworks in 15.284003915secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 13.86859815secs
Made 0 allocation in 12.538958231secs
Added 3000 agents in 56.028013ms
Added 3000 frameworks in 13.892577869secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.341724418secs
Made 0 allocation in 12.23022189secs
Added 3000 agents in 52.368219ms
Added 3000 frameworks in 13.978581104secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.701682501secs
Made 0 allocation in 12.141360313secs
Thanks,
Andrei Sekretenko
Re: Review Request 70591: Added `struct RoleInfo` to track role
reservations and framework IDs.
Posted by Andrei Sekretenko <as...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70591/
-----------------------------------------------------------
(Updated May 10, 2019, 4:31 p.m.)
Review request for mesos, Benjamin Mahler and Meng Zhu.
Bugs: MESOS-9701
https://issues.apache.org/jira/browse/MESOS-9701
Repository: mesos
Description (updated)
-------
This patch introduces struct RoleInfo which contains the framework IDs
and reservatons tracked under a role. Two separate hashmaps used for
this purpose previously are replaced by a single hashmap, thus the need
to keep the two consistent with each other is removed. This simplifies
the role tracking logic in the allocator. The patch introduces minimal
to no performance impact.
Diffs (updated)
-----
src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
src/master/allocator/mesos/hierarchical.cpp 64a076ddd29711437d539a06bb0470755828cc87
Diff: https://reviews.apache.org/r/70591/diff/3/
Changes: https://reviews.apache.org/r/70591/diff/2-3/
Testing
-------
make check
Benchmarking: 5 runs of `BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5` with the optimized build.
Performance impact in this benchmark seems to be negligible.
BEFORE:
Added 3000 agents in 51.553929ms
Added 3000 frameworks in 15.174748344secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.400805171secs
Made 0 allocation in 12.5850238secs
Added 3000 agents in 55.739336ms
Added 3000 frameworks in 14.730404769secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.563439682secs
Made 0 allocation in 13.063555055secs
Added 3000 agents in 54.414733ms
Added 3000 frameworks in 15.10136842secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 13.501664915secs
Made 0 allocation in 12.89034382secs
Added 3000 agents in 52.58252ms
Added 3000 frameworks in 14.048350298secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.299952145secs
Made 0 allocation in 11.888248811secs
Added 3000 agents in 52.821439ms
Added 3000 frameworks in 15.344450583secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.634000025secs
Made 0 allocation in 12.427171541secs
AFTER:
Added 3000 agents in 69.716648ms
Added 3000 frameworks in 15.249001979secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.860494226secs
Made 0 allocation in 12.228866329secs
Added 3000 agents in 52.639388ms
Added 3000 frameworks in 15.207895482secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.504777266secs
Made 0 allocation in 12.70388062secs
Added 3000 agents in 56.865794ms
Added 3000 frameworks in 15.284003915secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 13.86859815secs
Made 0 allocation in 12.538958231secs
Added 3000 agents in 56.028013ms
Added 3000 frameworks in 13.892577869secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.341724418secs
Made 0 allocation in 12.23022189secs
Added 3000 agents in 52.368219ms
Added 3000 frameworks in 13.978581104secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.701682501secs
Made 0 allocation in 12.141360313secs
Thanks,
Andrei Sekretenko