You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by Andrei Sekretenko <as...@mesosphere.io> on 2019/05/10 16:32:46 UTC
Re: Review Request 70618: Encapsulate a framework sorter inside a
RoleInfo.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70618/
-----------------------------------------------------------
(Updated May 10, 2019, 4:32 p.m.)
Review request for mesos, Benjamin Mahler and Meng Zhu.
Summary (updated)
-----------------
Encapsulate a framework sorter inside a RoleInfo.
Bugs: MESOS-9701
https://issues.apache.org/jira/browse/MESOS-9701
Repository: mesos
Description (updated)
-------
This patch consolidates the logic of tracking frameworks tied to
a specific role in the `RoleInfo`.
To do this properly, `RoleInfo` is turned into a class which wraps
access to the framework sorter so that no other entity is capable
of adding/removing clients (the frameworks) to/from the sorter.
The patch introduces almost no performance impact.
Diffs (updated)
-----
src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
src/master/allocator/mesos/hierarchical.cpp 64a076ddd29711437d539a06bb0470755828cc87
Diff: https://reviews.apache.org/r/70618/diff/2/
Changes: https://reviews.apache.org/r/70618/diff/1-2/
Testing
-------
make check
Benchmarking: 5 runs of BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5 with the optimized build.
BEFORE (master):
Added 3000 agents in 52.729305ms
Added 3000 frameworks in 13.799200231secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.560201008secs
Made 0 allocation in 12.143722849secs
Added 3000 agents in 64.789364ms
Added 3000 frameworks in 14.175436362secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.505877513secs
Made 0 allocation in 12.424587206secs
Added 3000 agents in 50.942631ms
Added 3000 frameworks in 14.26206239secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.220950624secs
Made 0 allocation in 12.495832704secs
Added 3000 agents in 50.660372ms
Added 3000 frameworks in 14.246438788secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.505903032secs
Made 0 allocation in 12.334731087secs
Added 3000 agents in 50.292757ms
Added 3000 frameworks in 14.236187327secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.531212952secs
Made 0 allocation in 12.282740343secs
-----------------------------------
AFTER (master + previous patch https://reviews.apache.org/r/70591/ + this patch):
Added 3000 agents in 51.465368ms
Added 3000 frameworks in 13.599017611secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.272279231secs
Made 0 allocation in 12.026509432secs
Added 3000 agents in 52.5567ms
Added 3000 frameworks in 13.67345101secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.447551363secs
Made 0 allocation in 12.045692187secs
Added 3000 agents in 52.455703ms
Added 3000 frameworks in 13.344338641secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 11.988632162secs
Made 0 allocation in 11.558150131secs
Added 3000 agents in 53.579201ms
Added 3000 frameworks in 13.681435728secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 11.966754231secs
Made 0 allocation in 12.720889223secs
Added 3000 agents in 52.08713ms
Added 3000 frameworks in 13.562008608secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.19201724secs
Made 0 allocation in 11.727037034secs
Thanks,
Andrei Sekretenko
Re: Review Request 70618: Store framework sorters inside RoleInfos.
Posted by Andrei Sekretenko <as...@mesosphere.io>.
> On May 14, 2019, 2:07 p.m., Meng Zhu wrote:
> > src/master/allocator/mesos/hierarchical.hpp
> > Lines 144-147 (patched)
> > <https://reviews.apache.org/r/70618/diff/2/?file=2144459#file2144459line144>
> >
> > (1) maybe we can create it in the e.g. constructor?
> >
> > (2) let's just use unique here
Switched to initialization in constructor, replaced Owned with unique_ptr.
- Andrei
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70618/#review215245
-----------------------------------------------------------
On May 16, 2019, 2:17 p.m., Andrei Sekretenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70618/
> -----------------------------------------------------------
>
> (Updated May 16, 2019, 2:17 p.m.)
>
>
> Review request for mesos, Benjamin Mahler and Meng Zhu.
>
>
> Bugs: MESOS-9701
> https://issues.apache.org/jira/browse/MESOS-9701
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch replaces a hashmap<Owned<Sorter>> used for tracking
> framework sorters with a unique_ptr<Sorter> inside of the RoleInfo.
> This simplifies the framework tracking logic and slightly improves
> performance.
>
>
> Diffs
> -----
>
> src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
> src/master/allocator/mesos/hierarchical.cpp 875cfcd091f5f58afb89e752da5100a75828dd67
>
>
> Diff: https://reviews.apache.org/r/70618/diff/3/
>
>
> Testing
> -------
>
> make check
>
> Benchmarking: 5 runs of BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5 with the optimized build.
>
> BEFORE (master):
>
> Added 3000 agents in 57.444301ms
> Added 3000 frameworks in 15.100499419secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.359926851secs
> Made 0 allocation in 12.147765014secs
>
> Added 3000 agents in 58.651887ms
> Added 3000 frameworks in 14.485925735secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.6526783secs
> Made 0 allocation in 12.138439924secs
>
> Added 3000 agents in 59.028581ms
> Added 3000 frameworks in 14.72945866secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.739135078secs
> Made 0 allocation in 12.673302496secs
>
> Added 3000 agents in 59.050577ms
> Added 3000 frameworks in 14.78273576secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.20400492secs
> Made 0 allocation in 13.289808943secs
>
> Added 3000 agents in 58.629888ms
> Added 3000 frameworks in 14.714786337secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.721094744secs
> Made 0 allocation in 12.688377237secs
>
> -----------------------------------
> AFTER (master + previous patch https://reviews.apache.org/r/70591/ + this patch):
>
> Added 3000 agents in 58.04155ms
> Added 3000 frameworks in 13.694651977secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.417541213secs
> Made 0 allocation in 12.130755905secs
>
> Added 3000 agents in 55.246813ms
> Added 3000 frameworks in 13.460479936secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.393765514secs
> Made 0 allocation in 12.196981426secs
>
> Added 3000 agents in 58.013477ms
> Added 3000 frameworks in 13.69361015secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.245916159secs
> Made 0 allocation in 11.699553888secs
>
> Added 3000 agents in 57.163681ms
> Added 3000 frameworks in 13.738218369secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.809206431secs
> Made 0 allocation in 12.400140164secs
>
> Added 3000 agents in 57.942087ms
> Added 3000 frameworks in 13.836390727secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.75595625secs
> Made 0 allocation in 11.939263998secs
>
>
> Thanks,
>
> Andrei Sekretenko
>
>
Re: Review Request 70618: Encapsulate a framework sorter inside a
RoleInfo.
Posted by Meng Zhu <mz...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70618/#review215245
-----------------------------------------------------------
src/master/allocator/mesos/hierarchical.hpp
Lines 113-116 (patched)
<https://reviews.apache.org/r/70618/#comment301876>
I thought we removed this TODO in previous patch?
src/master/allocator/mesos/hierarchical.hpp
Lines 117 (patched)
<https://reviews.apache.org/r/70618/#comment301886>
a space?
src/master/allocator/mesos/hierarchical.hpp
Lines 119-122 (patched)
<https://reviews.apache.org/r/70618/#comment301877>
This is a lot of code in the header.
Let's put the implementation in the cpp file.
src/master/allocator/mesos/hierarchical.hpp
Lines 144-147 (patched)
<https://reviews.apache.org/r/70618/#comment301885>
(1) maybe we can create it in the e.g. constructor?
(2) let's just use unique here
- Meng Zhu
On May 10, 2019, 9:32 a.m., Andrei Sekretenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70618/
> -----------------------------------------------------------
>
> (Updated May 10, 2019, 9:32 a.m.)
>
>
> Review request for mesos, Benjamin Mahler and Meng Zhu.
>
>
> Bugs: MESOS-9701
> https://issues.apache.org/jira/browse/MESOS-9701
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch consolidates the logic of tracking frameworks tied to
> a specific role in the `RoleInfo`.
>
> To do this properly, `RoleInfo` is turned into a class which wraps
> access to the framework sorter so that no other entity is capable
> of adding/removing clients (the frameworks) to/from the sorter.
>
> The patch introduces almost no performance impact.
>
>
> Diffs
> -----
>
> src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
> src/master/allocator/mesos/hierarchical.cpp 64a076ddd29711437d539a06bb0470755828cc87
>
>
> Diff: https://reviews.apache.org/r/70618/diff/2/
>
>
> Testing
> -------
>
> make check
>
> Benchmarking: 5 runs of BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5 with the optimized build.
>
> BEFORE (master):
>
> Added 3000 agents in 52.729305ms
> Added 3000 frameworks in 13.799200231secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.560201008secs
> Made 0 allocation in 12.143722849secs
>
> Added 3000 agents in 64.789364ms
> Added 3000 frameworks in 14.175436362secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.505877513secs
> Made 0 allocation in 12.424587206secs
>
> Added 3000 agents in 50.942631ms
> Added 3000 frameworks in 14.26206239secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.220950624secs
> Made 0 allocation in 12.495832704secs
>
> Added 3000 agents in 50.660372ms
> Added 3000 frameworks in 14.246438788secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.505903032secs
> Made 0 allocation in 12.334731087secs
>
> Added 3000 agents in 50.292757ms
> Added 3000 frameworks in 14.236187327secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.531212952secs
> Made 0 allocation in 12.282740343secs
>
> -----------------------------------
> AFTER (master + previous patch https://reviews.apache.org/r/70591/ + this patch):
>
> Added 3000 agents in 51.465368ms
> Added 3000 frameworks in 13.599017611secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.272279231secs
> Made 0 allocation in 12.026509432secs
>
> Added 3000 agents in 52.5567ms
> Added 3000 frameworks in 13.67345101secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.447551363secs
> Made 0 allocation in 12.045692187secs
>
> Added 3000 agents in 52.455703ms
> Added 3000 frameworks in 13.344338641secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 11.988632162secs
> Made 0 allocation in 11.558150131secs
>
> Added 3000 agents in 53.579201ms
> Added 3000 frameworks in 13.681435728secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 11.966754231secs
> Made 0 allocation in 12.720889223secs
>
> Added 3000 agents in 52.08713ms
> Added 3000 frameworks in 13.562008608secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.19201724secs
> Made 0 allocation in 11.727037034secs
>
>
> Thanks,
>
> Andrei Sekretenko
>
>
Re: Review Request 70618: Encapsulate a framework sorter inside a
RoleInfo.
Posted by Mesos Reviewbot <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70618/#review215194
-----------------------------------------------------------
Patch looks great!
Reviews applied: [70626, 70591, 70618]
Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' CONFIGURATION='--verbose --disable-libtool-wrappers --disable-parallel-test-execution' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; ./support/docker-build.sh
- Mesos Reviewbot
On May 10, 2019, 4:32 p.m., Andrei Sekretenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70618/
> -----------------------------------------------------------
>
> (Updated May 10, 2019, 4:32 p.m.)
>
>
> Review request for mesos, Benjamin Mahler and Meng Zhu.
>
>
> Bugs: MESOS-9701
> https://issues.apache.org/jira/browse/MESOS-9701
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch consolidates the logic of tracking frameworks tied to
> a specific role in the `RoleInfo`.
>
> To do this properly, `RoleInfo` is turned into a class which wraps
> access to the framework sorter so that no other entity is capable
> of adding/removing clients (the frameworks) to/from the sorter.
>
> The patch introduces almost no performance impact.
>
>
> Diffs
> -----
>
> src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
> src/master/allocator/mesos/hierarchical.cpp 64a076ddd29711437d539a06bb0470755828cc87
>
>
> Diff: https://reviews.apache.org/r/70618/diff/2/
>
>
> Testing
> -------
>
> make check
>
> Benchmarking: 5 runs of BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5 with the optimized build.
>
> BEFORE (master):
>
> Added 3000 agents in 52.729305ms
> Added 3000 frameworks in 13.799200231secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.560201008secs
> Made 0 allocation in 12.143722849secs
>
> Added 3000 agents in 64.789364ms
> Added 3000 frameworks in 14.175436362secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.505877513secs
> Made 0 allocation in 12.424587206secs
>
> Added 3000 agents in 50.942631ms
> Added 3000 frameworks in 14.26206239secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.220950624secs
> Made 0 allocation in 12.495832704secs
>
> Added 3000 agents in 50.660372ms
> Added 3000 frameworks in 14.246438788secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.505903032secs
> Made 0 allocation in 12.334731087secs
>
> Added 3000 agents in 50.292757ms
> Added 3000 frameworks in 14.236187327secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.531212952secs
> Made 0 allocation in 12.282740343secs
>
> -----------------------------------
> AFTER (master + previous patch https://reviews.apache.org/r/70591/ + this patch):
>
> Added 3000 agents in 51.465368ms
> Added 3000 frameworks in 13.599017611secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.272279231secs
> Made 0 allocation in 12.026509432secs
>
> Added 3000 agents in 52.5567ms
> Added 3000 frameworks in 13.67345101secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.447551363secs
> Made 0 allocation in 12.045692187secs
>
> Added 3000 agents in 52.455703ms
> Added 3000 frameworks in 13.344338641secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 11.988632162secs
> Made 0 allocation in 11.558150131secs
>
> Added 3000 agents in 53.579201ms
> Added 3000 frameworks in 13.681435728secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 11.966754231secs
> Made 0 allocation in 12.720889223secs
>
> Added 3000 agents in 52.08713ms
> Added 3000 frameworks in 13.562008608secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.19201724secs
> Made 0 allocation in 11.727037034secs
>
>
> Thanks,
>
> Andrei Sekretenko
>
>
Re: Review Request 70618: Store framework sorters inside RoleInfos.
Posted by Andrei Sekretenko <as...@mesosphere.io>.
> On May 21, 2019, 10:01 a.m., Meng Zhu wrote:
> > After some thought, I think we should not do the refactoring at the moment. I think a good next step is to deprecate the "sorter" such that each `roleInfo` does not need to have a sorter instance. The fact that when constructing a `RoleInfo` we need to take in global args `options` and add all slave resources is unfortunate. This also prevents RoleInfo from having a default constructor which prevents us from using the map insertion interface.
> >
> > But there are some good points in this patch (e.g. comments fix), lets pull those out if possible.
Pulled the comment fix into https://reviews.apache.org/r/70700
- Andrei
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70618/#review215402
-----------------------------------------------------------
On May 20, 2019, 4:25 p.m., Andrei Sekretenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70618/
> -----------------------------------------------------------
>
> (Updated May 20, 2019, 4:25 p.m.)
>
>
> Review request for mesos, Benjamin Mahler and Meng Zhu.
>
>
> Bugs: MESOS-9701
> https://issues.apache.org/jira/browse/MESOS-9701
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch replaces a hashmap<Owned<Sorter>> used for tracking
> framework sorters with a unique_ptr<Sorter> inside of the RoleInfo.
> This simplifies the framework tracking logic and slightly improves
> performance.
>
>
> Diffs
> -----
>
> src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
> src/master/allocator/mesos/hierarchical.cpp 6df9a3b4ef470e57862815a862494cdb5e26d3e3
>
>
> Diff: https://reviews.apache.org/r/70618/diff/4/
>
>
> Testing
> -------
>
> make check
>
> Benchmarking: 5 runs of BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5 with the optimized build.
>
> BEFORE (master):
>
> Added 3000 agents in 57.444301ms
> Added 3000 frameworks in 15.100499419secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.359926851secs
> Made 0 allocation in 12.147765014secs
>
> Added 3000 agents in 58.651887ms
> Added 3000 frameworks in 14.485925735secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.6526783secs
> Made 0 allocation in 12.138439924secs
>
> Added 3000 agents in 59.028581ms
> Added 3000 frameworks in 14.72945866secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.739135078secs
> Made 0 allocation in 12.673302496secs
>
> Added 3000 agents in 59.050577ms
> Added 3000 frameworks in 14.78273576secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.20400492secs
> Made 0 allocation in 13.289808943secs
>
> Added 3000 agents in 58.629888ms
> Added 3000 frameworks in 14.714786337secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.721094744secs
> Made 0 allocation in 12.688377237secs
>
> -----------------------------------
> AFTER (master + previous patch https://reviews.apache.org/r/70591/ + this patch):
>
> Added 3000 agents in 58.04155ms
> Added 3000 frameworks in 13.694651977secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.417541213secs
> Made 0 allocation in 12.130755905secs
>
> Added 3000 agents in 55.246813ms
> Added 3000 frameworks in 13.460479936secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.393765514secs
> Made 0 allocation in 12.196981426secs
>
> Added 3000 agents in 58.013477ms
> Added 3000 frameworks in 13.69361015secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.245916159secs
> Made 0 allocation in 11.699553888secs
>
> Added 3000 agents in 57.163681ms
> Added 3000 frameworks in 13.738218369secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.809206431secs
> Made 0 allocation in 12.400140164secs
>
> Added 3000 agents in 57.942087ms
> Added 3000 frameworks in 13.836390727secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.75595625secs
> Made 0 allocation in 11.939263998secs
>
>
> Thanks,
>
> Andrei Sekretenko
>
>
Re: Review Request 70618: Store framework sorters inside RoleInfos.
Posted by Meng Zhu <mz...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70618/#review215402
-----------------------------------------------------------
After some thought, I think we should not do the refactoring at the moment. I think a good next step is to deprecate the "sorter" such that each `roleInfo` does not need to have a sorter instance. The fact that when constructing a `RoleInfo` we need to take in global args `options` and add all slave resources is unfortunate. This also prevents RoleInfo from having a default constructor which prevents us from using the map insertion interface.
But there are some good points in this patch (e.g. comments fix), lets pull those out if possible.
- Meng Zhu
On May 20, 2019, 9:25 a.m., Andrei Sekretenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70618/
> -----------------------------------------------------------
>
> (Updated May 20, 2019, 9:25 a.m.)
>
>
> Review request for mesos, Benjamin Mahler and Meng Zhu.
>
>
> Bugs: MESOS-9701
> https://issues.apache.org/jira/browse/MESOS-9701
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch replaces a hashmap<Owned<Sorter>> used for tracking
> framework sorters with a unique_ptr<Sorter> inside of the RoleInfo.
> This simplifies the framework tracking logic and slightly improves
> performance.
>
>
> Diffs
> -----
>
> src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
> src/master/allocator/mesos/hierarchical.cpp 6df9a3b4ef470e57862815a862494cdb5e26d3e3
>
>
> Diff: https://reviews.apache.org/r/70618/diff/4/
>
>
> Testing
> -------
>
> make check
>
> Benchmarking: 5 runs of BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5 with the optimized build.
>
> BEFORE (master):
>
> Added 3000 agents in 57.444301ms
> Added 3000 frameworks in 15.100499419secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.359926851secs
> Made 0 allocation in 12.147765014secs
>
> Added 3000 agents in 58.651887ms
> Added 3000 frameworks in 14.485925735secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.6526783secs
> Made 0 allocation in 12.138439924secs
>
> Added 3000 agents in 59.028581ms
> Added 3000 frameworks in 14.72945866secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.739135078secs
> Made 0 allocation in 12.673302496secs
>
> Added 3000 agents in 59.050577ms
> Added 3000 frameworks in 14.78273576secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.20400492secs
> Made 0 allocation in 13.289808943secs
>
> Added 3000 agents in 58.629888ms
> Added 3000 frameworks in 14.714786337secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.721094744secs
> Made 0 allocation in 12.688377237secs
>
> -----------------------------------
> AFTER (master + previous patch https://reviews.apache.org/r/70591/ + this patch):
>
> Added 3000 agents in 58.04155ms
> Added 3000 frameworks in 13.694651977secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.417541213secs
> Made 0 allocation in 12.130755905secs
>
> Added 3000 agents in 55.246813ms
> Added 3000 frameworks in 13.460479936secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.393765514secs
> Made 0 allocation in 12.196981426secs
>
> Added 3000 agents in 58.013477ms
> Added 3000 frameworks in 13.69361015secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.245916159secs
> Made 0 allocation in 11.699553888secs
>
> Added 3000 agents in 57.163681ms
> Added 3000 frameworks in 13.738218369secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.809206431secs
> Made 0 allocation in 12.400140164secs
>
> Added 3000 agents in 57.942087ms
> Added 3000 frameworks in 13.836390727secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.75595625secs
> Made 0 allocation in 11.939263998secs
>
>
> Thanks,
>
> Andrei Sekretenko
>
>
Re: Review Request 70618: Store framework sorters inside RoleInfos.
Posted by Mesos Reviewbot <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70618/#review215376
-----------------------------------------------------------
Patch looks great!
Reviews applied: [70626, 70591, 70679, 70618]
Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' CONFIGURATION='--verbose --disable-libtool-wrappers --disable-parallel-test-execution' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; ./support/docker-build.sh
- Mesos Reviewbot
On May 20, 2019, 9:25 a.m., Andrei Sekretenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70618/
> -----------------------------------------------------------
>
> (Updated May 20, 2019, 9:25 a.m.)
>
>
> Review request for mesos, Benjamin Mahler and Meng Zhu.
>
>
> Bugs: MESOS-9701
> https://issues.apache.org/jira/browse/MESOS-9701
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch replaces a hashmap<Owned<Sorter>> used for tracking
> framework sorters with a unique_ptr<Sorter> inside of the RoleInfo.
> This simplifies the framework tracking logic and slightly improves
> performance.
>
>
> Diffs
> -----
>
> src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
> src/master/allocator/mesos/hierarchical.cpp 6df9a3b4ef470e57862815a862494cdb5e26d3e3
>
>
> Diff: https://reviews.apache.org/r/70618/diff/4/
>
>
> Testing
> -------
>
> make check
>
> Benchmarking: 5 runs of BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5 with the optimized build.
>
> BEFORE (master):
>
> Added 3000 agents in 57.444301ms
> Added 3000 frameworks in 15.100499419secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.359926851secs
> Made 0 allocation in 12.147765014secs
>
> Added 3000 agents in 58.651887ms
> Added 3000 frameworks in 14.485925735secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.6526783secs
> Made 0 allocation in 12.138439924secs
>
> Added 3000 agents in 59.028581ms
> Added 3000 frameworks in 14.72945866secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.739135078secs
> Made 0 allocation in 12.673302496secs
>
> Added 3000 agents in 59.050577ms
> Added 3000 frameworks in 14.78273576secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.20400492secs
> Made 0 allocation in 13.289808943secs
>
> Added 3000 agents in 58.629888ms
> Added 3000 frameworks in 14.714786337secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.721094744secs
> Made 0 allocation in 12.688377237secs
>
> -----------------------------------
> AFTER (master + previous patch https://reviews.apache.org/r/70591/ + this patch):
>
> Added 3000 agents in 58.04155ms
> Added 3000 frameworks in 13.694651977secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.417541213secs
> Made 0 allocation in 12.130755905secs
>
> Added 3000 agents in 55.246813ms
> Added 3000 frameworks in 13.460479936secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.393765514secs
> Made 0 allocation in 12.196981426secs
>
> Added 3000 agents in 58.013477ms
> Added 3000 frameworks in 13.69361015secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.245916159secs
> Made 0 allocation in 11.699553888secs
>
> Added 3000 agents in 57.163681ms
> Added 3000 frameworks in 13.738218369secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.809206431secs
> Made 0 allocation in 12.400140164secs
>
> Added 3000 agents in 57.942087ms
> Added 3000 frameworks in 13.836390727secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.75595625secs
> Made 0 allocation in 11.939263998secs
>
>
> Thanks,
>
> Andrei Sekretenko
>
>
Re: Review Request 70618: Store framework sorters inside RoleInfos.
Posted by Mesos Reviewbot <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70618/#review215469
-----------------------------------------------------------
Bad review!
Reviews applied: [70618, 70679, 70591, 70626]
Error:
2019-05-22 19:11:14 URL:https://reviews.apache.org/r/70626/diff/raw/ [1191/1191] -> "70626.patch" [1]
error: patch failed: src/master/allocator/mesos/hierarchical.cpp:2653
error: src/master/allocator/mesos/hierarchical.cpp: patch does not apply
- Mesos Reviewbot
On May 20, 2019, 4:25 p.m., Andrei Sekretenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70618/
> -----------------------------------------------------------
>
> (Updated May 20, 2019, 4:25 p.m.)
>
>
> Review request for mesos, Benjamin Mahler and Meng Zhu.
>
>
> Bugs: MESOS-9701
> https://issues.apache.org/jira/browse/MESOS-9701
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch replaces a hashmap<Owned<Sorter>> used for tracking
> framework sorters with a unique_ptr<Sorter> inside of the RoleInfo.
> This simplifies the framework tracking logic and slightly improves
> performance.
>
>
> Diffs
> -----
>
> src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
> src/master/allocator/mesos/hierarchical.cpp 6df9a3b4ef470e57862815a862494cdb5e26d3e3
>
>
> Diff: https://reviews.apache.org/r/70618/diff/4/
>
>
> Testing
> -------
>
> make check
>
> Benchmarking: 5 runs of BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5 with the optimized build.
>
> BEFORE (master):
>
> Added 3000 agents in 57.444301ms
> Added 3000 frameworks in 15.100499419secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.359926851secs
> Made 0 allocation in 12.147765014secs
>
> Added 3000 agents in 58.651887ms
> Added 3000 frameworks in 14.485925735secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.6526783secs
> Made 0 allocation in 12.138439924secs
>
> Added 3000 agents in 59.028581ms
> Added 3000 frameworks in 14.72945866secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.739135078secs
> Made 0 allocation in 12.673302496secs
>
> Added 3000 agents in 59.050577ms
> Added 3000 frameworks in 14.78273576secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.20400492secs
> Made 0 allocation in 13.289808943secs
>
> Added 3000 agents in 58.629888ms
> Added 3000 frameworks in 14.714786337secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.721094744secs
> Made 0 allocation in 12.688377237secs
>
> -----------------------------------
> AFTER (master + previous patch https://reviews.apache.org/r/70591/ + this patch):
>
> Added 3000 agents in 58.04155ms
> Added 3000 frameworks in 13.694651977secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.417541213secs
> Made 0 allocation in 12.130755905secs
>
> Added 3000 agents in 55.246813ms
> Added 3000 frameworks in 13.460479936secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.393765514secs
> Made 0 allocation in 12.196981426secs
>
> Added 3000 agents in 58.013477ms
> Added 3000 frameworks in 13.69361015secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.245916159secs
> Made 0 allocation in 11.699553888secs
>
> Added 3000 agents in 57.163681ms
> Added 3000 frameworks in 13.738218369secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.809206431secs
> Made 0 allocation in 12.400140164secs
>
> Added 3000 agents in 57.942087ms
> Added 3000 frameworks in 13.836390727secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.75595625secs
> Made 0 allocation in 11.939263998secs
>
>
> Thanks,
>
> Andrei Sekretenko
>
>
Re: Review Request 70618: Store framework sorters inside RoleInfos.
Posted by Andrei Sekretenko <as...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70618/
-----------------------------------------------------------
(Updated May 20, 2019, 4:25 p.m.)
Review request for mesos, Benjamin Mahler and Meng Zhu.
Bugs: MESOS-9701
https://issues.apache.org/jira/browse/MESOS-9701
Repository: mesos
Description
-------
This patch replaces a hashmap<Owned<Sorter>> used for tracking
framework sorters with a unique_ptr<Sorter> inside of the RoleInfo.
This simplifies the framework tracking logic and slightly improves
performance.
Diffs (updated)
-----
src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
src/master/allocator/mesos/hierarchical.cpp 6df9a3b4ef470e57862815a862494cdb5e26d3e3
Diff: https://reviews.apache.org/r/70618/diff/4/
Changes: https://reviews.apache.org/r/70618/diff/3-4/
Testing
-------
make check
Benchmarking: 5 runs of BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5 with the optimized build.
BEFORE (master):
Added 3000 agents in 57.444301ms
Added 3000 frameworks in 15.100499419secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 13.359926851secs
Made 0 allocation in 12.147765014secs
Added 3000 agents in 58.651887ms
Added 3000 frameworks in 14.485925735secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 13.6526783secs
Made 0 allocation in 12.138439924secs
Added 3000 agents in 59.028581ms
Added 3000 frameworks in 14.72945866secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.739135078secs
Made 0 allocation in 12.673302496secs
Added 3000 agents in 59.050577ms
Added 3000 frameworks in 14.78273576secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 13.20400492secs
Made 0 allocation in 13.289808943secs
Added 3000 agents in 58.629888ms
Added 3000 frameworks in 14.714786337secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 13.721094744secs
Made 0 allocation in 12.688377237secs
-----------------------------------
AFTER (master + previous patch https://reviews.apache.org/r/70591/ + this patch):
Added 3000 agents in 58.04155ms
Added 3000 frameworks in 13.694651977secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.417541213secs
Made 0 allocation in 12.130755905secs
Added 3000 agents in 55.246813ms
Added 3000 frameworks in 13.460479936secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.393765514secs
Made 0 allocation in 12.196981426secs
Added 3000 agents in 58.013477ms
Added 3000 frameworks in 13.69361015secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.245916159secs
Made 0 allocation in 11.699553888secs
Added 3000 agents in 57.163681ms
Added 3000 frameworks in 13.738218369secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.809206431secs
Made 0 allocation in 12.400140164secs
Added 3000 agents in 57.942087ms
Added 3000 frameworks in 13.836390727secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.75595625secs
Made 0 allocation in 11.939263998secs
Thanks,
Andrei Sekretenko
Re: Review Request 70618: Store framework sorters inside RoleInfos.
Posted by Andrei Sekretenko <as...@mesosphere.io>.
> On May 20, 2019, 12:18 p.m., Meng Zhu wrote:
> > src/master/allocator/mesos/hierarchical.cpp
> > Line 367 (original), 367 (patched)
> > <https://reviews.apache.org/r/70618/diff/3/?file=2145504#file2145504line367>
> >
> > Not yours, but this could be just a reference?
> >
> > const hashmap<SlaveID, Resources>& allocation = ...
> >
> > Want to add a quick patch for that?
It turns out that such change breaks a lot of tests.
Indeed, the code below is modifying the hashmap reference to which is returned by `frameworkSorter->allocation()`, but I could not immediately understand what _exactly_ breaks. I'll have a deeper look some time later.
> On May 20, 2019, 12:18 p.m., Meng Zhu wrote:
> > src/master/allocator/mesos/hierarchical.cpp
> > Line 1853 (original), 1853 (patched)
> > <https://reviews.apache.org/r/70618/diff/3/?file=2145504#file2145504line1859>
> >
> > Not yours, but this comment is tied to DRF, we should remove/modify it.
Fixed in preceding https://reviews.apache.org/r/70679/
- Andrei
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70618/#review215354
-----------------------------------------------------------
On May 20, 2019, 4:25 p.m., Andrei Sekretenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70618/
> -----------------------------------------------------------
>
> (Updated May 20, 2019, 4:25 p.m.)
>
>
> Review request for mesos, Benjamin Mahler and Meng Zhu.
>
>
> Bugs: MESOS-9701
> https://issues.apache.org/jira/browse/MESOS-9701
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch replaces a hashmap<Owned<Sorter>> used for tracking
> framework sorters with a unique_ptr<Sorter> inside of the RoleInfo.
> This simplifies the framework tracking logic and slightly improves
> performance.
>
>
> Diffs
> -----
>
> src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
> src/master/allocator/mesos/hierarchical.cpp 6df9a3b4ef470e57862815a862494cdb5e26d3e3
>
>
> Diff: https://reviews.apache.org/r/70618/diff/4/
>
>
> Testing
> -------
>
> make check
>
> Benchmarking: 5 runs of BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5 with the optimized build.
>
> BEFORE (master):
>
> Added 3000 agents in 57.444301ms
> Added 3000 frameworks in 15.100499419secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.359926851secs
> Made 0 allocation in 12.147765014secs
>
> Added 3000 agents in 58.651887ms
> Added 3000 frameworks in 14.485925735secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.6526783secs
> Made 0 allocation in 12.138439924secs
>
> Added 3000 agents in 59.028581ms
> Added 3000 frameworks in 14.72945866secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.739135078secs
> Made 0 allocation in 12.673302496secs
>
> Added 3000 agents in 59.050577ms
> Added 3000 frameworks in 14.78273576secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.20400492secs
> Made 0 allocation in 13.289808943secs
>
> Added 3000 agents in 58.629888ms
> Added 3000 frameworks in 14.714786337secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.721094744secs
> Made 0 allocation in 12.688377237secs
>
> -----------------------------------
> AFTER (master + previous patch https://reviews.apache.org/r/70591/ + this patch):
>
> Added 3000 agents in 58.04155ms
> Added 3000 frameworks in 13.694651977secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.417541213secs
> Made 0 allocation in 12.130755905secs
>
> Added 3000 agents in 55.246813ms
> Added 3000 frameworks in 13.460479936secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.393765514secs
> Made 0 allocation in 12.196981426secs
>
> Added 3000 agents in 58.013477ms
> Added 3000 frameworks in 13.69361015secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.245916159secs
> Made 0 allocation in 11.699553888secs
>
> Added 3000 agents in 57.163681ms
> Added 3000 frameworks in 13.738218369secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.809206431secs
> Made 0 allocation in 12.400140164secs
>
> Added 3000 agents in 57.942087ms
> Added 3000 frameworks in 13.836390727secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.75595625secs
> Made 0 allocation in 11.939263998secs
>
>
> Thanks,
>
> Andrei Sekretenko
>
>
Re: Review Request 70618: Store framework sorters inside RoleInfos.
Posted by Andrei Sekretenko <as...@mesosphere.io>.
> On May 20, 2019, 12:18 p.m., Meng Zhu wrote:
> > src/master/allocator/mesos/hierarchical.cpp
> > Lines 1708-1709 (original), 1706-1709 (patched)
> > <https://reviews.apache.org/r/70618/diff/3/?file=2145504#file2145504line1712>
> >
> > The logic here is not related to this patch.
> > I am OK with either doing this in the previous patch or keep it as is.
I cannot keep it as it is - RoleInfo has no default constructor now (for the purpose of maintaining RoleInfo's only - as for now - class invariant: that it always has a framework sorter).
On the other side, doing this change in the previous patch will improve readability, I'll do that.
- Andrei
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70618/#review215354
-----------------------------------------------------------
On May 16, 2019, 2:17 p.m., Andrei Sekretenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70618/
> -----------------------------------------------------------
>
> (Updated May 16, 2019, 2:17 p.m.)
>
>
> Review request for mesos, Benjamin Mahler and Meng Zhu.
>
>
> Bugs: MESOS-9701
> https://issues.apache.org/jira/browse/MESOS-9701
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch replaces a hashmap<Owned<Sorter>> used for tracking
> framework sorters with a unique_ptr<Sorter> inside of the RoleInfo.
> This simplifies the framework tracking logic and slightly improves
> performance.
>
>
> Diffs
> -----
>
> src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
> src/master/allocator/mesos/hierarchical.cpp 875cfcd091f5f58afb89e752da5100a75828dd67
>
>
> Diff: https://reviews.apache.org/r/70618/diff/3/
>
>
> Testing
> -------
>
> make check
>
> Benchmarking: 5 runs of BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5 with the optimized build.
>
> BEFORE (master):
>
> Added 3000 agents in 57.444301ms
> Added 3000 frameworks in 15.100499419secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.359926851secs
> Made 0 allocation in 12.147765014secs
>
> Added 3000 agents in 58.651887ms
> Added 3000 frameworks in 14.485925735secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.6526783secs
> Made 0 allocation in 12.138439924secs
>
> Added 3000 agents in 59.028581ms
> Added 3000 frameworks in 14.72945866secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.739135078secs
> Made 0 allocation in 12.673302496secs
>
> Added 3000 agents in 59.050577ms
> Added 3000 frameworks in 14.78273576secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.20400492secs
> Made 0 allocation in 13.289808943secs
>
> Added 3000 agents in 58.629888ms
> Added 3000 frameworks in 14.714786337secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.721094744secs
> Made 0 allocation in 12.688377237secs
>
> -----------------------------------
> AFTER (master + previous patch https://reviews.apache.org/r/70591/ + this patch):
>
> Added 3000 agents in 58.04155ms
> Added 3000 frameworks in 13.694651977secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.417541213secs
> Made 0 allocation in 12.130755905secs
>
> Added 3000 agents in 55.246813ms
> Added 3000 frameworks in 13.460479936secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.393765514secs
> Made 0 allocation in 12.196981426secs
>
> Added 3000 agents in 58.013477ms
> Added 3000 frameworks in 13.69361015secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.245916159secs
> Made 0 allocation in 11.699553888secs
>
> Added 3000 agents in 57.163681ms
> Added 3000 frameworks in 13.738218369secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.809206431secs
> Made 0 allocation in 12.400140164secs
>
> Added 3000 agents in 57.942087ms
> Added 3000 frameworks in 13.836390727secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.75595625secs
> Made 0 allocation in 11.939263998secs
>
>
> Thanks,
>
> Andrei Sekretenko
>
>
Re: Review Request 70618: Store framework sorters inside RoleInfos.
Posted by Meng Zhu <mz...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70618/#review215354
-----------------------------------------------------------
Just a partial review, will post the rest later
src/master/allocator/mesos/hierarchical.hpp
Line 110 (original), 111 (patched)
<https://reviews.apache.org/r/70618/#comment302016>
space before brace
Did I mention this in the last patch? Did you rebase?
src/master/allocator/mesos/hierarchical.hpp
Lines 112 (patched)
<https://reviews.apache.org/r/70618/#comment302017>
You have a user-defined constructor declared below. So this is not strictly needed. And (correct me if I am wrong) I do not think our style guide requires this, I would suggest removing this for brevity.
src/master/allocator/mesos/hierarchical.hpp
Lines 115 (patched)
<https://reviews.apache.org/r/70618/#comment302018>
Put this in the line above.
src/master/allocator/mesos/hierarchical.hpp
Lines 118-120 (original), 128-133 (patched)
<https://reviews.apache.org/r/70618/#comment302019>
The original comment is verbose and outdated. Let's take this opportunity to improve it. Also, let's make an effort to align the comment length :)
- Let's remove the ties to DRF (as it can be any sorter algorithm)
- Let's remove the details of level and stage, readers who are familiar with
the matter do not need to comment, those who are not will just be confused. Also,
the structure of the code could change in the future.
- The comment regarding resource pool is no longer true due to:
https://github.com/apache/mesos/commit/a3626219b402ff837192208b6d35946b3a069ce6
How about:
// The sorter determines the resource allocation order of the
// frameworks subscribed under this role.
src/master/allocator/mesos/hierarchical.cpp
Lines 358-359 (original), 358-359 (patched)
<https://reviews.apache.org/r/70618/#comment302020>
// FrameworkID may not be present in RoleInfo because the
// framework was previously deactivated and never re-added.
src/master/allocator/mesos/hierarchical.cpp
Line 367 (original), 367 (patched)
<https://reviews.apache.org/r/70618/#comment302021>
Not yours, but this could be just a reference?
const hashmap<SlaveID, Resources>& allocation = ...
Want to add a quick patch for that?
src/master/allocator/mesos/hierarchical.cpp
Line 888 (original), 888 (patched)
<https://reviews.apache.org/r/70618/#comment302022>
hmm, it is unfortunate that we need to make a copy here due to the sorter interface....
src/master/allocator/mesos/hierarchical.cpp
Lines 982-1000 (original), 981-999 (patched)
<https://reviews.apache.org/r/70618/#comment302023>
We should just use `ResourceQuantities` for these.
Feel free to do this in another patch.
src/master/allocator/mesos/hierarchical.cpp
Line 1369 (original), 1367 (patched)
<https://reviews.apache.org/r/70618/#comment302024>
suppressingRoles
src/master/allocator/mesos/hierarchical.cpp
Line 1373 (original), 1372 (patched)
<https://reviews.apache.org/r/70618/#comment302025>
newline not necessary?
src/master/allocator/mesos/hierarchical.cpp
Line 1395 (original), 1393 (patched)
<https://reviews.apache.org/r/70618/#comment302026>
revivingRoles
src/master/allocator/mesos/hierarchical.cpp
Line 1402 (original), 1401 (patched)
<https://reviews.apache.org/r/70618/#comment302027>
newline not necessary?
src/master/allocator/mesos/hierarchical.cpp
Lines 1708-1709 (original), 1706-1709 (patched)
<https://reviews.apache.org/r/70618/#comment302028>
The logic here is not related to this patch.
I am OK with either doing this in the previous patch or keep it as is.
src/master/allocator/mesos/hierarchical.cpp
Line 1714 (original), 1714 (patched)
<https://reviews.apache.org/r/70618/#comment302030>
This should be in the previous patch (I just raised an issue in the patch)
src/master/allocator/mesos/hierarchical.cpp
Line 1853 (original), 1853 (patched)
<https://reviews.apache.org/r/70618/#comment302032>
Not yours, but this comment is tied to DRF, we should remove/modify it.
src/master/allocator/mesos/hierarchical.cpp
Lines 2169 (patched)
<https://reviews.apache.org/r/70618/#comment302034>
space before braces
src/master/allocator/mesos/hierarchical.cpp
Lines 2171 (patched)
<https://reviews.apache.org/r/70618/#comment302033>
newline here
- Meng Zhu
On May 16, 2019, 7:17 a.m., Andrei Sekretenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70618/
> -----------------------------------------------------------
>
> (Updated May 16, 2019, 7:17 a.m.)
>
>
> Review request for mesos, Benjamin Mahler and Meng Zhu.
>
>
> Bugs: MESOS-9701
> https://issues.apache.org/jira/browse/MESOS-9701
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch replaces a hashmap<Owned<Sorter>> used for tracking
> framework sorters with a unique_ptr<Sorter> inside of the RoleInfo.
> This simplifies the framework tracking logic and slightly improves
> performance.
>
>
> Diffs
> -----
>
> src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
> src/master/allocator/mesos/hierarchical.cpp 875cfcd091f5f58afb89e752da5100a75828dd67
>
>
> Diff: https://reviews.apache.org/r/70618/diff/3/
>
>
> Testing
> -------
>
> make check
>
> Benchmarking: 5 runs of BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5 with the optimized build.
>
> BEFORE (master):
>
> Added 3000 agents in 57.444301ms
> Added 3000 frameworks in 15.100499419secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.359926851secs
> Made 0 allocation in 12.147765014secs
>
> Added 3000 agents in 58.651887ms
> Added 3000 frameworks in 14.485925735secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.6526783secs
> Made 0 allocation in 12.138439924secs
>
> Added 3000 agents in 59.028581ms
> Added 3000 frameworks in 14.72945866secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.739135078secs
> Made 0 allocation in 12.673302496secs
>
> Added 3000 agents in 59.050577ms
> Added 3000 frameworks in 14.78273576secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.20400492secs
> Made 0 allocation in 13.289808943secs
>
> Added 3000 agents in 58.629888ms
> Added 3000 frameworks in 14.714786337secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.721094744secs
> Made 0 allocation in 12.688377237secs
>
> -----------------------------------
> AFTER (master + previous patch https://reviews.apache.org/r/70591/ + this patch):
>
> Added 3000 agents in 58.04155ms
> Added 3000 frameworks in 13.694651977secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.417541213secs
> Made 0 allocation in 12.130755905secs
>
> Added 3000 agents in 55.246813ms
> Added 3000 frameworks in 13.460479936secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.393765514secs
> Made 0 allocation in 12.196981426secs
>
> Added 3000 agents in 58.013477ms
> Added 3000 frameworks in 13.69361015secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.245916159secs
> Made 0 allocation in 11.699553888secs
>
> Added 3000 agents in 57.163681ms
> Added 3000 frameworks in 13.738218369secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.809206431secs
> Made 0 allocation in 12.400140164secs
>
> Added 3000 agents in 57.942087ms
> Added 3000 frameworks in 13.836390727secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.75595625secs
> Made 0 allocation in 11.939263998secs
>
>
> Thanks,
>
> Andrei Sekretenko
>
>
Re: Review Request 70618: Store framework sorters inside RoleInfos.
Posted by Mesos Reviewbot <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70618/#review215304
-----------------------------------------------------------
Patch looks great!
Reviews applied: [70626, 70591, 70618]
Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' CONFIGURATION='--verbose --disable-libtool-wrappers --disable-parallel-test-execution' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; ./support/docker-build.sh
- Mesos Reviewbot
On May 16, 2019, 2:17 p.m., Andrei Sekretenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70618/
> -----------------------------------------------------------
>
> (Updated May 16, 2019, 2:17 p.m.)
>
>
> Review request for mesos, Benjamin Mahler and Meng Zhu.
>
>
> Bugs: MESOS-9701
> https://issues.apache.org/jira/browse/MESOS-9701
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This patch replaces a hashmap<Owned<Sorter>> used for tracking
> framework sorters with a unique_ptr<Sorter> inside of the RoleInfo.
> This simplifies the framework tracking logic and slightly improves
> performance.
>
>
> Diffs
> -----
>
> src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
> src/master/allocator/mesos/hierarchical.cpp 875cfcd091f5f58afb89e752da5100a75828dd67
>
>
> Diff: https://reviews.apache.org/r/70618/diff/3/
>
>
> Testing
> -------
>
> make check
>
> Benchmarking: 5 runs of BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5 with the optimized build.
>
> BEFORE (master):
>
> Added 3000 agents in 57.444301ms
> Added 3000 frameworks in 15.100499419secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.359926851secs
> Made 0 allocation in 12.147765014secs
>
> Added 3000 agents in 58.651887ms
> Added 3000 frameworks in 14.485925735secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.6526783secs
> Made 0 allocation in 12.138439924secs
>
> Added 3000 agents in 59.028581ms
> Added 3000 frameworks in 14.72945866secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.739135078secs
> Made 0 allocation in 12.673302496secs
>
> Added 3000 agents in 59.050577ms
> Added 3000 frameworks in 14.78273576secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.20400492secs
> Made 0 allocation in 13.289808943secs
>
> Added 3000 agents in 58.629888ms
> Added 3000 frameworks in 14.714786337secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 13.721094744secs
> Made 0 allocation in 12.688377237secs
>
> -----------------------------------
> AFTER (master + previous patch https://reviews.apache.org/r/70591/ + this patch):
>
> Added 3000 agents in 58.04155ms
> Added 3000 frameworks in 13.694651977secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.417541213secs
> Made 0 allocation in 12.130755905secs
>
> Added 3000 agents in 55.246813ms
> Added 3000 frameworks in 13.460479936secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.393765514secs
> Made 0 allocation in 12.196981426secs
>
> Added 3000 agents in 58.013477ms
> Added 3000 frameworks in 13.69361015secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.245916159secs
> Made 0 allocation in 11.699553888secs
>
> Added 3000 agents in 57.163681ms
> Added 3000 frameworks in 13.738218369secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.809206431secs
> Made 0 allocation in 12.400140164secs
>
> Added 3000 agents in 57.942087ms
> Added 3000 frameworks in 13.836390727secs
> Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
> Made 3856 allocations in 12.75595625secs
> Made 0 allocation in 11.939263998secs
>
>
> Thanks,
>
> Andrei Sekretenko
>
>
Re: Review Request 70618: Store framework sorters inside RoleInfos.
Posted by Andrei Sekretenko <as...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70618/
-----------------------------------------------------------
(Updated May 16, 2019, 2:17 p.m.)
Review request for mesos, Benjamin Mahler and Meng Zhu.
Summary (updated)
-----------------
Store framework sorters inside RoleInfos.
Bugs: MESOS-9701
https://issues.apache.org/jira/browse/MESOS-9701
Repository: mesos
Description (updated)
-------
This patch replaces a hashmap<Owned<Sorter>> used for tracking
framework sorters with a unique_ptr<Sorter> inside of the RoleInfo.
This simplifies the framework tracking logic and slightly improves
performance.
Diffs (updated)
-----
src/master/allocator/mesos/hierarchical.hpp c2058baca5159da4cdcab77afd5de3c0d5ae6c48
src/master/allocator/mesos/hierarchical.cpp 875cfcd091f5f58afb89e752da5100a75828dd67
Diff: https://reviews.apache.org/r/70618/diff/3/
Changes: https://reviews.apache.org/r/70618/diff/2-3/
Testing (updated)
-------
make check
Benchmarking: 5 runs of BENCHMARK_HierarchicalAllocator_WithQuotaParam.LargeAndSmallQuota/5 with the optimized build.
BEFORE (master):
Added 3000 agents in 57.444301ms
Added 3000 frameworks in 15.100499419secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 13.359926851secs
Made 0 allocation in 12.147765014secs
Added 3000 agents in 58.651887ms
Added 3000 frameworks in 14.485925735secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 13.6526783secs
Made 0 allocation in 12.138439924secs
Added 3000 agents in 59.028581ms
Added 3000 frameworks in 14.72945866secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.739135078secs
Made 0 allocation in 12.673302496secs
Added 3000 agents in 59.050577ms
Added 3000 frameworks in 14.78273576secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 13.20400492secs
Made 0 allocation in 13.289808943secs
Added 3000 agents in 58.629888ms
Added 3000 frameworks in 14.714786337secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 13.721094744secs
Made 0 allocation in 12.688377237secs
-----------------------------------
AFTER (master + previous patch https://reviews.apache.org/r/70591/ + this patch):
Added 3000 agents in 58.04155ms
Added 3000 frameworks in 13.694651977secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.417541213secs
Made 0 allocation in 12.130755905secs
Added 3000 agents in 55.246813ms
Added 3000 frameworks in 13.460479936secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.393765514secs
Made 0 allocation in 12.196981426secs
Added 3000 agents in 58.013477ms
Added 3000 frameworks in 13.69361015secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.245916159secs
Made 0 allocation in 11.699553888secs
Added 3000 agents in 57.163681ms
Added 3000 frameworks in 13.738218369secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.809206431secs
Made 0 allocation in 12.400140164secs
Added 3000 agents in 57.942087ms
Added 3000 frameworks in 13.836390727secs
Benchmark setup: 3000 agents, 3000 roles, 3000 frameworks, with random sorter
Made 3856 allocations in 12.75595625secs
Made 0 allocation in 11.939263998secs
Thanks,
Andrei Sekretenko