You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Michael Park <mc...@gmail.com> on 2015/02/23 19:14:50 UTC
Re: Review Request 31183: Distinguished resources from different
slaves in DRFSorter.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/
-----------------------------------------------------------
(Updated Feb. 23, 2015, 6:14 p.m.)
Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
Changes
-------
s/Distinguish/Distinguished
Summary (updated)
-----------------
Distinguished resources from different slaves in DRFSorter.
Bugs: MESOS-2373
https://issues.apache.org/jira/browse/MESOS-2373
Repository: mesos
Description
-------
Distinguish resources from different slaves in DRFSorter.
Diffs
-----
src/master/allocator/allocator.hpp c2461a32e18dbafe637f37180f86bfa2e1d78735
src/master/allocator/mesos/allocator.hpp 99b40a01749b77fc7c1fa025a7ac81d5cca6272f
src/master/allocator/mesos/hierarchical.hpp 2680d6231927867d5a8d75cbc42b81d6c75fc7f2
src/master/allocator/sorter/drf/sorter.hpp 966fe03de389246dc71e933f9a51ad5b55f2834d
src/master/allocator/sorter/drf/sorter.cpp 7baee453848c7587472b2e9358c9e4ee6df6e860
src/master/allocator/sorter/sorter.hpp 8cd70dd91f72109222d8a0cd67cbf66ee4188ab5
src/master/http.cpp 5e3afdf8f32a7bf146eb39e53d1543c92e6180ba
src/master/master.hpp 6a39df04514c756415354fae66c5835ada191c52
src/master/master.cpp f10a3cffdb0b9831b2b8af0dfd30f73d1d8ed937
src/tests/hierarchical_allocator_tests.cpp eeecfb64540b16666915074aaffaa5d506b203bc
src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124
src/tests/sorter_tests.cpp 7ba32b78bdedf754f76f0695dfd1d86aa1f1b5bf
Diff: https://reviews.apache.org/r/31183/diff/
Testing
-------
make check
Thanks,
Michael Park
Re: Review Request 31183: Distinguished resources from different
slaves in DRFSorter.
Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review73964
-----------------------------------------------------------
Patch looks great!
Reviews applied: [31183]
All tests passed.
- Mesos ReviewBot
On Feb. 25, 2015, 1:16 a.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated Feb. 25, 2015, 1:16 a.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> See summary.
>
>
> Diffs
> -----
>
> include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
> src/common/resources.cpp 2c99b6884d7296099e19e2e3182cbe11b5e1e059
> src/master/allocator/allocator.hpp b67b8fddbd7a3fffc6fe24d5e77cd1db8cb6f69b
> src/master/allocator/mesos/allocator.hpp fb898f1175b61b442204e6e38c69ccc2838a646f
> src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
> src/master/allocator/sorter/drf/sorter.hpp 4366710d6530b784aa5094813328d0e338239ba0
> src/master/allocator/sorter/drf/sorter.cpp 2f69f384b95ff20d3ee429a4570a8cffa74d8e8b
> src/master/allocator/sorter/sorter.hpp e2efb27b11dbea42dd73f81e5db0d6d2b0a6034b
> src/master/http.cpp 117c0ee720a60a1d8a25359028bad803f1fc2b07
> src/master/master.hpp d414061b1aa7b4cac6ab9a3cc14e8e028f9370d0
> src/master/master.cpp 76e217d16c03e587ea4c0afca94c58b2212f0f93
> src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
> src/tests/mesos.hpp 6b9656b2b858df75e725aaa042c447b755899c87
> src/tests/sorter_tests.cpp 42442353afe7bd3d1a5b43992f8ae191ac19bdcd
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Michael Park <mc...@gmail.com>.
> On March 3, 2015, 6:45 p.m., Jie Yu wrote:
> > include/mesos/resources.hpp, lines 301-311
> > <https://reviews.apache.org/r/31183/diff/5/?file=882671#file882671line301>
> >
> > My only concern here is that this could potentially polute the namespace. What if someone wants to introduce `sum` for other types? How about moving this into `Resources`?
> >
> > ```
> > slaves[slaveId].available = total - Resources::sum(used.values());
> > ```
Fixed and rebased subsequent patches.
- Michael
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review75011
-----------------------------------------------------------
On March 3, 2015, 8:26 p.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated March 3, 2015, 8:26 p.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This is the start of a patch chain for converting `Resources` to `hashmap<SlaveID, Resources>` in places where we have an aggregate of resources across multiple slaves. In short, scalar resources aggregate correctly across multiple slaves, but non-scalar resources (e.g. ports) do not aggregate correctly. Please refer to the JIRA ticket for further details.
>
> The function `sum` is factored out to facilitate small patches. The following is the strategy and why this function will be useful:
>
> (1) Update a variable `resources` in a component (e.g. master) from `Resources` to `hashmap<SlaveID, Resources>`.
> (2) For callsites where we pass `resources` to another component (e.g. allocator), we'll pass `sum(resources.values())` instead to preserve the existing behavior in the other component.
>
>
> Diffs
> -----
>
> include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
> src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
> src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review75011
-----------------------------------------------------------
include/mesos/resources.hpp
<https://reviews.apache.org/r/31183/#comment121925>
My only concern here is that this could potentially polute the namespace. What if someone wants to introduce `sum` for other types? How about moving this into `Resources`?
```
slaves[slaveId].available = total - Resources::sum(used.values());
```
- Jie Yu
On March 3, 2015, 7:21 a.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated March 3, 2015, 7:21 a.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This is the start of a patch chain for converting `Resources` to `hashmap<SlaveID, Resources>` in places where we have an aggregate of resources across multiple slaves. In short, scalar resources aggregate correctly across multiple slaves, but non-scalar resources (e.g. ports) do not aggregate correctly. Please refer to the JIRA ticket for further details.
>
> The function `sum` is factored out to facilitate small patches. The following is the strategy and why this function will be useful:
>
> (1) Update a variable `resources` in a component (e.g. master) from `Resources` to `hashmap<SlaveID, Resources>`.
> (2) For callsites where we pass `resources` to another component (e.g. allocator), we'll pass `sum(resources.values())` instead to preserve the existing behavior in the other component.
>
>
> Diffs
> -----
>
> include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
> src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
> src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Michael Park <mc...@gmail.com>.
> On March 6, 2015, 2:37 a.m., Alexander Rukletsov wrote:
> > include/mesos/resources.hpp, lines 107-108
> > <https://reviews.apache.org/r/31183/diff/6/?file=883408#file883408line107>
> >
> > We can't really sum non-scalar resources, right? Does it make sense to rename the function to something like "aggregate" to indicate that?
I'm not sure if `s/sum/aggregate/` suffices as an indicator. I'll add a comment saying that this function does not yield correct results over non-scalar resources and that it should be deprecated once we introduce a concept such as `ClusterResources` which handles resources across multiple slaves correctly.
- Michael
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review75451
-----------------------------------------------------------
On March 3, 2015, 8:26 p.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated March 3, 2015, 8:26 p.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This is the start of a patch chain for converting `Resources` to `hashmap<SlaveID, Resources>` in places where we have an aggregate of resources across multiple slaves. In short, scalar resources aggregate correctly across multiple slaves, but non-scalar resources (e.g. ports) do not aggregate correctly. Please refer to the JIRA ticket for further details.
>
> The function `sum` is factored out to facilitate small patches. The following is the strategy and why this function will be useful:
>
> (1) Update a variable `resources` in a component (e.g. master) from `Resources` to `hashmap<SlaveID, Resources>`.
> (2) For callsites where we pass `resources` to another component (e.g. allocator), we'll pass `sum(resources.values())` instead to preserve the existing behavior in the other component.
>
>
> Diffs
> -----
>
> include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
> src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
> src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Alexander Rukletsov <al...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review75451
-----------------------------------------------------------
include/mesos/resources.hpp
<https://reviews.apache.org/r/31183/#comment122500>
We can't really sum non-scalar resources, right? Does it make sense to rename the function to something like "aggregate" to indicate that?
- Alexander Rukletsov
On March 3, 2015, 8:26 p.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated March 3, 2015, 8:26 p.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This is the start of a patch chain for converting `Resources` to `hashmap<SlaveID, Resources>` in places where we have an aggregate of resources across multiple slaves. In short, scalar resources aggregate correctly across multiple slaves, but non-scalar resources (e.g. ports) do not aggregate correctly. Please refer to the JIRA ticket for further details.
>
> The function `sum` is factored out to facilitate small patches. The following is the strategy and why this function will be useful:
>
> (1) Update a variable `resources` in a component (e.g. master) from `Resources` to `hashmap<SlaveID, Resources>`.
> (2) For callsites where we pass `resources` to another component (e.g. allocator), we'll pass `sum(resources.values())` instead to preserve the existing behavior in the other component.
>
>
> Diffs
> -----
>
> include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
> src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
> src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Michael Park <mc...@gmail.com>.
> On March 6, 2015, 9:42 p.m., Alexander Rukletsov wrote:
> > Sorry for commenting after giving a "Ship It". I was checking `r/31665` and realised that we do an extra copy while calculating the aggregate of resources: `std::list<Resources>` out of `hashmap<SlaveID, Resources>`. Does it make sense to avoid it?
I think it makes to have this operation which takes `hashmap<Key, Resources>` and collapses it to `Resources` merely to facilitate preserving existing functionalities (e.g. HTTP endpoints, `calculateShare` in `DRFSorter`). I've made `sum` function take a `hashmap<Key, Resources>` instead, since we want to use this for `hashmap<FrameworkID, Resources>` as well.
- Michael
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review75550
-----------------------------------------------------------
On March 7, 2015, 10:08 a.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated March 7, 2015, 10:08 a.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This is the start of a patch chain for converting `Resources` to `hashmap<SlaveID, Resources>` in places where we have an aggregate of resources across multiple slaves. In short, scalar resources aggregate correctly across multiple slaves, but non-scalar resources (e.g. ports) do not aggregate correctly. Please refer to the JIRA ticket for further details.
>
> The function `sum` is factored out to facilitate small patches. The following is the strategy and why this function will be useful:
>
> (1) Update a variable `resources` in a component (e.g. master) from `Resources` to `hashmap<SlaveID, Resources>`.
> (2) For callsites where we pass `resources` to another component (e.g. allocator), we'll pass `sum(resources)` instead to preserve the existing behavior in the other component.
>
>
> Diffs
> -----
>
> include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
> src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
> src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Alexander Rukletsov <al...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review75550
-----------------------------------------------------------
Sorry for commenting after giving a "Ship It". I was checking `r/31665` and realised that we do an extra copy while calculating the aggregate of resources: `std::list<Resources>` out of `hashmap<SlaveID, Resources>`. Does it make sense to avoid it?
- Alexander Rukletsov
On March 6, 2015, 6:31 p.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated March 6, 2015, 6:31 p.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This is the start of a patch chain for converting `Resources` to `hashmap<SlaveID, Resources>` in places where we have an aggregate of resources across multiple slaves. In short, scalar resources aggregate correctly across multiple slaves, but non-scalar resources (e.g. ports) do not aggregate correctly. Please refer to the JIRA ticket for further details.
>
> The function `sum` is factored out to facilitate small patches. The following is the strategy and why this function will be useful:
>
> (1) Update a variable `resources` in a component (e.g. master) from `Resources` to `hashmap<SlaveID, Resources>`.
> (2) For callsites where we pass `resources` to another component (e.g. allocator), we'll pass `sum(resources.values())` instead to preserve the existing behavior in the other component.
>
>
> Diffs
> -----
>
> include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
> src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
> src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Alexander Rukletsov <al...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review75543
-----------------------------------------------------------
Ship it!
Ship It!
- Alexander Rukletsov
On March 6, 2015, 6:31 p.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated March 6, 2015, 6:31 p.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This is the start of a patch chain for converting `Resources` to `hashmap<SlaveID, Resources>` in places where we have an aggregate of resources across multiple slaves. In short, scalar resources aggregate correctly across multiple slaves, but non-scalar resources (e.g. ports) do not aggregate correctly. Please refer to the JIRA ticket for further details.
>
> The function `sum` is factored out to facilitate small patches. The following is the strategy and why this function will be useful:
>
> (1) Update a variable `resources` in a component (e.g. master) from `Resources` to `hashmap<SlaveID, Resources>`.
> (2) For callsites where we pass `resources` to another component (e.g. allocator), we'll pass `sum(resources.values())` instead to preserve the existing behavior in the other component.
>
>
> Diffs
> -----
>
> include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
> src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
> src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Joerg Schad <jo...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review75868
-----------------------------------------------------------
lgtm!
- Joerg Schad
On March 7, 2015, 10:08 a.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated March 7, 2015, 10:08 a.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This is the start of a patch chain for converting `Resources` to `hashmap<SlaveID, Resources>` in places where we have an aggregate of resources across multiple slaves. In short, scalar resources aggregate correctly across multiple slaves, but non-scalar resources (e.g. ports) do not aggregate correctly. Please refer to the JIRA ticket for further details.
>
> The function `sum` is factored out to facilitate small patches. The following is the strategy and why this function will be useful:
>
> (1) Update a variable `resources` in a component (e.g. master) from `Resources` to `hashmap<SlaveID, Resources>`.
> (2) For callsites where we pass `resources` to another component (e.g. allocator), we'll pass `sum(resources)` instead to preserve the existing behavior in the other component.
>
>
> Diffs
> -----
>
> include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
> src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
> src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Alexander Rukletsov <al...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review75748
-----------------------------------------------------------
Ship it!
Ship It!
- Alexander Rukletsov
On March 7, 2015, 10:08 a.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated March 7, 2015, 10:08 a.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This is the start of a patch chain for converting `Resources` to `hashmap<SlaveID, Resources>` in places where we have an aggregate of resources across multiple slaves. In short, scalar resources aggregate correctly across multiple slaves, but non-scalar resources (e.g. ports) do not aggregate correctly. Please refer to the JIRA ticket for further details.
>
> The function `sum` is factored out to facilitate small patches. The following is the strategy and why this function will be useful:
>
> (1) Update a variable `resources` in a component (e.g. master) from `Resources` to `hashmap<SlaveID, Resources>`.
> (2) For callsites where we pass `resources` to another component (e.g. allocator), we'll pass `sum(resources)` instead to preserve the existing behavior in the other component.
>
>
> Diffs
> -----
>
> include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
> src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
> src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Michael Park <mc...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/
-----------------------------------------------------------
(Updated March 7, 2015, 10:08 a.m.)
Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
Changes
-------
Updated **Description** to reflect the change of pattern from `sum(resources.values())` to `sum(resources)`.
Bugs: MESOS-2373
https://issues.apache.org/jira/browse/MESOS-2373
Repository: mesos
Description (updated)
-------
This is the start of a patch chain for converting `Resources` to `hashmap<SlaveID, Resources>` in places where we have an aggregate of resources across multiple slaves. In short, scalar resources aggregate correctly across multiple slaves, but non-scalar resources (e.g. ports) do not aggregate correctly. Please refer to the JIRA ticket for further details.
The function `sum` is factored out to facilitate small patches. The following is the strategy and why this function will be useful:
(1) Update a variable `resources` in a component (e.g. master) from `Resources` to `hashmap<SlaveID, Resources>`.
(2) For callsites where we pass `resources` to another component (e.g. allocator), we'll pass `sum(resources)` instead to preserve the existing behavior in the other component.
Diffs
-----
include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
Diff: https://reviews.apache.org/r/31183/diff/
Testing
-------
make check
Thanks,
Michael Park
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Michael Park <mc...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/
-----------------------------------------------------------
(Updated March 7, 2015, 9:50 a.m.)
Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
Changes
-------
Changed `sum` to take a `hashmap<Key, Resources>` instead of a templated `Iterable`.
Bugs: MESOS-2373
https://issues.apache.org/jira/browse/MESOS-2373
Repository: mesos
Description
-------
This is the start of a patch chain for converting `Resources` to `hashmap<SlaveID, Resources>` in places where we have an aggregate of resources across multiple slaves. In short, scalar resources aggregate correctly across multiple slaves, but non-scalar resources (e.g. ports) do not aggregate correctly. Please refer to the JIRA ticket for further details.
The function `sum` is factored out to facilitate small patches. The following is the strategy and why this function will be useful:
(1) Update a variable `resources` in a component (e.g. master) from `Resources` to `hashmap<SlaveID, Resources>`.
(2) For callsites where we pass `resources` to another component (e.g. allocator), we'll pass `sum(resources.values())` instead to preserve the existing behavior in the other component.
Diffs (updated)
-----
include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
Diff: https://reviews.apache.org/r/31183/diff/
Testing
-------
make check
Thanks,
Michael Park
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Michael Park <mc...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/
-----------------------------------------------------------
(Updated March 6, 2015, 6:31 p.m.)
Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
Changes
-------
Added a comment for `Resources::sum` with a `NOTE` about the incorrect semantics over non-scalar resources, and a `TODO` to deprecate once we introduce the concept of cluster resources.
Bugs: MESOS-2373
https://issues.apache.org/jira/browse/MESOS-2373
Repository: mesos
Description
-------
This is the start of a patch chain for converting `Resources` to `hashmap<SlaveID, Resources>` in places where we have an aggregate of resources across multiple slaves. In short, scalar resources aggregate correctly across multiple slaves, but non-scalar resources (e.g. ports) do not aggregate correctly. Please refer to the JIRA ticket for further details.
The function `sum` is factored out to facilitate small patches. The following is the strategy and why this function will be useful:
(1) Update a variable `resources` in a component (e.g. master) from `Resources` to `hashmap<SlaveID, Resources>`.
(2) For callsites where we pass `resources` to another component (e.g. allocator), we'll pass `sum(resources.values())` instead to preserve the existing behavior in the other component.
Diffs (updated)
-----
include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
Diff: https://reviews.apache.org/r/31183/diff/
Testing
-------
make check
Thanks,
Michael Park
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Michael Park <mc...@gmail.com>.
> On March 6, 2015, 1:50 p.m., Joerg Schad wrote:
> > The semantics of sum is still the old semantics (i.e. not considering non-scalar ressources). If yes could you add a small note about this in the description?
That's correct. I'll add a comment saying that this function does not yield correct results over non-scalar resources and that it should be deprecated once we introduce a concept such as ClusterResources which handles resources across multiple slaves correctly.
> On March 6, 2015, 1:50 p.m., Joerg Schad wrote:
> > include/mesos/resources.hpp, lines 107-108
> > <https://reviews.apache.org/r/31183/diff/6/?file=883408#file883408line107>
> >
> > Could you provide a comment about the functionality and especially the (for non-scalar resources non-obvious) semantics here?
See my comment above.
- Michael
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review75486
-----------------------------------------------------------
On March 3, 2015, 8:26 p.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated March 3, 2015, 8:26 p.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This is the start of a patch chain for converting `Resources` to `hashmap<SlaveID, Resources>` in places where we have an aggregate of resources across multiple slaves. In short, scalar resources aggregate correctly across multiple slaves, but non-scalar resources (e.g. ports) do not aggregate correctly. Please refer to the JIRA ticket for further details.
>
> The function `sum` is factored out to facilitate small patches. The following is the strategy and why this function will be useful:
>
> (1) Update a variable `resources` in a component (e.g. master) from `Resources` to `hashmap<SlaveID, Resources>`.
> (2) For callsites where we pass `resources` to another component (e.g. allocator), we'll pass `sum(resources.values())` instead to preserve the existing behavior in the other component.
>
>
> Diffs
> -----
>
> include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
> src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
> src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Joerg Schad <jo...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review75486
-----------------------------------------------------------
The semantics of sum is still the old semantics (i.e. not considering non-scalar ressources). If yes could you add a small note about this in the description?
include/mesos/resources.hpp
<https://reviews.apache.org/r/31183/#comment122563>
Could you provide a comment about the functionality and especially the (for non-scalar resources non-obvious) semantics here?
- Joerg Schad
On March 3, 2015, 8:26 p.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated March 3, 2015, 8:26 p.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This is the start of a patch chain for converting `Resources` to `hashmap<SlaveID, Resources>` in places where we have an aggregate of resources across multiple slaves. In short, scalar resources aggregate correctly across multiple slaves, but non-scalar resources (e.g. ports) do not aggregate correctly. Please refer to the JIRA ticket for further details.
>
> The function `sum` is factored out to facilitate small patches. The following is the strategy and why this function will be useful:
>
> (1) Update a variable `resources` in a component (e.g. master) from `Resources` to `hashmap<SlaveID, Resources>`.
> (2) For callsites where we pass `resources` to another component (e.g. allocator), we'll pass `sum(resources.values())` instead to preserve the existing behavior in the other component.
>
>
> Diffs
> -----
>
> include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
> src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
> src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review75031
-----------------------------------------------------------
Ship it!
Ship It!
- Jie Yu
On March 3, 2015, 8:26 p.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated March 3, 2015, 8:26 p.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> This is the start of a patch chain for converting `Resources` to `hashmap<SlaveID, Resources>` in places where we have an aggregate of resources across multiple slaves. In short, scalar resources aggregate correctly across multiple slaves, but non-scalar resources (e.g. ports) do not aggregate correctly. Please refer to the JIRA ticket for further details.
>
> The function `sum` is factored out to facilitate small patches. The following is the strategy and why this function will be useful:
>
> (1) Update a variable `resources` in a component (e.g. master) from `Resources` to `hashmap<SlaveID, Resources>`.
> (2) For callsites where we pass `resources` to another component (e.g. allocator), we'll pass `sum(resources.values())` instead to preserve the existing behavior in the other component.
>
>
> Diffs
> -----
>
> include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
> src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
> src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Michael Park <mc...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/
-----------------------------------------------------------
(Updated March 3, 2015, 8:26 p.m.)
Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
Changes
-------
Moved `sum` into `Resources`.
Bugs: MESOS-2373
https://issues.apache.org/jira/browse/MESOS-2373
Repository: mesos
Description
-------
This is the start of a patch chain for converting `Resources` to `hashmap<SlaveID, Resources>` in places where we have an aggregate of resources across multiple slaves. In short, scalar resources aggregate correctly across multiple slaves, but non-scalar resources (e.g. ports) do not aggregate correctly. Please refer to the JIRA ticket for further details.
The function `sum` is factored out to facilitate small patches. The following is the strategy and why this function will be useful:
(1) Update a variable `resources` in a component (e.g. master) from `Resources` to `hashmap<SlaveID, Resources>`.
(2) For callsites where we pass `resources` to another component (e.g. allocator), we'll pass `sum(resources.values())` instead to preserve the existing behavior in the other component.
Diffs (updated)
-----
include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
Diff: https://reviews.apache.org/r/31183/diff/
Testing
-------
make check
Thanks,
Michael Park
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Michael Park <mc...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/
-----------------------------------------------------------
(Updated March 3, 2015, 7:21 a.m.)
Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
Bugs: MESOS-2373
https://issues.apache.org/jira/browse/MESOS-2373
Repository: mesos
Description (updated)
-------
This is the start of a patch chain for converting `Resources` to `hashmap<SlaveID, Resources>` in places where we have an aggregate of resources across multiple slaves. In short, scalar resources aggregate correctly across multiple slaves, but non-scalar resources (e.g. ports) do not aggregate correctly. Please refer to the JIRA ticket for further details.
The function `sum` is factored out to facilitate small patches. The following is the strategy and why this function will be useful:
(1) Update a variable `resources` in a component (e.g. master) from `Resources` to `hashmap<SlaveID, Resources>`.
(2) For callsites where we pass `resources` to another component (e.g. allocator), we'll pass `sum(resources.values())` instead to preserve the existing behavior in the other component.
Diffs
-----
include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
Diff: https://reviews.apache.org/r/31183/diff/
Testing
-------
make check
Thanks,
Michael Park
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Michael Park <mc...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/
-----------------------------------------------------------
(Updated March 3, 2015, 7:20 a.m.)
Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
Bugs: MESOS-2373
https://issues.apache.org/jira/browse/MESOS-2373
Repository: mesos
Description (updated)
-------
This is the start of a patch chain for converting `Resources` to `hashmap<SlaveID, Resources>` in places where we have an aggregate of resources across multiple slaves. In short, scalar resources aggregate correctly across multiple slaves, but non-scalar resources (e.g. ports) do not aggregate correctly. Please refer to the JIRA ticket for further details.
The function `sum` is factored out to facilitate small patches. The following is the strategy and why this function will be useful:
(1) Update a variable `resources` in a component (e.g. master) from `Resources` to `hashmap<SlaveID, Resources>`.
(2) For callsites where we pass `resources` to another component (e.g. allocator), we'll pass `sum(resources.values())` instead to preserve the existing behavior in the other component.
Diffs
-----
include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
Diff: https://reviews.apache.org/r/31183/diff/
Testing
-------
make check
Thanks,
Michael Park
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Michael Park <mc...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/
-----------------------------------------------------------
(Updated March 3, 2015, 7:19 a.m.)
Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
Changes
-------
Indented and changed list numbering style.
Bugs: MESOS-2373
https://issues.apache.org/jira/browse/MESOS-2373
Repository: mesos
Description (updated)
-------
This is the start of a patch chain for converting `Resources` to `hashmap<SlaveID, Resources>` in places where we have an aggregate of resources across multiple slaves. In short, scalar resources aggregate correctly across multiple slaves, but non-scalar resources (e.g. ports) do not aggregate correctly. Please refer to the JIRA ticket for further details.
The function `sum` is factored out to facilitate small patches. The following is the strategy and why this function will be useful:
(1) Update a variable `resources` in a component (e.g. master) from `Resources` to `hashmap<SlaveID, Resources>`.
(2) For callsites where we pass `resources` to another component (e.g. allocator), we'll pass `sum(resources.values())` instead to preserve the existing behavior in the other component.
Diffs
-----
include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
Diff: https://reviews.apache.org/r/31183/diff/
Testing
-------
make check
Thanks,
Michael Park
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Michael Park <mc...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/
-----------------------------------------------------------
(Updated March 3, 2015, 6:56 a.m.)
Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
Changes
-------
Added a description for why this patch is useful.
Bugs: MESOS-2373
https://issues.apache.org/jira/browse/MESOS-2373
Repository: mesos
Description (updated)
-------
This is the start of a patch chain for converting `Resources` to `hashmap<SlaveID, Resources>` in places where we have an aggregate of resources across multiple slaves. In short, scalar resources aggregate correctly across multiple slaves, but non-scalar resources (e.g. ports) do not aggregate correctly. Please refer to the JIRA ticket for further details.
The function `sum` is factored out to facilitate small patches. The following is the strategy and why this function will be useful:
1. Update a variable `resources` in a component (e.g. master) from `Resources` to `hashmap<SlaveID, Resources>`.
2. For callsites where we pass `resources` to another component (e.g. allocator), we'll pass `sum(resources.values())` instead to preserve the existing behavior in the other component.
Diffs
-----
include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
Diff: https://reviews.apache.org/r/31183/diff/
Testing
-------
make check
Thanks,
Michael Park
Re: Review Request 31183: Factored out sum function for Resources.
Posted by Michael Park <mc...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/
-----------------------------------------------------------
(Updated March 3, 2015, 6:45 a.m.)
Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
Summary (updated)
-----------------
Factored out sum function for Resources.
Bugs: MESOS-2373
https://issues.apache.org/jira/browse/MESOS-2373
Repository: mesos
Description
-------
See summary.
Diffs (updated)
-----
include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
Diff: https://reviews.apache.org/r/31183/diff/
Testing
-------
make check
Thanks,
Michael Park
Re: Review Request 31183: Distinguished resources from different
slaves in DRFSorter.
Posted by Michael Park <mc...@gmail.com>.
> On Feb. 27, 2015, 1:18 p.m., Alexander Rojas wrote:
> > include/mesos/resources.hpp, line 302
> > <https://reviews.apache.org/r/31183/diff/4/?file=874692#file874692line302>
> >
> > Why not use `std::accumulate`?
Do you mean why we don't use `std::accumulate` instead of introducing this? or do you mean why don't we use `std::accumulate` in the implementation?
For the former, it's because `std::accumulate` is harder to use, with having to pass the iterators, etc. It's similar to `stout/set.hpp`'s `operator &` which is a wrapper around `std::set_intersection`, or `contains` function which is a wrapper around the `std::find(elems.begin(), elems.end(), elem) != elems.end()` pattern.
For the latter, it would have to look something like:
```cpp
template <typename Iterable>
Resources sum(const Iterable& iterable) {
return std::accumulate(
std::begin(iterable),
std::end(iterable),
Resources(),
[](Resources& result, const Resources& resources) -> Resources & {
return result += resources;
});
}
```
Without lambdas however I think it's simpler to just write the for-loop.
- Michael
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review74482
-----------------------------------------------------------
On Feb. 25, 2015, 1:16 a.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated Feb. 25, 2015, 1:16 a.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> See summary.
>
>
> Diffs
> -----
>
> include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
> src/common/resources.cpp 2c99b6884d7296099e19e2e3182cbe11b5e1e059
> src/master/allocator/allocator.hpp b67b8fddbd7a3fffc6fe24d5e77cd1db8cb6f69b
> src/master/allocator/mesos/allocator.hpp fb898f1175b61b442204e6e38c69ccc2838a646f
> src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
> src/master/allocator/sorter/drf/sorter.hpp 4366710d6530b784aa5094813328d0e338239ba0
> src/master/allocator/sorter/drf/sorter.cpp 2f69f384b95ff20d3ee429a4570a8cffa74d8e8b
> src/master/allocator/sorter/sorter.hpp e2efb27b11dbea42dd73f81e5db0d6d2b0a6034b
> src/master/http.cpp 117c0ee720a60a1d8a25359028bad803f1fc2b07
> src/master/master.hpp d414061b1aa7b4cac6ab9a3cc14e8e028f9370d0
> src/master/master.cpp 76e217d16c03e587ea4c0afca94c58b2212f0f93
> src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
> src/tests/mesos.hpp 6b9656b2b858df75e725aaa042c447b755899c87
> src/tests/sorter_tests.cpp 42442353afe7bd3d1a5b43992f8ae191ac19bdcd
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Distinguished resources from different
slaves in DRFSorter.
Posted by Alexander Rojas <al...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review74482
-----------------------------------------------------------
include/mesos/resources.hpp
<https://reviews.apache.org/r/31183/#comment121058>
Why not use `std::accumulate`?
- Alexander Rojas
On Feb. 25, 2015, 2:16 a.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated Feb. 25, 2015, 2:16 a.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> See summary.
>
>
> Diffs
> -----
>
> include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
> src/common/resources.cpp 2c99b6884d7296099e19e2e3182cbe11b5e1e059
> src/master/allocator/allocator.hpp b67b8fddbd7a3fffc6fe24d5e77cd1db8cb6f69b
> src/master/allocator/mesos/allocator.hpp fb898f1175b61b442204e6e38c69ccc2838a646f
> src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
> src/master/allocator/sorter/drf/sorter.hpp 4366710d6530b784aa5094813328d0e338239ba0
> src/master/allocator/sorter/drf/sorter.cpp 2f69f384b95ff20d3ee429a4570a8cffa74d8e8b
> src/master/allocator/sorter/sorter.hpp e2efb27b11dbea42dd73f81e5db0d6d2b0a6034b
> src/master/http.cpp 117c0ee720a60a1d8a25359028bad803f1fc2b07
> src/master/master.hpp d414061b1aa7b4cac6ab9a3cc14e8e028f9370d0
> src/master/master.cpp 76e217d16c03e587ea4c0afca94c58b2212f0f93
> src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
> src/tests/mesos.hpp 6b9656b2b858df75e725aaa042c447b755899c87
> src/tests/sorter_tests.cpp 42442353afe7bd3d1a5b43992f8ae191ac19bdcd
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Distinguished resources from different
slaves in DRFSorter.
Posted by Michael Park <mc...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/
-----------------------------------------------------------
(Updated Feb. 25, 2015, 1:16 a.m.)
Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
Changes
-------
Rebased.
Bugs: MESOS-2373
https://issues.apache.org/jira/browse/MESOS-2373
Repository: mesos
Description
-------
See summary.
Diffs (updated)
-----
include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
src/common/resources.cpp 2c99b6884d7296099e19e2e3182cbe11b5e1e059
src/master/allocator/allocator.hpp b67b8fddbd7a3fffc6fe24d5e77cd1db8cb6f69b
src/master/allocator/mesos/allocator.hpp fb898f1175b61b442204e6e38c69ccc2838a646f
src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
src/master/allocator/sorter/drf/sorter.hpp 4366710d6530b784aa5094813328d0e338239ba0
src/master/allocator/sorter/drf/sorter.cpp 2f69f384b95ff20d3ee429a4570a8cffa74d8e8b
src/master/allocator/sorter/sorter.hpp e2efb27b11dbea42dd73f81e5db0d6d2b0a6034b
src/master/http.cpp 117c0ee720a60a1d8a25359028bad803f1fc2b07
src/master/master.hpp d414061b1aa7b4cac6ab9a3cc14e8e028f9370d0
src/master/master.cpp 76e217d16c03e587ea4c0afca94c58b2212f0f93
src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
src/tests/mesos.hpp 6b9656b2b858df75e725aaa042c447b755899c87
src/tests/sorter_tests.cpp 42442353afe7bd3d1a5b43992f8ae191ac19bdcd
Diff: https://reviews.apache.org/r/31183/diff/
Testing
-------
make check
Thanks,
Michael Park
Re: Review Request 31183: Distinguished resources from different
slaves in DRFSorter.
Posted by Mesos ReviewBot <de...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review73799
-----------------------------------------------------------
Bad patch!
Reviews applied: [31183]
Failed command: ./support/apply-review.sh -n -r 31183
Error:
2015-02-24 11:58:05 URL:https://reviews.apache.org/r/31183/diff/raw/ [56098/56098] -> "31183.patch" [1]
Successfully applied: Distinguished resources from different slaves in DRFSorter.
See summary.
Review: https://reviews.apache.org/r/31183
Checking 20 files using filter --filter=-,+build/class,+build/deprecated,+build/endif_comment,+readability/todo,+readability/namespace,+runtime/vlog,+whitespace/blank_line,+whitespace/comma,+whitespace/end_of_line,+whitespace/ending_newline,+whitespace/forcolon,+whitespace/indent,+whitespace/line_length,+whitespace/operators,+whitespace/semicolon,+whitespace/tab,+whitespace/todo
Total errors found: 0
ERROR: Commit spanning multiple projects.
Please use separate commits for mesos, libprocess and stout.
Paths grouped by project:
mesos:
configure.ac
docs/modules.md
docs/slave-recovery.md
include/mesos/resources.hpp
src/common/resources.cpp
src/master/allocator/allocator.hpp
src/master/allocator/mesos/allocator.hpp
src/master/allocator/mesos/hierarchical.hpp
src/master/allocator/sorter/drf/sorter.cpp
src/master/allocator/sorter/drf/sorter.hpp
src/master/allocator/sorter/sorter.hpp
src/master/http.cpp
src/master/master.cpp
src/master/master.hpp
src/slave/containerizer/isolators/network/port_mapping.cpp
src/tests/hierarchical_allocator_tests.cpp
src/tests/master_slave_reconciliation_tests.cpp
src/tests/mesos.hpp
src/tests/port_mapping_tests.cpp
src/tests/slave_tests.cpp
src/tests/sorter_tests.cpp
stout:
3rdparty/libprocess/3rdparty/stout/include/Makefile.am
3rdparty/libprocess/3rdparty/stout/include/stout/ip.hpp
3rdparty/libprocess/3rdparty/stout/tests/ip_tests.cpp
Failed to commit patch
- Mesos ReviewBot
On Feb. 24, 2015, 11:27 a.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated Feb. 24, 2015, 11:27 a.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> See summary.
>
>
> Diffs
> -----
>
> 3rdparty/libprocess/3rdparty/stout/include/Makefile.am ac2bbed6fe86623fb51cac3613d79d7b1372df9d
> 3rdparty/libprocess/3rdparty/stout/include/stout/ip.hpp 3293086a009a8f7cf7bd343eb7d3e85623636550
> 3rdparty/libprocess/3rdparty/stout/tests/ip_tests.cpp fb98317a68986cb1228c584a8cd83b07737895a8
> configure.ac 9b2d7f15f535aaaf85faf9b4f7af750f1dbdf472
> docs/modules.md a8b471541cdfa584eeb89fbe96643f93c712cfd4
> docs/slave-recovery.md 57eb786f94b2b1dee7bb35017618af90b4dc4a31
> include/mesos/resources.hpp c242bcc29c490841354d6fdc8d0de16eeea602ed
> src/common/resources.cpp a45bbaf696a6cc61a06daaa52a84f0014e7fe8cb
> src/master/allocator/allocator.hpp b67b8fddbd7a3fffc6fe24d5e77cd1db8cb6f69b
> src/master/allocator/mesos/allocator.hpp fb898f1175b61b442204e6e38c69ccc2838a646f
> src/master/allocator/mesos/hierarchical.hpp 9b7ded34ad7546f36dd41f64fe2e3cf41c1cf702
> src/master/allocator/sorter/drf/sorter.hpp 4366710d6530b784aa5094813328d0e338239ba0
> src/master/allocator/sorter/drf/sorter.cpp 2f69f384b95ff20d3ee429a4570a8cffa74d8e8b
> src/master/allocator/sorter/sorter.hpp e2efb27b11dbea42dd73f81e5db0d6d2b0a6034b
> src/master/http.cpp 117c0ee720a60a1d8a25359028bad803f1fc2b07
> src/master/master.hpp 8c44d6ed57ad1b94a17bef8142a5e6a15889a810
> src/master/master.cpp 713307e1be596651283cc2cc95f114c42ad34a5e
> src/slave/containerizer/isolators/network/port_mapping.cpp 5227987cdb7b904c2f4bb2bdf5c5d705a435010d
> src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
> src/tests/master_slave_reconciliation_tests.cpp 54cae62fda6be56ce438371c0e37bdf8cc8ace15
> src/tests/mesos.hpp e17afe74796ffe148274e278b5fa574e5c1bd34c
> src/tests/port_mapping_tests.cpp e2c8ba12b5574b06a6ba60c3c3a30b63cea1d23c
> src/tests/slave_tests.cpp 7ea012ab0883cc030b3e62f59879613866dab0fa
> src/tests/sorter_tests.cpp 42442353afe7bd3d1a5b43992f8ae191ac19bdcd
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Distinguished resources from different
slaves in DRFSorter.
Posted by Michael Park <mc...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/
-----------------------------------------------------------
(Updated Feb. 24, 2015, 11:27 a.m.)
Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
Bugs: MESOS-2373
https://issues.apache.org/jira/browse/MESOS-2373
Repository: mesos
Description
-------
See summary.
Diffs (updated)
-----
3rdparty/libprocess/3rdparty/stout/include/Makefile.am ac2bbed6fe86623fb51cac3613d79d7b1372df9d
3rdparty/libprocess/3rdparty/stout/include/stout/ip.hpp 3293086a009a8f7cf7bd343eb7d3e85623636550
3rdparty/libprocess/3rdparty/stout/tests/ip_tests.cpp fb98317a68986cb1228c584a8cd83b07737895a8
configure.ac 9b2d7f15f535aaaf85faf9b4f7af750f1dbdf472
docs/modules.md a8b471541cdfa584eeb89fbe96643f93c712cfd4
docs/slave-recovery.md 57eb786f94b2b1dee7bb35017618af90b4dc4a31
include/mesos/resources.hpp c242bcc29c490841354d6fdc8d0de16eeea602ed
src/common/resources.cpp a45bbaf696a6cc61a06daaa52a84f0014e7fe8cb
src/master/allocator/allocator.hpp b67b8fddbd7a3fffc6fe24d5e77cd1db8cb6f69b
src/master/allocator/mesos/allocator.hpp fb898f1175b61b442204e6e38c69ccc2838a646f
src/master/allocator/mesos/hierarchical.hpp 9b7ded34ad7546f36dd41f64fe2e3cf41c1cf702
src/master/allocator/sorter/drf/sorter.hpp 4366710d6530b784aa5094813328d0e338239ba0
src/master/allocator/sorter/drf/sorter.cpp 2f69f384b95ff20d3ee429a4570a8cffa74d8e8b
src/master/allocator/sorter/sorter.hpp e2efb27b11dbea42dd73f81e5db0d6d2b0a6034b
src/master/http.cpp 117c0ee720a60a1d8a25359028bad803f1fc2b07
src/master/master.hpp 8c44d6ed57ad1b94a17bef8142a5e6a15889a810
src/master/master.cpp 713307e1be596651283cc2cc95f114c42ad34a5e
src/slave/containerizer/isolators/network/port_mapping.cpp 5227987cdb7b904c2f4bb2bdf5c5d705a435010d
src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
src/tests/master_slave_reconciliation_tests.cpp 54cae62fda6be56ce438371c0e37bdf8cc8ace15
src/tests/mesos.hpp e17afe74796ffe148274e278b5fa574e5c1bd34c
src/tests/port_mapping_tests.cpp e2c8ba12b5574b06a6ba60c3c3a30b63cea1d23c
src/tests/slave_tests.cpp 7ea012ab0883cc030b3e62f59879613866dab0fa
src/tests/sorter_tests.cpp 42442353afe7bd3d1a5b43992f8ae191ac19bdcd
Diff: https://reviews.apache.org/r/31183/diff/
Testing
-------
make check
Thanks,
Michael Park
Re: Review Request 31183: Distinguished resources from different
slaves in DRFSorter.
Posted by Michael Park <mc...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/
-----------------------------------------------------------
(Updated Feb. 24, 2015, 9:07 a.m.)
Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
Bugs: MESOS-2373
https://issues.apache.org/jira/browse/MESOS-2373
Repository: mesos
Description (updated)
-------
See summary.
Diffs
-----
src/master/allocator/allocator.hpp c2461a32e18dbafe637f37180f86bfa2e1d78735
src/master/allocator/mesos/allocator.hpp 99b40a01749b77fc7c1fa025a7ac81d5cca6272f
src/master/allocator/mesos/hierarchical.hpp 2680d6231927867d5a8d75cbc42b81d6c75fc7f2
src/master/allocator/sorter/drf/sorter.hpp 966fe03de389246dc71e933f9a51ad5b55f2834d
src/master/allocator/sorter/drf/sorter.cpp 7baee453848c7587472b2e9358c9e4ee6df6e860
src/master/allocator/sorter/sorter.hpp 8cd70dd91f72109222d8a0cd67cbf66ee4188ab5
src/master/http.cpp 5e3afdf8f32a7bf146eb39e53d1543c92e6180ba
src/master/master.hpp 6a39df04514c756415354fae66c5835ada191c52
src/master/master.cpp f10a3cffdb0b9831b2b8af0dfd30f73d1d8ed937
src/tests/hierarchical_allocator_tests.cpp eeecfb64540b16666915074aaffaa5d506b203bc
src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124
src/tests/sorter_tests.cpp 7ba32b78bdedf754f76f0695dfd1d86aa1f1b5bf
Diff: https://reviews.apache.org/r/31183/diff/
Testing
-------
make check
Thanks,
Michael Park
Re: Review Request 31183: Distinguished resources from different
slaves in DRFSorter.
Posted by Alexander Rukletsov <al...@mesosphere.io>.
> On Feb. 23, 2015, 7:36 p.m., Alexander Rukletsov wrote:
> > src/master/allocator/sorter/sorter.hpp, lines 64-66
> > <https://reviews.apache.org/r/31183/diff/2/?file=869986#file869986line64>
> >
> > Maybe we can provide overrides here to avoid duplicating `foreach`s in the client code in `HierarchicalAllocator`?
>
> Michael Park wrote:
> I had considered this and pulled back on it because the call-sites currently look like:
>
> ```cpp
> foreachpair (const SlaveID& slaveId, const Resources& allocated, used) {
> roleSorter->allocated(role, slaveId, allocated.unreserved());
> frameworkSorters[role]->add(slaveId, allocated);
> frameworkSorters[role]->allocated(frameworkId.value(), slaveId, allocated);
> }
> ```
>
> With the overload, it doesn't get all that much better because of the `allocated.unreserved()` call.
>
> ```cpp
> foreachpair (const SlaveID& slaveId, const Resources& allocated, used) {
> roleSorter->allocated(role, slaveId, allocated.unreserved());
> }
> frameworkSorters[role]->add(used);
> frameworkSorters[role]->allocated(frameworkId.value(), used);
> ```
>
> I think with 2 such loops we can punt on it for now?
Your comment does make sense to me. I don't have enough context here. The general questions here are: do we want to aggregate resources across slaves somehow? what API do we want to expose (per slave or aggregated)?
- Alexander
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review73586
-----------------------------------------------------------
On Feb. 24, 2015, 11:27 a.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated Feb. 24, 2015, 11:27 a.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> See summary.
>
>
> Diffs
> -----
>
> 3rdparty/libprocess/3rdparty/stout/include/Makefile.am ac2bbed6fe86623fb51cac3613d79d7b1372df9d
> 3rdparty/libprocess/3rdparty/stout/include/stout/ip.hpp 3293086a009a8f7cf7bd343eb7d3e85623636550
> 3rdparty/libprocess/3rdparty/stout/tests/ip_tests.cpp fb98317a68986cb1228c584a8cd83b07737895a8
> configure.ac 9b2d7f15f535aaaf85faf9b4f7af750f1dbdf472
> docs/modules.md a8b471541cdfa584eeb89fbe96643f93c712cfd4
> docs/slave-recovery.md 57eb786f94b2b1dee7bb35017618af90b4dc4a31
> include/mesos/resources.hpp c242bcc29c490841354d6fdc8d0de16eeea602ed
> src/common/resources.cpp a45bbaf696a6cc61a06daaa52a84f0014e7fe8cb
> src/master/allocator/allocator.hpp b67b8fddbd7a3fffc6fe24d5e77cd1db8cb6f69b
> src/master/allocator/mesos/allocator.hpp fb898f1175b61b442204e6e38c69ccc2838a646f
> src/master/allocator/mesos/hierarchical.hpp 9b7ded34ad7546f36dd41f64fe2e3cf41c1cf702
> src/master/allocator/sorter/drf/sorter.hpp 4366710d6530b784aa5094813328d0e338239ba0
> src/master/allocator/sorter/drf/sorter.cpp 2f69f384b95ff20d3ee429a4570a8cffa74d8e8b
> src/master/allocator/sorter/sorter.hpp e2efb27b11dbea42dd73f81e5db0d6d2b0a6034b
> src/master/http.cpp 117c0ee720a60a1d8a25359028bad803f1fc2b07
> src/master/master.hpp 8c44d6ed57ad1b94a17bef8142a5e6a15889a810
> src/master/master.cpp 713307e1be596651283cc2cc95f114c42ad34a5e
> src/slave/containerizer/isolators/network/port_mapping.cpp 5227987cdb7b904c2f4bb2bdf5c5d705a435010d
> src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
> src/tests/master_slave_reconciliation_tests.cpp 54cae62fda6be56ce438371c0e37bdf8cc8ace15
> src/tests/mesos.hpp e17afe74796ffe148274e278b5fa574e5c1bd34c
> src/tests/port_mapping_tests.cpp e2c8ba12b5574b06a6ba60c3c3a30b63cea1d23c
> src/tests/slave_tests.cpp 7ea012ab0883cc030b3e62f59879613866dab0fa
> src/tests/sorter_tests.cpp 42442353afe7bd3d1a5b43992f8ae191ac19bdcd
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Distinguished resources from different
slaves in DRFSorter.
Posted by Alexander Rukletsov <al...@mesosphere.io>.
> On Feb. 23, 2015, 7:36 p.m., Alexander Rukletsov wrote:
> > src/master/allocator/mesos/hierarchical.hpp, lines 348-349
> > <https://reviews.apache.org/r/31183/diff/2/?file=869983#file869983line348>
> >
> > Tabs?
>
> Michael Park wrote:
> Looks like just the way review board displays whitespace changes? I checked the file and I don't see any tabs.
Weird. Feel free to drop then!
- Alexander
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review73586
-----------------------------------------------------------
On Feb. 24, 2015, 9:07 a.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated Feb. 24, 2015, 9:07 a.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> See summary.
>
>
> Diffs
> -----
>
> src/master/allocator/allocator.hpp c2461a32e18dbafe637f37180f86bfa2e1d78735
> src/master/allocator/mesos/allocator.hpp 99b40a01749b77fc7c1fa025a7ac81d5cca6272f
> src/master/allocator/mesos/hierarchical.hpp 2680d6231927867d5a8d75cbc42b81d6c75fc7f2
> src/master/allocator/sorter/drf/sorter.hpp 966fe03de389246dc71e933f9a51ad5b55f2834d
> src/master/allocator/sorter/drf/sorter.cpp 7baee453848c7587472b2e9358c9e4ee6df6e860
> src/master/allocator/sorter/sorter.hpp 8cd70dd91f72109222d8a0cd67cbf66ee4188ab5
> src/master/http.cpp 5e3afdf8f32a7bf146eb39e53d1543c92e6180ba
> src/master/master.hpp 6a39df04514c756415354fae66c5835ada191c52
> src/master/master.cpp f10a3cffdb0b9831b2b8af0dfd30f73d1d8ed937
> src/tests/hierarchical_allocator_tests.cpp eeecfb64540b16666915074aaffaa5d506b203bc
> src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124
> src/tests/sorter_tests.cpp 7ba32b78bdedf754f76f0695dfd1d86aa1f1b5bf
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Distinguished resources from different
slaves in DRFSorter.
Posted by Michael Park <mc...@gmail.com>.
> On Feb. 23, 2015, 7:36 p.m., Alexander Rukletsov wrote:
> > src/master/allocator/mesos/hierarchical.hpp, lines 348-349
> > <https://reviews.apache.org/r/31183/diff/2/?file=869983#file869983line348>
> >
> > Tabs?
Looks like just the way review board displays whitespace changes? I checked the file and I don't see any tabs.
- Michael
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review73586
-----------------------------------------------------------
On Feb. 24, 2015, 9:07 a.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated Feb. 24, 2015, 9:07 a.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> See summary.
>
>
> Diffs
> -----
>
> src/master/allocator/allocator.hpp c2461a32e18dbafe637f37180f86bfa2e1d78735
> src/master/allocator/mesos/allocator.hpp 99b40a01749b77fc7c1fa025a7ac81d5cca6272f
> src/master/allocator/mesos/hierarchical.hpp 2680d6231927867d5a8d75cbc42b81d6c75fc7f2
> src/master/allocator/sorter/drf/sorter.hpp 966fe03de389246dc71e933f9a51ad5b55f2834d
> src/master/allocator/sorter/drf/sorter.cpp 7baee453848c7587472b2e9358c9e4ee6df6e860
> src/master/allocator/sorter/sorter.hpp 8cd70dd91f72109222d8a0cd67cbf66ee4188ab5
> src/master/http.cpp 5e3afdf8f32a7bf146eb39e53d1543c92e6180ba
> src/master/master.hpp 6a39df04514c756415354fae66c5835ada191c52
> src/master/master.cpp f10a3cffdb0b9831b2b8af0dfd30f73d1d8ed937
> src/tests/hierarchical_allocator_tests.cpp eeecfb64540b16666915074aaffaa5d506b203bc
> src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124
> src/tests/sorter_tests.cpp 7ba32b78bdedf754f76f0695dfd1d86aa1f1b5bf
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Distinguished resources from different
slaves in DRFSorter.
Posted by Michael Park <mc...@gmail.com>.
> On Feb. 23, 2015, 7:36 p.m., Alexander Rukletsov wrote:
> > src/master/allocator/sorter/sorter.hpp, lines 64-66
> > <https://reviews.apache.org/r/31183/diff/2/?file=869986#file869986line64>
> >
> > Maybe we can provide overrides here to avoid duplicating `foreach`s in the client code in `HierarchicalAllocator`?
>
> Michael Park wrote:
> I had considered this and pulled back on it because the call-sites currently look like:
>
> ```cpp
> foreachpair (const SlaveID& slaveId, const Resources& allocated, used) {
> roleSorter->allocated(role, slaveId, allocated.unreserved());
> frameworkSorters[role]->add(slaveId, allocated);
> frameworkSorters[role]->allocated(frameworkId.value(), slaveId, allocated);
> }
> ```
>
> With the overload, it doesn't get all that much better because of the `allocated.unreserved()` call.
>
> ```cpp
> foreachpair (const SlaveID& slaveId, const Resources& allocated, used) {
> roleSorter->allocated(role, slaveId, allocated.unreserved());
> }
> frameworkSorters[role]->add(used);
> frameworkSorters[role]->allocated(frameworkId.value(), used);
> ```
>
> I think with 2 such loops we can punt on it for now?
>
> Alexander Rukletsov wrote:
> Your comment does make sense to me. I don't have enough context here. The general questions here are: do we want to aggregate resources across slaves somehow? what API do we want to expose (per slave or aggregated)?
Yeah, BenM actually reached out to Vinod and BenH to get more context on this topic for precisely that question. I think since we already have the notion of `hashmap<SlaveID, Resources>` floating around for Offers related code, this is the less distruptive solution for now. But I guess this is just a prototype patch until we get that question clearly answered.
- Michael
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review73586
-----------------------------------------------------------
On Feb. 25, 2015, 1:16 a.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated Feb. 25, 2015, 1:16 a.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> See summary.
>
>
> Diffs
> -----
>
> include/mesos/resources.hpp da6d48871a0061d8bbf5e681dd6e7edac659d812
> src/common/resources.cpp 2c99b6884d7296099e19e2e3182cbe11b5e1e059
> src/master/allocator/allocator.hpp b67b8fddbd7a3fffc6fe24d5e77cd1db8cb6f69b
> src/master/allocator/mesos/allocator.hpp fb898f1175b61b442204e6e38c69ccc2838a646f
> src/master/allocator/mesos/hierarchical.hpp c0b1da75565d9dc7728c5566f01815234163fc47
> src/master/allocator/sorter/drf/sorter.hpp 4366710d6530b784aa5094813328d0e338239ba0
> src/master/allocator/sorter/drf/sorter.cpp 2f69f384b95ff20d3ee429a4570a8cffa74d8e8b
> src/master/allocator/sorter/sorter.hpp e2efb27b11dbea42dd73f81e5db0d6d2b0a6034b
> src/master/http.cpp 117c0ee720a60a1d8a25359028bad803f1fc2b07
> src/master/master.hpp d414061b1aa7b4cac6ab9a3cc14e8e028f9370d0
> src/master/master.cpp 76e217d16c03e587ea4c0afca94c58b2212f0f93
> src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
> src/tests/mesos.hpp 6b9656b2b858df75e725aaa042c447b755899c87
> src/tests/sorter_tests.cpp 42442353afe7bd3d1a5b43992f8ae191ac19bdcd
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Distinguished resources from different
slaves in DRFSorter.
Posted by Michael Park <mc...@gmail.com>.
> On Feb. 23, 2015, 7:36 p.m., Alexander Rukletsov wrote:
> > src/master/allocator/sorter/sorter.hpp, lines 64-66
> > <https://reviews.apache.org/r/31183/diff/2/?file=869986#file869986line64>
> >
> > Maybe we can provide overrides here to avoid duplicating `foreach`s in the client code in `HierarchicalAllocator`?
I had considered this and pulled back on it because the call-sites currently look like:
```cpp
foreachpair (const SlaveID& slaveId, const Resources& allocated, used) {
roleSorter->allocated(role, slaveId, allocated.unreserved());
frameworkSorters[role]->add(slaveId, allocated);
frameworkSorters[role]->allocated(frameworkId.value(), slaveId, allocated);
}
```
With the overload, it doesn't get all that much better because of the `allocated.unreserved()` call.
```cpp
foreachpair (const SlaveID& slaveId, const Resources& allocated, used) {
roleSorter->allocated(role, slaveId, allocated.unreserved());
}
frameworkSorters[role]->add(used);
frameworkSorters[role]->allocated(frameworkId.value(), used);
```
I think with 2 such loops we can punt on it for now?
> On Feb. 23, 2015, 7:36 p.m., Alexander Rukletsov wrote:
> > src/master/master.hpp, lines 1194-1205
> > <https://reviews.apache.org/r/31183/diff/2/?file=869988#file869988line1194>
> >
> > Looks like this code does the same as one in `master/http.cpp`. Doesn't it make sense to introduce a resource union method? Something like:
> > ```
> > hashmap<SlaveID, Resources> union(
> > hashmap<SlaveID, Resources> arg1,
> > const hashmap<SlaveID, Resources>& arg2)
> > { ...
> > return arg1;
> > }
> > ```
Introduced `operator+=` and `operator+` for `hashmap<SlaveID, Resources>`.
> On Feb. 23, 2015, 7:36 p.m., Alexander Rukletsov wrote:
> > src/master/allocator/sorter/sorter.hpp, lines 72-75
> > <https://reviews.apache.org/r/31183/diff/2/?file=869986#file869986line72>
> >
> > Ditto here and below for `add()` and `remove()`.
See my comment above for `add` and `remove`. For `update`, I think we actually want to take (`slaveId`, `oldAllocation`, `newAllocation`), rather than (`oldSlaveId`, `oldAllocation`, `newSlaveId`, `newAllocation`).
> On Feb. 23, 2015, 7:36 p.m., Alexander Rukletsov wrote:
> > src/master/http.cpp, lines 90-94
> > <https://reviews.apache.org/r/31183/diff/2/?file=869987#file869987line90>
> >
> > Shouldn't it go before Ben Mahler's comment?
Fixed.
- Michael
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review73586
-----------------------------------------------------------
On Feb. 24, 2015, 11:27 a.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated Feb. 24, 2015, 11:27 a.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> See summary.
>
>
> Diffs
> -----
>
> 3rdparty/libprocess/3rdparty/stout/include/Makefile.am ac2bbed6fe86623fb51cac3613d79d7b1372df9d
> 3rdparty/libprocess/3rdparty/stout/include/stout/ip.hpp 3293086a009a8f7cf7bd343eb7d3e85623636550
> 3rdparty/libprocess/3rdparty/stout/tests/ip_tests.cpp fb98317a68986cb1228c584a8cd83b07737895a8
> configure.ac 9b2d7f15f535aaaf85faf9b4f7af750f1dbdf472
> docs/modules.md a8b471541cdfa584eeb89fbe96643f93c712cfd4
> docs/slave-recovery.md 57eb786f94b2b1dee7bb35017618af90b4dc4a31
> include/mesos/resources.hpp c242bcc29c490841354d6fdc8d0de16eeea602ed
> src/common/resources.cpp a45bbaf696a6cc61a06daaa52a84f0014e7fe8cb
> src/master/allocator/allocator.hpp b67b8fddbd7a3fffc6fe24d5e77cd1db8cb6f69b
> src/master/allocator/mesos/allocator.hpp fb898f1175b61b442204e6e38c69ccc2838a646f
> src/master/allocator/mesos/hierarchical.hpp 9b7ded34ad7546f36dd41f64fe2e3cf41c1cf702
> src/master/allocator/sorter/drf/sorter.hpp 4366710d6530b784aa5094813328d0e338239ba0
> src/master/allocator/sorter/drf/sorter.cpp 2f69f384b95ff20d3ee429a4570a8cffa74d8e8b
> src/master/allocator/sorter/sorter.hpp e2efb27b11dbea42dd73f81e5db0d6d2b0a6034b
> src/master/http.cpp 117c0ee720a60a1d8a25359028bad803f1fc2b07
> src/master/master.hpp 8c44d6ed57ad1b94a17bef8142a5e6a15889a810
> src/master/master.cpp 713307e1be596651283cc2cc95f114c42ad34a5e
> src/slave/containerizer/isolators/network/port_mapping.cpp 5227987cdb7b904c2f4bb2bdf5c5d705a435010d
> src/tests/hierarchical_allocator_tests.cpp 93753d1c04159a04a733927a487eb69505438e32
> src/tests/master_slave_reconciliation_tests.cpp 54cae62fda6be56ce438371c0e37bdf8cc8ace15
> src/tests/mesos.hpp e17afe74796ffe148274e278b5fa574e5c1bd34c
> src/tests/port_mapping_tests.cpp e2c8ba12b5574b06a6ba60c3c3a30b63cea1d23c
> src/tests/slave_tests.cpp 7ea012ab0883cc030b3e62f59879613866dab0fa
> src/tests/sorter_tests.cpp 42442353afe7bd3d1a5b43992f8ae191ac19bdcd
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Re: Review Request 31183: Distinguished resources from different
slaves in DRFSorter.
Posted by Alexander Rukletsov <al...@mesosphere.io>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/#review73586
-----------------------------------------------------------
src/master/allocator/mesos/hierarchical.hpp
<https://reviews.apache.org/r/31183/#comment120022>
Tabs?
src/master/allocator/sorter/sorter.hpp
<https://reviews.apache.org/r/31183/#comment120035>
Maybe we can provide overrides here to avoid duplicating `foreach`s in the client code in `HierarchicalAllocator`?
src/master/allocator/sorter/sorter.hpp
<https://reviews.apache.org/r/31183/#comment120036>
Ditto here and below for `add()` and `remove()`.
src/master/http.cpp
<https://reviews.apache.org/r/31183/#comment119983>
Shouldn't it go before Ben Mahler's comment?
src/master/master.hpp
<https://reviews.apache.org/r/31183/#comment120021>
Looks like this code does the same as one in `master/http.cpp`. Doesn't it make sense to introduce a resource union method? Something like:
```
hashmap<SlaveID, Resources> union(
hashmap<SlaveID, Resources> arg1,
const hashmap<SlaveID, Resources>& arg2)
{ ...
return arg1;
}
```
- Alexander Rukletsov
On Feb. 23, 2015, 6:14 p.m., Michael Park wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31183/
> -----------------------------------------------------------
>
> (Updated Feb. 23, 2015, 6:14 p.m.)
>
>
> Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben Mahler, and Jie Yu.
>
>
> Bugs: MESOS-2373
> https://issues.apache.org/jira/browse/MESOS-2373
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Distinguish resources from different slaves in DRFSorter.
>
>
> Diffs
> -----
>
> src/master/allocator/allocator.hpp c2461a32e18dbafe637f37180f86bfa2e1d78735
> src/master/allocator/mesos/allocator.hpp 99b40a01749b77fc7c1fa025a7ac81d5cca6272f
> src/master/allocator/mesos/hierarchical.hpp 2680d6231927867d5a8d75cbc42b81d6c75fc7f2
> src/master/allocator/sorter/drf/sorter.hpp 966fe03de389246dc71e933f9a51ad5b55f2834d
> src/master/allocator/sorter/drf/sorter.cpp 7baee453848c7587472b2e9358c9e4ee6df6e860
> src/master/allocator/sorter/sorter.hpp 8cd70dd91f72109222d8a0cd67cbf66ee4188ab5
> src/master/http.cpp 5e3afdf8f32a7bf146eb39e53d1543c92e6180ba
> src/master/master.hpp 6a39df04514c756415354fae66c5835ada191c52
> src/master/master.cpp f10a3cffdb0b9831b2b8af0dfd30f73d1d8ed937
> src/tests/hierarchical_allocator_tests.cpp eeecfb64540b16666915074aaffaa5d506b203bc
> src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124
> src/tests/sorter_tests.cpp 7ba32b78bdedf754f76f0695dfd1d86aa1f1b5bf
>
> Diff: https://reviews.apache.org/r/31183/diff/
>
>
> Testing
> -------
>
> make check
>
>
> Thanks,
>
> Michael Park
>
>
Fwd: Review Request 31183: Distinguished resources from different
slaves in DRFSorter.
Posted by Benjamin Mahler <be...@gmail.com>.
+ben and vinod for more context
This review fixes the issue related to aggregating 'Resources' across
slaves:
https://issues.apache.org/jira/browse/MESOS-2373
Today, this only works for scalars inside 'Resources' objects. For sets
(and ranges, which are sets), 'Resources' will merge these as it doesn't
have an understanding of the namespacing of resources. For example
([31000-32000]) + ([31000-32000]) = ([31000-32000]), not
([31000-32000], [31000-32000]).
There's kind of two approaches here:
(1) Accept that all 'Resources' objects are implicitly namespaced (e.g. by
SlaveID), and they cannot be merged across namespaces. For example, you
can't sum the ports across slaves because it will try to merge them
together. e.g. [31000-32000] + [31000-32000] = [31000-32000], which leads
to information loss. This means we have to update all 'Resources'
aggregation to deal with the namespacing. Often this means 'Resources'
becomes 'hashmap<SlaveID, Resources>'.
(2) But (1) seems to go against the original intent of the code, where
there seems to be an assumption that aggregating 'Resources' across slaves
is safe.
I was leaning towards (1) where we add explicit namespacing to our
aggregated Resources. But that seems a bit clunky. Ben and Vinod, do you
guys have more context on this? Was the intent to make 'Resources'
correctly handle aggregation across slaves? Or was it an oversight that
'Resources' cannot be aggregated across slaves?
---------- Forwarded message ----------
From: Michael Park <mc...@gmail.com>
Date: Mon, Feb 23, 2015 at 10:14 AM
Subject: Re: Review Request 31183: Distinguished resources from different
slaves in DRFSorter.
To: Alexander Rukletsov <al...@mesosphere.io>, Benjamin Hindman <
benh@berkeley.edu>, Ben Mahler <be...@gmail.com>, Jie Yu <
yujie.jay@gmail.com>
Cc: Mesos ReviewBot <de...@mesos.apache.org>, Till Toenshoff <to...@me.com>,
Michael Park <mc...@gmail.com>, Alexander Rojas <al...@mesosphere.io>
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31183/
-----------------------------------------------------------
(Updated Feb. 23, 2015, 6:14 p.m.)
Review request for mesos, Alexander Rukletsov, Benjamin Hindman, Ben
Mahler, and Jie Yu.
Changes
-------
s/Distinguish/Distinguished
Summary (updated)
-----------------
Distinguished resources from different slaves in DRFSorter.
Bugs: MESOS-2373
https://issues.apache.org/jira/browse/MESOS-2373
Repository: mesos
Description
-------
Distinguish resources from different slaves in DRFSorter.
Diffs
-----
src/master/allocator/allocator.hpp
c2461a32e18dbafe637f37180f86bfa2e1d78735
src/master/allocator/mesos/allocator.hpp
99b40a01749b77fc7c1fa025a7ac81d5cca6272f
src/master/allocator/mesos/hierarchical.hpp
2680d6231927867d5a8d75cbc42b81d6c75fc7f2
src/master/allocator/sorter/drf/sorter.hpp
966fe03de389246dc71e933f9a51ad5b55f2834d
src/master/allocator/sorter/drf/sorter.cpp
7baee453848c7587472b2e9358c9e4ee6df6e860
src/master/allocator/sorter/sorter.hpp
8cd70dd91f72109222d8a0cd67cbf66ee4188ab5
src/master/http.cpp 5e3afdf8f32a7bf146eb39e53d1543c92e6180ba
src/master/master.hpp 6a39df04514c756415354fae66c5835ada191c52
src/master/master.cpp f10a3cffdb0b9831b2b8af0dfd30f73d1d8ed937
src/tests/hierarchical_allocator_tests.cpp
eeecfb64540b16666915074aaffaa5d506b203bc
src/tests/mesos.hpp 60c70043c8266a422ffa03ab5a949da0bc822124
src/tests/sorter_tests.cpp 7ba32b78bdedf754f76f0695dfd1d86aa1f1b5bf
Diff: https://reviews.apache.org/r/31183/diff/
Testing
-------
make check
Thanks,
Michael Park