You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by Benjamin Mahler <bm...@apache.org> on 2017/01/23 22:59:34 UTC
Review Request 55863: Introduce a helper for injecting AllocationInfo
into offer operations.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55863/
-----------------------------------------------------------
Review request for mesos, Benjamin Bannier, Jay Guo, Guangya Liu, and Michael Park.
Bugs: MESOS-6967
https://issues.apache.org/jira/browse/MESOS-6967
Repository: mesos
Description
-------
Previously, `Resource` did not contain `AllocationInfo`. So for
backwards compatibility with old schedulers and tooling, we must
allow operations to contain `Resource`s without an `AllocationInfo`.
This introduces a function which allows the master to inject the
offer's `AllocationInfo` into the operation's resources.
Diffs
-----
src/common/protobuf_utils.hpp faa7e2a759fd2b1ce5def662679f573ec6fefd28
src/common/protobuf_utils.cpp dd20759affe3adf2867a33bf875c9ee82862038d
src/tests/protobuf_utils_tests.cpp bc2a3d0ebe544a58d0de8f2f7174c170113ddf2e
Diff: https://reviews.apache.org/r/55863/diff/
Testing
-------
Added a test.
Thanks,
Benjamin Mahler
Re: Review Request 55863: Introduce a helper for injecting
AllocationInfo into offer operations.
Posted by Benjamin Mahler <bm...@apache.org>.
> On Jan. 30, 2017, 1:23 a.m., Michael Park wrote:
> > src/common/protobuf_utils.cpp, lines 332-335
> > <https://reviews.apache.org/r/55863/diff/1/?file=1613193#file1613193line332>
> >
> > Is there a reason why we don't just mutate it in-place?
> > ```cpp
> > void adjustOfferOperation(
> > Offer::Operation* operation,
> > const Resource::AllocationInfo& allocationInfo);
> > ```
Sounds good to me.
> On Jan. 30, 2017, 1:23 a.m., Michael Park wrote:
> > src/common/protobuf_utils.cpp, lines 340-341
> > <https://reviews.apache.org/r/55863/diff/1/?file=1613193#file1613193line340>
> >
> > I think it'd be clearer for us to write:
> >
> > ```cpp
> > foreach (TaskInfo& task, *result.launch().mutable_task_infos()) {
> > // ...
> > }
> > ```
Now I remember why I avoided this, note that it has to be mutable the whole way through:
```
foreach (TaskInfo& task,
*operation->mutable_launch()->mutable_task_infos()) {
// ...
}
```
I'll work around this by storing the launch and other operations into variables.
> On Jan. 30, 2017, 1:23 a.m., Michael Park wrote:
> > src/common/protobuf_utils.cpp, lines 343-348
> > <https://reviews.apache.org/r/55863/diff/1/?file=1613193#file1613193line343>
> >
> > We basically do this for every repeated resources field within `Offer::Operation`. Can we introduce a function similar to `adjustOfferOperation`?
> >
> > ```cpp
> > void adjustResources(
> > RepeatedPtrField<Resource>* resources,
> > const Resource::AllocationInfo& allocationInfo);
> > ```
> >
> > The `injectAllocationInfo` helper which operates on `FrameworkInfo`
> > I think should just be:
> >
> > ```cpp
> > auto injectAllocationInfo = [](
> > RepeatedPtrField<Resource>* resources,
> > const FrameworkInfo& frameworkInfo)
> > {
> > if (protobuf::frameworkHasCapability(
> > frameworkInfo,
> > FrameworkInfo::Capability::MULTI_ROLE) {
> > return;
> > }
> >
> > Resource::AllocationInfo allocationInfo;
> > allocationInfo.set_role(frameworkInfo.role);
> > adjustResources(resources, allocationInfo);
> > };
> > ```
Added a lambda to simplify this implementation. Will look into your suggestion for the subsequent patch.
- Benjamin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55863/#review163448
-----------------------------------------------------------
On Jan. 23, 2017, 10:59 p.m., Benjamin Mahler wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55863/
> -----------------------------------------------------------
>
> (Updated Jan. 23, 2017, 10:59 p.m.)
>
>
> Review request for mesos, Benjamin Bannier, Jay Guo, Guangya Liu, and Michael Park.
>
>
> Bugs: MESOS-6967
> https://issues.apache.org/jira/browse/MESOS-6967
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Previously, `Resource` did not contain `AllocationInfo`. So for
> backwards compatibility with old schedulers and tooling, we must
> allow operations to contain `Resource`s without an `AllocationInfo`.
>
> This introduces a function which allows the master to inject the
> offer's `AllocationInfo` into the operation's resources.
>
>
> Diffs
> -----
>
> src/common/protobuf_utils.hpp faa7e2a759fd2b1ce5def662679f573ec6fefd28
> src/common/protobuf_utils.cpp dd20759affe3adf2867a33bf875c9ee82862038d
> src/tests/protobuf_utils_tests.cpp bc2a3d0ebe544a58d0de8f2f7174c170113ddf2e
>
> Diff: https://reviews.apache.org/r/55863/diff/
>
>
> Testing
> -------
>
> Added a test.
>
>
> Thanks,
>
> Benjamin Mahler
>
>
Re: Review Request 55863: Introduce a helper for injecting
AllocationInfo into offer operations.
Posted by Michael Park <mp...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55863/#review163448
-----------------------------------------------------------
Fix it, then Ship it!
src/common/protobuf_utils.cpp (lines 332 - 335)
<https://reviews.apache.org/r/55863/#comment234913>
Is there a reason why we don't just mutate it in-place?
```cpp
void adjustOfferOperation(
Offer::Operation* operation,
const Resource::AllocationInfo& allocationInfo);
```
src/common/protobuf_utils.cpp (lines 340 - 341)
<https://reviews.apache.org/r/55863/#comment234909>
I think it'd be clearer for us to write:
```cpp
foreach (TaskInfo& task, *result.launch().mutable_task_infos()) {
// ...
}
```
src/common/protobuf_utils.cpp (lines 343 - 348)
<https://reviews.apache.org/r/55863/#comment234914>
We basically do this for every repeated resources field within `Offer::Operation`. Can we introduce a function similar to `adjustOfferOperation`?
```cpp
void adjustResources(
RepeatedPtrField<Resource>* resources,
const Resource::AllocationInfo& allocationInfo);
```
The `injectAllocationInfo` helper which operates on `FrameworkInfo`
I think should just be:
```cpp
auto injectAllocationInfo = [](
RepeatedPtrField<Resource>* resources,
const FrameworkInfo& frameworkInfo)
{
if (protobuf::frameworkHasCapability(
frameworkInfo,
FrameworkInfo::Capability::MULTI_ROLE) {
return;
}
Resource::AllocationInfo allocationInfo;
allocationInfo.set_role(frameworkInfo.role);
adjustResources(resources, allocationInfo);
};
```
- Michael Park
On Jan. 23, 2017, 2:59 p.m., Benjamin Mahler wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55863/
> -----------------------------------------------------------
>
> (Updated Jan. 23, 2017, 2:59 p.m.)
>
>
> Review request for mesos, Benjamin Bannier, Jay Guo, Guangya Liu, and Michael Park.
>
>
> Bugs: MESOS-6967
> https://issues.apache.org/jira/browse/MESOS-6967
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Previously, `Resource` did not contain `AllocationInfo`. So for
> backwards compatibility with old schedulers and tooling, we must
> allow operations to contain `Resource`s without an `AllocationInfo`.
>
> This introduces a function which allows the master to inject the
> offer's `AllocationInfo` into the operation's resources.
>
>
> Diffs
> -----
>
> src/common/protobuf_utils.hpp faa7e2a759fd2b1ce5def662679f573ec6fefd28
> src/common/protobuf_utils.cpp dd20759affe3adf2867a33bf875c9ee82862038d
> src/tests/protobuf_utils_tests.cpp bc2a3d0ebe544a58d0de8f2f7174c170113ddf2e
>
> Diff: https://reviews.apache.org/r/55863/diff/
>
>
> Testing
> -------
>
> Added a test.
>
>
> Thanks,
>
> Benjamin Mahler
>
>
Re: Review Request 55863: Introduce a helper for injecting
AllocationInfo into offer operations.
Posted by Guangya Liu <gy...@gmail.com>.
> On \u4e00\u6708 28, 2017, 11:02 a.m., Guangya Liu wrote:
> > src/common/protobuf_utils.cpp, lines 345-347
> > <https://reviews.apache.org/r/55863/diff/1/?file=1613193#file1613193line345>
> >
> > Since we do not support one `Resources` store a mix of allocated and unallocated resources, how about optimize this a bit as:
> >
> > ```
> > if (resource->has_allocation_info()) {
> > break;
> > }
> >
> > resource->mutable_allocation_info()->CopyFrom(allocationInfo);
> > ```
> >
> > Ditto for here and everywhere
>
> Benjamin Mahler wrote:
> Well, we do support mixing in Resources, but we likely do not want to. I would like to have a stronger invariant that callers can't store a mix of allocated and unallocated resources, since I don't see a good reason for it and it's error prone.
@bmahler, can you please show more detail in which case there are mixing in resources?
- Guangya
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55863/#review163396
-----------------------------------------------------------
On \u4e00\u6708 23, 2017, 10:59 p.m., Benjamin Mahler wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55863/
> -----------------------------------------------------------
>
> (Updated \u4e00\u6708 23, 2017, 10:59 p.m.)
>
>
> Review request for mesos, Benjamin Bannier, Jay Guo, Guangya Liu, and Michael Park.
>
>
> Bugs: MESOS-6967
> https://issues.apache.org/jira/browse/MESOS-6967
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Previously, `Resource` did not contain `AllocationInfo`. So for
> backwards compatibility with old schedulers and tooling, we must
> allow operations to contain `Resource`s without an `AllocationInfo`.
>
> This introduces a function which allows the master to inject the
> offer's `AllocationInfo` into the operation's resources.
>
>
> Diffs
> -----
>
> src/common/protobuf_utils.hpp faa7e2a759fd2b1ce5def662679f573ec6fefd28
> src/common/protobuf_utils.cpp dd20759affe3adf2867a33bf875c9ee82862038d
> src/tests/protobuf_utils_tests.cpp bc2a3d0ebe544a58d0de8f2f7174c170113ddf2e
>
> Diff: https://reviews.apache.org/r/55863/diff/
>
>
> Testing
> -------
>
> Added a test.
>
>
> Thanks,
>
> Benjamin Mahler
>
>
Re: Review Request 55863: Introduce a helper for injecting
AllocationInfo into offer operations.
Posted by Benjamin Mahler <bm...@apache.org>.
> On Jan. 28, 2017, 11:02 a.m., Guangya Liu wrote:
> > src/common/protobuf_utils.cpp, lines 345-347
> > <https://reviews.apache.org/r/55863/diff/1/?file=1613193#file1613193line345>
> >
> > Since we do not support one `Resources` store a mix of allocated and unallocated resources, how about optimize this a bit as:
> >
> > ```
> > if (resource->has_allocation_info()) {
> > break;
> > }
> >
> > resource->mutable_allocation_info()->CopyFrom(allocationInfo);
> > ```
> >
> > Ditto for here and everywhere
Well, we do support mixing in Resources, but we likely do not want to. I would like to have a stronger invariant that callers can't store a mix of allocated and unallocated resources, since I don't see a good reason for it and it's error prone.
- Benjamin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55863/#review163396
-----------------------------------------------------------
On Jan. 23, 2017, 10:59 p.m., Benjamin Mahler wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55863/
> -----------------------------------------------------------
>
> (Updated Jan. 23, 2017, 10:59 p.m.)
>
>
> Review request for mesos, Benjamin Bannier, Jay Guo, Guangya Liu, and Michael Park.
>
>
> Bugs: MESOS-6967
> https://issues.apache.org/jira/browse/MESOS-6967
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Previously, `Resource` did not contain `AllocationInfo`. So for
> backwards compatibility with old schedulers and tooling, we must
> allow operations to contain `Resource`s without an `AllocationInfo`.
>
> This introduces a function which allows the master to inject the
> offer's `AllocationInfo` into the operation's resources.
>
>
> Diffs
> -----
>
> src/common/protobuf_utils.hpp faa7e2a759fd2b1ce5def662679f573ec6fefd28
> src/common/protobuf_utils.cpp dd20759affe3adf2867a33bf875c9ee82862038d
> src/tests/protobuf_utils_tests.cpp bc2a3d0ebe544a58d0de8f2f7174c170113ddf2e
>
> Diff: https://reviews.apache.org/r/55863/diff/
>
>
> Testing
> -------
>
> Added a test.
>
>
> Thanks,
>
> Benjamin Mahler
>
>
Re: Review Request 55863: Introduce a helper for injecting
AllocationInfo into offer operations.
Posted by Guangya Liu <gy...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55863/#review163396
-----------------------------------------------------------
src/common/protobuf_utils.cpp (lines 345 - 347)
<https://reviews.apache.org/r/55863/#comment234834>
Since we do not support one `Resources` store a mix of allocated and unallocated resources, how about optimize this a bit as:
```
if (resource->has_allocation_info()) {
break;
}
resource->mutable_allocation_info()->CopyFrom(allocationInfo);
```
Ditto for here and everywhere
src/tests/protobuf_utils_tests.cpp (lines 101 - 103)
<https://reviews.apache.org/r/55863/#comment234835>
How about make this less jagged
```
// Test the LAUNCH_GROUP case. This should be constructing a valid
// task and executor, but for now this just sets the resources in
// order to verify the allocation info injection.
```
Ditto here and following comments.
- Guangya Liu
On \u4e00\u6708 23, 2017, 10:59 p.m., Benjamin Mahler wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55863/
> -----------------------------------------------------------
>
> (Updated \u4e00\u6708 23, 2017, 10:59 p.m.)
>
>
> Review request for mesos, Benjamin Bannier, Jay Guo, Guangya Liu, and Michael Park.
>
>
> Bugs: MESOS-6967
> https://issues.apache.org/jira/browse/MESOS-6967
>
>
> Repository: mesos
>
>
> Description
> -------
>
> Previously, `Resource` did not contain `AllocationInfo`. So for
> backwards compatibility with old schedulers and tooling, we must
> allow operations to contain `Resource`s without an `AllocationInfo`.
>
> This introduces a function which allows the master to inject the
> offer's `AllocationInfo` into the operation's resources.
>
>
> Diffs
> -----
>
> src/common/protobuf_utils.hpp faa7e2a759fd2b1ce5def662679f573ec6fefd28
> src/common/protobuf_utils.cpp dd20759affe3adf2867a33bf875c9ee82862038d
> src/tests/protobuf_utils_tests.cpp bc2a3d0ebe544a58d0de8f2f7174c170113ddf2e
>
> Diff: https://reviews.apache.org/r/55863/diff/
>
>
> Testing
> -------
>
> Added a test.
>
>
> Thanks,
>
> Benjamin Mahler
>
>