You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by Benjamin Bannier <be...@mesosphere.io> on 2017/02/03 10:45:35 UTC

Re: Review Request 51027: Track allocation candidates to bound allocator.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51027/#review164111
-----------------------------------------------------------




src/master/allocator/mesos/hierarchical.cpp (line 1257)
<https://reviews.apache.org/r/51027/#comment235691>

    Could you follow up with a patch so this call is `dispatch`'ed?
    
    Right now, if the `HierarchicalAllocatorProcess` goes away after `allocate`, but before the `AnyCallback` finishes, `this->allocationInterval` might be referencing garbage.


- Benjamin Bannier


On Jan. 31, 2017, 2:46 a.m., Jacob Janco wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51027/
> -----------------------------------------------------------
> 
> (Updated Jan. 31, 2017, 2:46 a.m.)
> 
> 
> Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus Ma, and Jiang Yan Xu.
> 
> 
> Bugs: MESOS-6904
>     https://issues.apache.org/jira/browse/MESOS-6904
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> - Triggered allocations dispatch allocate() only
>   if there is no pending allocation in the queue.
> - Allocation candidates are accumulated and only
>   cleared when enqueued allocations are processed.
> 
> 
> Diffs
> -----
> 
>   src/master/allocator/mesos/hierarchical.hpp 2cda3e311ce339d82065d68de83e86439fa192ff 
>   src/master/allocator/mesos/hierarchical.cpp f471b6848bebae601a7a0509e9c6ad5eab4fa4a2 
> 
> Diff: https://reviews.apache.org/r/51027/diff/
> 
> 
> Testing
> -------
> 
> make check with the filters below
> 
> Broken tests: 
> - TEST_F(HierarchicalAllocatorTest, SuppressAndReviveOffers), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, AllocationRunsMetric), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, AllocationRunTimerMetrics), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, UpdateWeight), fix in 51028
> - TEST_P(HierarchicalAllocator_BENCHMARK_Test, AddAndUpdateSlave), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, SmallOfferFilterTimeout), fix in 52534
> - TEST_F(OversubscriptionTest, RescindRevocableOfferWithIncreasedRevocable), fix in 51621
> 
> 
> Thanks,
> 
> Jacob Janco
> 
>

Re: Review Request 51027: Track allocation candidates to bound allocator.

Posted by Benjamin Bannier <be...@mesosphere.io>.

> On Feb. 3, 2017, 11:45 a.m., Benjamin Bannier wrote:
> > src/master/allocator/mesos/hierarchical.cpp, line 1263
> > <https://reviews.apache.org/r/51027/diff/14/?file=1619668#file1619668line1263>
> >
> >     Could you follow up with a patch so this call is `dispatch`'ed?
> >     
> >     Right now, if the `HierarchicalAllocatorProcess` goes away after `allocate`, but before the `AnyCallback` finishes, `this->allocationInterval` might be referencing garbage.
> 
> Jiang Yan Xu wrote:
>     `delay` is already dispatched isn't it?

Yes, the `delay`'ed part is safe as it dispatches to a specific process. The issue here is with the `AnyCallBack` (the full argument to `onAny`) since it does capture a raw ptr (`this`), but is not dispatched.


- Benjamin


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51027/#review164111
-----------------------------------------------------------


On Jan. 31, 2017, 2:46 a.m., Jacob Janco wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51027/
> -----------------------------------------------------------
> 
> (Updated Jan. 31, 2017, 2:46 a.m.)
> 
> 
> Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus Ma, and Jiang Yan Xu.
> 
> 
> Bugs: MESOS-6904
>     https://issues.apache.org/jira/browse/MESOS-6904
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> - Triggered allocations dispatch allocate() only
>   if there is no pending allocation in the queue.
> - Allocation candidates are accumulated and only
>   cleared when enqueued allocations are processed.
> 
> 
> Diffs
> -----
> 
>   src/master/allocator/mesos/hierarchical.hpp 2cda3e311ce339d82065d68de83e86439fa192ff 
>   src/master/allocator/mesos/hierarchical.cpp f471b6848bebae601a7a0509e9c6ad5eab4fa4a2 
> 
> Diff: https://reviews.apache.org/r/51027/diff/
> 
> 
> Testing
> -------
> 
> make check with the filters below
> 
> Broken tests: 
> - TEST_F(HierarchicalAllocatorTest, SuppressAndReviveOffers), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, AllocationRunsMetric), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, AllocationRunTimerMetrics), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, UpdateWeight), fix in 51028
> - TEST_P(HierarchicalAllocator_BENCHMARK_Test, AddAndUpdateSlave), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, SmallOfferFilterTimeout), fix in 52534
> - TEST_F(OversubscriptionTest, RescindRevocableOfferWithIncreasedRevocable), fix in 51621
> 
> 
> Thanks,
> 
> Jacob Janco
> 
>

Re: Review Request 51027: Track allocation candidates to bound allocator.

Posted by Jacob Janco <jj...@gmail.com>.

> On Feb. 3, 2017, 10:45 a.m., Benjamin Bannier wrote:
> > src/master/allocator/mesos/hierarchical.cpp, line 1263
> > <https://reviews.apache.org/r/51027/diff/14/?file=1619668#file1619668line1263>
> >
> >     Could you follow up with a patch so this call is `dispatch`'ed?
> >     
> >     Right now, if the `HierarchicalAllocatorProcess` goes away after `allocate`, but before the `AnyCallback` finishes, `this->allocationInterval` might be referencing garbage.
> 
> Jiang Yan Xu wrote:
>     `delay` is already dispatched isn't it?
> 
> Benjamin Bannier wrote:
>     Yes, the `delay`'ed part is safe as it dispatches to a specific process. The issue here is with the `AnyCallBack` (the full argument to `onAny`) since it does capture a raw ptr (`this`), but is not dispatched.
> 
> Jiang Yan Xu wrote:
>     Ah I see what you mean now, thanks! I guess we can copy `allocationInterval` in.
> 
> Jacob Janco wrote:
>     ^I'll submit a patch with the above fix.

https://reviews.apache.org/r/56296


- Jacob


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51027/#review164111
-----------------------------------------------------------


On Jan. 31, 2017, 1:46 a.m., Jacob Janco wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51027/
> -----------------------------------------------------------
> 
> (Updated Jan. 31, 2017, 1:46 a.m.)
> 
> 
> Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus Ma, and Jiang Yan Xu.
> 
> 
> Bugs: MESOS-6904
>     https://issues.apache.org/jira/browse/MESOS-6904
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> - Triggered allocations dispatch allocate() only
>   if there is no pending allocation in the queue.
> - Allocation candidates are accumulated and only
>   cleared when enqueued allocations are processed.
> 
> 
> Diffs
> -----
> 
>   src/master/allocator/mesos/hierarchical.hpp 2cda3e311ce339d82065d68de83e86439fa192ff 
>   src/master/allocator/mesos/hierarchical.cpp f471b6848bebae601a7a0509e9c6ad5eab4fa4a2 
> 
> Diff: https://reviews.apache.org/r/51027/diff/
> 
> 
> Testing
> -------
> 
> make check with the filters below
> 
> Broken tests: 
> - TEST_F(HierarchicalAllocatorTest, SuppressAndReviveOffers), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, AllocationRunsMetric), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, AllocationRunTimerMetrics), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, UpdateWeight), fix in 51028
> - TEST_P(HierarchicalAllocator_BENCHMARK_Test, AddAndUpdateSlave), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, SmallOfferFilterTimeout), fix in 52534
> - TEST_F(OversubscriptionTest, RescindRevocableOfferWithIncreasedRevocable), fix in 51621
> 
> 
> Thanks,
> 
> Jacob Janco
> 
>

Re: Review Request 51027: Track allocation candidates to bound allocator.

Posted by Jiang Yan Xu <xu...@apple.com>.

> On Feb. 3, 2017, 2:45 a.m., Benjamin Bannier wrote:
> > src/master/allocator/mesos/hierarchical.cpp, line 1263
> > <https://reviews.apache.org/r/51027/diff/14/?file=1619668#file1619668line1263>
> >
> >     Could you follow up with a patch so this call is `dispatch`'ed?
> >     
> >     Right now, if the `HierarchicalAllocatorProcess` goes away after `allocate`, but before the `AnyCallback` finishes, `this->allocationInterval` might be referencing garbage.
> 
> Jiang Yan Xu wrote:
>     `delay` is already dispatched isn't it?
> 
> Benjamin Bannier wrote:
>     Yes, the `delay`'ed part is safe as it dispatches to a specific process. The issue here is with the `AnyCallBack` (the full argument to `onAny`) since it does capture a raw ptr (`this`), but is not dispatched.

Ah I see what you mean now, thanks! I guess we can copy `allocationInterval` in.


- Jiang Yan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51027/#review164111
-----------------------------------------------------------


On Jan. 30, 2017, 5:46 p.m., Jacob Janco wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51027/
> -----------------------------------------------------------
> 
> (Updated Jan. 30, 2017, 5:46 p.m.)
> 
> 
> Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus Ma, and Jiang Yan Xu.
> 
> 
> Bugs: MESOS-6904
>     https://issues.apache.org/jira/browse/MESOS-6904
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> - Triggered allocations dispatch allocate() only
>   if there is no pending allocation in the queue.
> - Allocation candidates are accumulated and only
>   cleared when enqueued allocations are processed.
> 
> 
> Diffs
> -----
> 
>   src/master/allocator/mesos/hierarchical.hpp 2cda3e311ce339d82065d68de83e86439fa192ff 
>   src/master/allocator/mesos/hierarchical.cpp f471b6848bebae601a7a0509e9c6ad5eab4fa4a2 
> 
> Diff: https://reviews.apache.org/r/51027/diff/
> 
> 
> Testing
> -------
> 
> make check with the filters below
> 
> Broken tests: 
> - TEST_F(HierarchicalAllocatorTest, SuppressAndReviveOffers), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, AllocationRunsMetric), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, AllocationRunTimerMetrics), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, UpdateWeight), fix in 51028
> - TEST_P(HierarchicalAllocator_BENCHMARK_Test, AddAndUpdateSlave), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, SmallOfferFilterTimeout), fix in 52534
> - TEST_F(OversubscriptionTest, RescindRevocableOfferWithIncreasedRevocable), fix in 51621
> 
> 
> Thanks,
> 
> Jacob Janco
> 
>

Re: Review Request 51027: Track allocation candidates to bound allocator.

Posted by Jiang Yan Xu <xu...@apple.com>.

> On Feb. 3, 2017, 2:45 a.m., Benjamin Bannier wrote:
> > src/master/allocator/mesos/hierarchical.cpp, line 1263
> > <https://reviews.apache.org/r/51027/diff/14/?file=1619668#file1619668line1263>
> >
> >     Could you follow up with a patch so this call is `dispatch`'ed?
> >     
> >     Right now, if the `HierarchicalAllocatorProcess` goes away after `allocate`, but before the `AnyCallback` finishes, `this->allocationInterval` might be referencing garbage.

`delay` is already dispatched isn't it?


- Jiang Yan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51027/#review164111
-----------------------------------------------------------


On Jan. 30, 2017, 5:46 p.m., Jacob Janco wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51027/
> -----------------------------------------------------------
> 
> (Updated Jan. 30, 2017, 5:46 p.m.)
> 
> 
> Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus Ma, and Jiang Yan Xu.
> 
> 
> Bugs: MESOS-6904
>     https://issues.apache.org/jira/browse/MESOS-6904
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> - Triggered allocations dispatch allocate() only
>   if there is no pending allocation in the queue.
> - Allocation candidates are accumulated and only
>   cleared when enqueued allocations are processed.
> 
> 
> Diffs
> -----
> 
>   src/master/allocator/mesos/hierarchical.hpp 2cda3e311ce339d82065d68de83e86439fa192ff 
>   src/master/allocator/mesos/hierarchical.cpp f471b6848bebae601a7a0509e9c6ad5eab4fa4a2 
> 
> Diff: https://reviews.apache.org/r/51027/diff/
> 
> 
> Testing
> -------
> 
> make check with the filters below
> 
> Broken tests: 
> - TEST_F(HierarchicalAllocatorTest, SuppressAndReviveOffers), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, AllocationRunsMetric), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, AllocationRunTimerMetrics), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, UpdateWeight), fix in 51028
> - TEST_P(HierarchicalAllocator_BENCHMARK_Test, AddAndUpdateSlave), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, SmallOfferFilterTimeout), fix in 52534
> - TEST_F(OversubscriptionTest, RescindRevocableOfferWithIncreasedRevocable), fix in 51621
> 
> 
> Thanks,
> 
> Jacob Janco
> 
>

Re: Review Request 51027: Track allocation candidates to bound allocator.

Posted by Jacob Janco <jj...@gmail.com>.

> On Feb. 3, 2017, 10:45 a.m., Benjamin Bannier wrote:
> > src/master/allocator/mesos/hierarchical.cpp, line 1263
> > <https://reviews.apache.org/r/51027/diff/14/?file=1619668#file1619668line1263>
> >
> >     Could you follow up with a patch so this call is `dispatch`'ed?
> >     
> >     Right now, if the `HierarchicalAllocatorProcess` goes away after `allocate`, but before the `AnyCallback` finishes, `this->allocationInterval` might be referencing garbage.
> 
> Jiang Yan Xu wrote:
>     `delay` is already dispatched isn't it?
> 
> Benjamin Bannier wrote:
>     Yes, the `delay`'ed part is safe as it dispatches to a specific process. The issue here is with the `AnyCallBack` (the full argument to `onAny`) since it does capture a raw ptr (`this`), but is not dispatched.
> 
> Jiang Yan Xu wrote:
>     Ah I see what you mean now, thanks! I guess we can copy `allocationInterval` in.

^I'll submit a patch with the above fix.


- Jacob


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51027/#review164111
-----------------------------------------------------------


On Jan. 31, 2017, 1:46 a.m., Jacob Janco wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51027/
> -----------------------------------------------------------
> 
> (Updated Jan. 31, 2017, 1:46 a.m.)
> 
> 
> Review request for mesos, Benjamin Mahler, Guangya Liu, James Peach, Klaus Ma, and Jiang Yan Xu.
> 
> 
> Bugs: MESOS-6904
>     https://issues.apache.org/jira/browse/MESOS-6904
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> - Triggered allocations dispatch allocate() only
>   if there is no pending allocation in the queue.
> - Allocation candidates are accumulated and only
>   cleared when enqueued allocations are processed.
> 
> 
> Diffs
> -----
> 
>   src/master/allocator/mesos/hierarchical.hpp 2cda3e311ce339d82065d68de83e86439fa192ff 
>   src/master/allocator/mesos/hierarchical.cpp f471b6848bebae601a7a0509e9c6ad5eab4fa4a2 
> 
> Diff: https://reviews.apache.org/r/51027/diff/
> 
> 
> Testing
> -------
> 
> make check with the filters below
> 
> Broken tests: 
> - TEST_F(HierarchicalAllocatorTest, SuppressAndReviveOffers), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, AllocationRunsMetric), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, AllocationRunTimerMetrics), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, UpdateWeight), fix in 51028
> - TEST_P(HierarchicalAllocator_BENCHMARK_Test, AddAndUpdateSlave), fix in 51028
> - TEST_F(HierarchicalAllocatorTest, SmallOfferFilterTimeout), fix in 52534
> - TEST_F(OversubscriptionTest, RescindRevocableOfferWithIncreasedRevocable), fix in 51621
> 
> 
> Thanks,
> 
> Jacob Janco
> 
>