You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by Zhitao Li <zh...@gmail.com> on 2016/01/14 15:46:40 UTC
Re: Review Request 42126: Accept resource offers from multiple
framework roles.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42126/
-----------------------------------------------------------
(Updated Jan. 14, 2016, 6:46 a.m.)
Review request for Aurora, Maxim Khutornenko, Dmitriy Shirchenko, and Bill Farner.
Changes
-------
Tweaked review summary in preparation for commit.
-wfarner
Summary (updated)
-----------------
Accept resource offers from multiple framework roles.
Bugs: AURORA-1109
https://issues.apache.org/jira/browse/AURORA-1109
Repository: aurora
Description
-------
This review is a prototype for introducing multiple role support in Aurora.
This creates a new class OfferAllocation, which allcoates resources to resources field in TaskInfo and ExecutorInfo from an offer.
Current implementation prefers reserved resources over shared resources ('*' role) if both are present
Several caveats:
1. This performs the allocate after scheduling decision in TaskAssigner.maybeAssign is done, which leaves possibility of inconsistency and late failure.
Diffs
-----
NEWS acaff9eb2ab184b0ef750f8b8a00c20131997f6b
src/main/java/org/apache/aurora/scheduler/AcceptedOffer.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/ResourceSlot.java 7c3d681c216b78eeecebbe950186e5a79c6fe982
src/main/java/org/apache/aurora/scheduler/Resources.java db422a959ee7b982c2a44323de41ad75d1a40754
src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java 2255dd407cd1810c7df5baf17cfa85f79bfffeb8
src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 8fdadda67478bb3110aa442b7d78493cf9c3edb4
src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 7e8e456e288986eb0ce92a123b294e1e25d8ed18
src/test/java/org/apache/aurora/scheduler/AcceptedOfferTest.java PRE-CREATION
src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java e4ae943303823ac4bfbe999ed22f5999484462d8
src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java 33149ab415292eff04f38b61f2b1d1eac79f347a
src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java a5793bffabf4e5d6195b1b99f2363d241c0cecf9
src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 3cbe9acd75def14ae2e0986914ba621fb164b3e4
Diff: https://reviews.apache.org/r/42126/diff/
Testing
-------
1. Unit tested with old and new tests;
2. vagrant integration tests: I manually separate out the vagrant box's cpu and memory between 'aurora-test' role and '*' and verified that jobs can still be launched (I can post the vagrant change in another follow upon request).
Thanks,
Zhitao Li
Re: Review Request 42126: Accept resource offers from multiple
framework roles.
Posted by Zhitao Li <zh...@gmail.com>.
> On Jan. 14, 2016, 6:14 p.m., Maxim Khutornenko wrote:
> > src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java, line 95
> > <https://reviews.apache.org/r/42126/diff/10/?file=1196938#file1196938line95>
> >
> > This is still not technically correct. If role is not set Mesos will not send role with resources at all. All resources will be with role=Null in that case. I'd just drop (role="*") from help.
Fixed.
- Zhitao
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42126/#review114519
-----------------------------------------------------------
On Jan. 14, 2016, 6:23 p.m., Zhitao Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42126/
> -----------------------------------------------------------
>
> (Updated Jan. 14, 2016, 6:23 p.m.)
>
>
> Review request for Aurora, Maxim Khutornenko, Dmitriy Shirchenko, and Bill Farner.
>
>
> Bugs: AURORA-1109
> https://issues.apache.org/jira/browse/AURORA-1109
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This review is a prototype for introducing multiple role support in Aurora.
> This creates a new class OfferAllocation, which allcoates resources to resources field in TaskInfo and ExecutorInfo from an offer.
>
> Current implementation prefers reserved resources over shared resources ('*' role) if both are present
>
> Several caveats:
> 1. This performs the allocate after scheduling decision in TaskAssigner.maybeAssign is done, which leaves possibility of inconsistency and late failure.
>
>
> Diffs
> -----
>
> NEWS acaff9eb2ab184b0ef750f8b8a00c20131997f6b
> src/main/java/org/apache/aurora/scheduler/AcceptedOffer.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/ResourceSlot.java 7c3d681c216b78eeecebbe950186e5a79c6fe982
> src/main/java/org/apache/aurora/scheduler/Resources.java db422a959ee7b982c2a44323de41ad75d1a40754
> src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java 2255dd407cd1810c7df5baf17cfa85f79bfffeb8
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 8fdadda67478bb3110aa442b7d78493cf9c3edb4
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 7e8e456e288986eb0ce92a123b294e1e25d8ed18
> src/test/java/org/apache/aurora/scheduler/AcceptedOfferTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java e4ae943303823ac4bfbe999ed22f5999484462d8
> src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java 33149ab415292eff04f38b61f2b1d1eac79f347a
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java a5793bffabf4e5d6195b1b99f2363d241c0cecf9
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 3cbe9acd75def14ae2e0986914ba621fb164b3e4
>
> Diff: https://reviews.apache.org/r/42126/diff/
>
>
> Testing
> -------
>
> 1. Unit tested with old and new tests;
> 2. vagrant integration tests: I manually separate out the vagrant box's cpu and memory between 'aurora-test' role and '*' and verified that jobs can still be launched (I can post the vagrant change in another follow upon request).
>
>
> Thanks,
>
> Zhitao Li
>
>
Re: Review Request 42126: Accept resource offers from multiple
framework roles.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42126/#review114519
-----------------------------------------------------------
src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java (line 95)
<https://reviews.apache.org/r/42126/#comment175372>
This is still not technically correct. If role is not set Mesos will not send role with resources at all. All resources will be with role=Null in that case. I'd just drop (role="*") from help.
- Maxim Khutornenko
On Jan. 14, 2016, 4:32 p.m., Zhitao Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42126/
> -----------------------------------------------------------
>
> (Updated Jan. 14, 2016, 4:32 p.m.)
>
>
> Review request for Aurora, Maxim Khutornenko, Dmitriy Shirchenko, and Bill Farner.
>
>
> Bugs: AURORA-1109
> https://issues.apache.org/jira/browse/AURORA-1109
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This review is a prototype for introducing multiple role support in Aurora.
> This creates a new class OfferAllocation, which allcoates resources to resources field in TaskInfo and ExecutorInfo from an offer.
>
> Current implementation prefers reserved resources over shared resources ('*' role) if both are present
>
> Several caveats:
> 1. This performs the allocate after scheduling decision in TaskAssigner.maybeAssign is done, which leaves possibility of inconsistency and late failure.
>
>
> Diffs
> -----
>
> NEWS acaff9eb2ab184b0ef750f8b8a00c20131997f6b
> src/main/java/org/apache/aurora/scheduler/AcceptedOffer.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/ResourceSlot.java 7c3d681c216b78eeecebbe950186e5a79c6fe982
> src/main/java/org/apache/aurora/scheduler/Resources.java db422a959ee7b982c2a44323de41ad75d1a40754
> src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java 2255dd407cd1810c7df5baf17cfa85f79bfffeb8
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 8fdadda67478bb3110aa442b7d78493cf9c3edb4
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 7e8e456e288986eb0ce92a123b294e1e25d8ed18
> src/test/java/org/apache/aurora/scheduler/AcceptedOfferTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java e4ae943303823ac4bfbe999ed22f5999484462d8
> src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java 33149ab415292eff04f38b61f2b1d1eac79f347a
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java a5793bffabf4e5d6195b1b99f2363d241c0cecf9
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 3cbe9acd75def14ae2e0986914ba621fb164b3e4
>
> Diff: https://reviews.apache.org/r/42126/diff/
>
>
> Testing
> -------
>
> 1. Unit tested with old and new tests;
> 2. vagrant integration tests: I manually separate out the vagrant box's cpu and memory between 'aurora-test' role and '*' and verified that jobs can still be launched (I can post the vagrant change in another follow upon request).
>
>
> Thanks,
>
> Zhitao Li
>
>
Re: Review Request 42126: Accept resource offers from multiple
framework roles.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42126/#review114535
-----------------------------------------------------------
Ship it!
Master (952ef6d) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Jan. 14, 2016, 6:23 p.m., Zhitao Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42126/
> -----------------------------------------------------------
>
> (Updated Jan. 14, 2016, 6:23 p.m.)
>
>
> Review request for Aurora, Maxim Khutornenko, Dmitriy Shirchenko, and Bill Farner.
>
>
> Bugs: AURORA-1109
> https://issues.apache.org/jira/browse/AURORA-1109
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This review is a prototype for introducing multiple role support in Aurora.
> This creates a new class OfferAllocation, which allcoates resources to resources field in TaskInfo and ExecutorInfo from an offer.
>
> Current implementation prefers reserved resources over shared resources ('*' role) if both are present
>
> Several caveats:
> 1. This performs the allocate after scheduling decision in TaskAssigner.maybeAssign is done, which leaves possibility of inconsistency and late failure.
>
>
> Diffs
> -----
>
> NEWS acaff9eb2ab184b0ef750f8b8a00c20131997f6b
> src/main/java/org/apache/aurora/scheduler/AcceptedOffer.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/ResourceSlot.java 7c3d681c216b78eeecebbe950186e5a79c6fe982
> src/main/java/org/apache/aurora/scheduler/Resources.java db422a959ee7b982c2a44323de41ad75d1a40754
> src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java 2255dd407cd1810c7df5baf17cfa85f79bfffeb8
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 8fdadda67478bb3110aa442b7d78493cf9c3edb4
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 7e8e456e288986eb0ce92a123b294e1e25d8ed18
> src/test/java/org/apache/aurora/scheduler/AcceptedOfferTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java e4ae943303823ac4bfbe999ed22f5999484462d8
> src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java 33149ab415292eff04f38b61f2b1d1eac79f347a
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java a5793bffabf4e5d6195b1b99f2363d241c0cecf9
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 3cbe9acd75def14ae2e0986914ba621fb164b3e4
>
> Diff: https://reviews.apache.org/r/42126/diff/
>
>
> Testing
> -------
>
> 1. Unit tested with old and new tests;
> 2. vagrant integration tests: I manually separate out the vagrant box's cpu and memory between 'aurora-test' role and '*' and verified that jobs can still be launched (I can post the vagrant change in another follow upon request).
>
>
> Thanks,
>
> Zhitao Li
>
>
Re: Review Request 42126: Accept resource offers from multiple
framework roles.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On Jan. 14, 2016, 6:31 p.m., Maxim Khutornenko wrote:
> > Ship It!
Waiting for Dmitriy to sign off before committing it to master.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42126/#review114529
-----------------------------------------------------------
On Jan. 14, 2016, 6:23 p.m., Zhitao Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42126/
> -----------------------------------------------------------
>
> (Updated Jan. 14, 2016, 6:23 p.m.)
>
>
> Review request for Aurora, Maxim Khutornenko, Dmitriy Shirchenko, and Bill Farner.
>
>
> Bugs: AURORA-1109
> https://issues.apache.org/jira/browse/AURORA-1109
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This review is a prototype for introducing multiple role support in Aurora.
> This creates a new class OfferAllocation, which allcoates resources to resources field in TaskInfo and ExecutorInfo from an offer.
>
> Current implementation prefers reserved resources over shared resources ('*' role) if both are present
>
> Several caveats:
> 1. This performs the allocate after scheduling decision in TaskAssigner.maybeAssign is done, which leaves possibility of inconsistency and late failure.
>
>
> Diffs
> -----
>
> NEWS acaff9eb2ab184b0ef750f8b8a00c20131997f6b
> src/main/java/org/apache/aurora/scheduler/AcceptedOffer.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/ResourceSlot.java 7c3d681c216b78eeecebbe950186e5a79c6fe982
> src/main/java/org/apache/aurora/scheduler/Resources.java db422a959ee7b982c2a44323de41ad75d1a40754
> src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java 2255dd407cd1810c7df5baf17cfa85f79bfffeb8
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 8fdadda67478bb3110aa442b7d78493cf9c3edb4
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 7e8e456e288986eb0ce92a123b294e1e25d8ed18
> src/test/java/org/apache/aurora/scheduler/AcceptedOfferTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java e4ae943303823ac4bfbe999ed22f5999484462d8
> src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java 33149ab415292eff04f38b61f2b1d1eac79f347a
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java a5793bffabf4e5d6195b1b99f2363d241c0cecf9
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 3cbe9acd75def14ae2e0986914ba621fb164b3e4
>
> Diff: https://reviews.apache.org/r/42126/diff/
>
>
> Testing
> -------
>
> 1. Unit tested with old and new tests;
> 2. vagrant integration tests: I manually separate out the vagrant box's cpu and memory between 'aurora-test' role and '*' and verified that jobs can still be launched (I can post the vagrant change in another follow upon request).
>
>
> Thanks,
>
> Zhitao Li
>
>
Re: Review Request 42126: Accept resource offers from multiple
framework roles.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42126/#review114529
-----------------------------------------------------------
Ship it!
Ship It!
- Maxim Khutornenko
On Jan. 14, 2016, 6:23 p.m., Zhitao Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42126/
> -----------------------------------------------------------
>
> (Updated Jan. 14, 2016, 6:23 p.m.)
>
>
> Review request for Aurora, Maxim Khutornenko, Dmitriy Shirchenko, and Bill Farner.
>
>
> Bugs: AURORA-1109
> https://issues.apache.org/jira/browse/AURORA-1109
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This review is a prototype for introducing multiple role support in Aurora.
> This creates a new class OfferAllocation, which allcoates resources to resources field in TaskInfo and ExecutorInfo from an offer.
>
> Current implementation prefers reserved resources over shared resources ('*' role) if both are present
>
> Several caveats:
> 1. This performs the allocate after scheduling decision in TaskAssigner.maybeAssign is done, which leaves possibility of inconsistency and late failure.
>
>
> Diffs
> -----
>
> NEWS acaff9eb2ab184b0ef750f8b8a00c20131997f6b
> src/main/java/org/apache/aurora/scheduler/AcceptedOffer.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/ResourceSlot.java 7c3d681c216b78eeecebbe950186e5a79c6fe982
> src/main/java/org/apache/aurora/scheduler/Resources.java db422a959ee7b982c2a44323de41ad75d1a40754
> src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java 2255dd407cd1810c7df5baf17cfa85f79bfffeb8
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 8fdadda67478bb3110aa442b7d78493cf9c3edb4
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 7e8e456e288986eb0ce92a123b294e1e25d8ed18
> src/test/java/org/apache/aurora/scheduler/AcceptedOfferTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java e4ae943303823ac4bfbe999ed22f5999484462d8
> src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java 33149ab415292eff04f38b61f2b1d1eac79f347a
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java a5793bffabf4e5d6195b1b99f2363d241c0cecf9
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 3cbe9acd75def14ae2e0986914ba621fb164b3e4
>
> Diff: https://reviews.apache.org/r/42126/diff/
>
>
> Testing
> -------
>
> 1. Unit tested with old and new tests;
> 2. vagrant integration tests: I manually separate out the vagrant box's cpu and memory between 'aurora-test' role and '*' and verified that jobs can still be launched (I can post the vagrant change in another follow upon request).
>
>
> Thanks,
>
> Zhitao Li
>
>
Re: Review Request 42126: Accept resource offers from multiple
framework roles.
Posted by Dmitriy Shirchenko <ca...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42126/#review114532
-----------------------------------------------------------
Ship it!
Ship It!
- Dmitriy Shirchenko
On Jan. 14, 2016, 6:23 p.m., Zhitao Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42126/
> -----------------------------------------------------------
>
> (Updated Jan. 14, 2016, 6:23 p.m.)
>
>
> Review request for Aurora, Maxim Khutornenko, Dmitriy Shirchenko, and Bill Farner.
>
>
> Bugs: AURORA-1109
> https://issues.apache.org/jira/browse/AURORA-1109
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This review is a prototype for introducing multiple role support in Aurora.
> This creates a new class OfferAllocation, which allcoates resources to resources field in TaskInfo and ExecutorInfo from an offer.
>
> Current implementation prefers reserved resources over shared resources ('*' role) if both are present
>
> Several caveats:
> 1. This performs the allocate after scheduling decision in TaskAssigner.maybeAssign is done, which leaves possibility of inconsistency and late failure.
>
>
> Diffs
> -----
>
> NEWS acaff9eb2ab184b0ef750f8b8a00c20131997f6b
> src/main/java/org/apache/aurora/scheduler/AcceptedOffer.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/ResourceSlot.java 7c3d681c216b78eeecebbe950186e5a79c6fe982
> src/main/java/org/apache/aurora/scheduler/Resources.java db422a959ee7b982c2a44323de41ad75d1a40754
> src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java 2255dd407cd1810c7df5baf17cfa85f79bfffeb8
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 8fdadda67478bb3110aa442b7d78493cf9c3edb4
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 7e8e456e288986eb0ce92a123b294e1e25d8ed18
> src/test/java/org/apache/aurora/scheduler/AcceptedOfferTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java e4ae943303823ac4bfbe999ed22f5999484462d8
> src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java 33149ab415292eff04f38b61f2b1d1eac79f347a
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java a5793bffabf4e5d6195b1b99f2363d241c0cecf9
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 3cbe9acd75def14ae2e0986914ba621fb164b3e4
>
> Diff: https://reviews.apache.org/r/42126/diff/
>
>
> Testing
> -------
>
> 1. Unit tested with old and new tests;
> 2. vagrant integration tests: I manually separate out the vagrant box's cpu and memory between 'aurora-test' role and '*' and verified that jobs can still be launched (I can post the vagrant change in another follow upon request).
>
>
> Thanks,
>
> Zhitao Li
>
>
Re: Review Request 42126: Accept resource offers from multiple
framework roles.
Posted by Zhitao Li <zh...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42126/
-----------------------------------------------------------
(Updated Jan. 14, 2016, 6:23 p.m.)
Review request for Aurora, Maxim Khutornenko, Dmitriy Shirchenko, and Bill Farner.
Changes
-------
Clearer helper message as requested by Maxim
Bugs: AURORA-1109
https://issues.apache.org/jira/browse/AURORA-1109
Repository: aurora
Description
-------
This review is a prototype for introducing multiple role support in Aurora.
This creates a new class OfferAllocation, which allcoates resources to resources field in TaskInfo and ExecutorInfo from an offer.
Current implementation prefers reserved resources over shared resources ('*' role) if both are present
Several caveats:
1. This performs the allocate after scheduling decision in TaskAssigner.maybeAssign is done, which leaves possibility of inconsistency and late failure.
Diffs (updated)
-----
NEWS acaff9eb2ab184b0ef750f8b8a00c20131997f6b
src/main/java/org/apache/aurora/scheduler/AcceptedOffer.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/ResourceSlot.java 7c3d681c216b78eeecebbe950186e5a79c6fe982
src/main/java/org/apache/aurora/scheduler/Resources.java db422a959ee7b982c2a44323de41ad75d1a40754
src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java 2255dd407cd1810c7df5baf17cfa85f79bfffeb8
src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 8fdadda67478bb3110aa442b7d78493cf9c3edb4
src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 7e8e456e288986eb0ce92a123b294e1e25d8ed18
src/test/java/org/apache/aurora/scheduler/AcceptedOfferTest.java PRE-CREATION
src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java e4ae943303823ac4bfbe999ed22f5999484462d8
src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java 33149ab415292eff04f38b61f2b1d1eac79f347a
src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java a5793bffabf4e5d6195b1b99f2363d241c0cecf9
src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 3cbe9acd75def14ae2e0986914ba621fb164b3e4
Diff: https://reviews.apache.org/r/42126/diff/
Testing
-------
1. Unit tested with old and new tests;
2. vagrant integration tests: I manually separate out the vagrant box's cpu and memory between 'aurora-test' role and '*' and verified that jobs can still be launched (I can post the vagrant change in another follow upon request).
Thanks,
Zhitao Li
Re: Review Request 42126: Accept resource offers from multiple
framework roles.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42126/#review114496
-----------------------------------------------------------
Ship it!
Master (952ef6d) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Jan. 14, 2016, 4:32 p.m., Zhitao Li wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42126/
> -----------------------------------------------------------
>
> (Updated Jan. 14, 2016, 4:32 p.m.)
>
>
> Review request for Aurora, Maxim Khutornenko, Dmitriy Shirchenko, and Bill Farner.
>
>
> Bugs: AURORA-1109
> https://issues.apache.org/jira/browse/AURORA-1109
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This review is a prototype for introducing multiple role support in Aurora.
> This creates a new class OfferAllocation, which allcoates resources to resources field in TaskInfo and ExecutorInfo from an offer.
>
> Current implementation prefers reserved resources over shared resources ('*' role) if both are present
>
> Several caveats:
> 1. This performs the allocate after scheduling decision in TaskAssigner.maybeAssign is done, which leaves possibility of inconsistency and late failure.
>
>
> Diffs
> -----
>
> NEWS acaff9eb2ab184b0ef750f8b8a00c20131997f6b
> src/main/java/org/apache/aurora/scheduler/AcceptedOffer.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/ResourceSlot.java 7c3d681c216b78eeecebbe950186e5a79c6fe982
> src/main/java/org/apache/aurora/scheduler/Resources.java db422a959ee7b982c2a44323de41ad75d1a40754
> src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java 2255dd407cd1810c7df5baf17cfa85f79bfffeb8
> src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 8fdadda67478bb3110aa442b7d78493cf9c3edb4
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 7e8e456e288986eb0ce92a123b294e1e25d8ed18
> src/test/java/org/apache/aurora/scheduler/AcceptedOfferTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java e4ae943303823ac4bfbe999ed22f5999484462d8
> src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java 33149ab415292eff04f38b61f2b1d1eac79f347a
> src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java a5793bffabf4e5d6195b1b99f2363d241c0cecf9
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 3cbe9acd75def14ae2e0986914ba621fb164b3e4
>
> Diff: https://reviews.apache.org/r/42126/diff/
>
>
> Testing
> -------
>
> 1. Unit tested with old and new tests;
> 2. vagrant integration tests: I manually separate out the vagrant box's cpu and memory between 'aurora-test' role and '*' and verified that jobs can still be launched (I can post the vagrant change in another follow upon request).
>
>
> Thanks,
>
> Zhitao Li
>
>
Re: Review Request 42126: Accept resource offers from multiple
framework roles.
Posted by Zhitao Li <zh...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42126/
-----------------------------------------------------------
(Updated Jan. 14, 2016, 4:32 p.m.)
Review request for Aurora, Maxim Khutornenko, Dmitriy Shirchenko, and Bill Farner.
Changes
-------
Use a `nearZero()` function to avoid comparison floating point number with zero, and restore a test case using `SOME_OVERHEAD_EXECUTOR`
Bugs: AURORA-1109
https://issues.apache.org/jira/browse/AURORA-1109
Repository: aurora
Description
-------
This review is a prototype for introducing multiple role support in Aurora.
This creates a new class OfferAllocation, which allcoates resources to resources field in TaskInfo and ExecutorInfo from an offer.
Current implementation prefers reserved resources over shared resources ('*' role) if both are present
Several caveats:
1. This performs the allocate after scheduling decision in TaskAssigner.maybeAssign is done, which leaves possibility of inconsistency and late failure.
Diffs (updated)
-----
NEWS acaff9eb2ab184b0ef750f8b8a00c20131997f6b
src/main/java/org/apache/aurora/scheduler/AcceptedOffer.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/ResourceSlot.java 7c3d681c216b78eeecebbe950186e5a79c6fe982
src/main/java/org/apache/aurora/scheduler/Resources.java db422a959ee7b982c2a44323de41ad75d1a40754
src/main/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModule.java 2255dd407cd1810c7df5baf17cfa85f79bfffeb8
src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java 8fdadda67478bb3110aa442b7d78493cf9c3edb4
src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 7e8e456e288986eb0ce92a123b294e1e25d8ed18
src/test/java/org/apache/aurora/scheduler/AcceptedOfferTest.java PRE-CREATION
src/test/java/org/apache/aurora/scheduler/ResourceSlotTest.java e4ae943303823ac4bfbe999ed22f5999484462d8
src/test/java/org/apache/aurora/scheduler/mesos/CommandLineDriverSettingsModuleTest.java 33149ab415292eff04f38b61f2b1d1eac79f347a
src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java a5793bffabf4e5d6195b1b99f2363d241c0cecf9
src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 3cbe9acd75def14ae2e0986914ba621fb164b3e4
Diff: https://reviews.apache.org/r/42126/diff/
Testing
-------
1. Unit tested with old and new tests;
2. vagrant integration tests: I manually separate out the vagrant box's cpu and memory between 'aurora-test' role and '*' and verified that jobs can still be launched (I can post the vagrant change in another follow upon request).
Thanks,
Zhitao Li