You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by Maxim Khutornenko <ma...@apache.org> on 2015/08/01 01:03:31 UTC
Review Request 37001: Centralizing offer/task matching in
TaskAssigner.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37001/
-----------------------------------------------------------
Review request for Aurora, Bill Farner and Zameer Manji.
Bugs: AURORA-1416
https://issues.apache.org/jira/browse/AURORA-1416
Repository: aurora
Description
-------
This releaves `OfferManager` of task matching duties and encapsulates that responsibility entirely within `TaskAssigner`.
Apologies for the large diff as there is no easy way to carve out multiple commits here. The bulk of yellow/green is in test though.
Summary of changes:
- OfferManager becomes a purely API-based solution managing and exposing offers
- TaskAssigner inherits a simplified version of OfferManager.launchFirst() and is now fully responsible for requesting a task launch
- AssignmentResult is gone along with Function<HostOffer, Assignment>
- TaskScheduler does not need OfferManager and StateManager anymore
- TaskSchedulerTest is gone as the coverage it provides has been split between OfferManagerImplTest and TaskSchedulerImplTest. TaskGroups coverage was added in https://reviews.apache.org/r/36994
Diffs
-----
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 5bc73d58c5914e0ece67f2c3845c62c50ea55222
src/jmh/java/org/apache/aurora/benchmark/fakes/FakeOfferManager.java f41330129b221008012fe2bd3632621badb2e125
src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java 14bf265a9260548fc0a26696970fa8a84ebf687f
src/main/java/org/apache/aurora/scheduler/scheduling/SchedulingModule.java c7a1a46b52cfb60d595cd31d84390e85c657cf32
src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java d4bd5299ddbdf108e2520b2ab8a7517cb4c6fff9
src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 3acb45a9ffd88ccff0817acde602e133bf56ac4e
src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 04be32e8bfe431f3b6a158753936e796b0cc906b
src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java a2e2d4c918ad946f8bec256cb1be5e1ba8e2c40f
src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerTest.java 9a91e63e5b7e72848ec7cac1dfadc4b6e9431216
src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java f98818fe8649e5aa0104de197bc8573feeecd37c
Diff: https://reviews.apache.org/r/37001/diff/
Testing
-------
./gradlew -Pq build
./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
Benchmark testing - no noticable difference in general apart from the limit constraint benchmark (the only dynamic group veto), which benefited from reduced complexity (no isStaticallyBanned() call on every match iteration).
Before:
```
Benchmark (numPendingTasks) Mode Cnt Score Error Units
SchedulingBenchmarks.ClusterFullUtilizationBenchmark.runBenchmark N/A thrpt 5 201330.070 ± 13581.745 ops/s
SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark N/A thrpt 5 28745.739 ± 3323.054 ops/s
SchedulingBenchmarks.LimitConstraintMismatchSchedulingBenchmark.runBenchmark N/A thrpt 5 2789.896 ± 447.990 ops/s
SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1 thrpt 5 53.487 ± 4.865 ops/s
SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 10 thrpt 5 54.061 ± 3.724 ops/s
SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 100 thrpt 5 52.667 ± 5.017 ops/s
SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1000 thrpt 5 43.241 ± 1.809 ops/s
SchedulingBenchmarks.ValueConstraintMismatchSchedulingBenchmark.runBenchmark N/A thrpt 5 20517.686 ± 1312.178 ops/s
```
After:
```
Benchmark (numPendingTasks) Mode Cnt Score Error Units
SchedulingBenchmarks.ClusterFullUtilizationBenchmark.runBenchmark N/A thrpt 5 200476.140 ± 16369.588 ops/s
SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark N/A thrpt 5 26137.553 ± 2704.818 ops/s
SchedulingBenchmarks.LimitConstraintMismatchSchedulingBenchmark.runBenchmark N/A thrpt 5 14111.832 ± 79.047 ops/s
SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1 thrpt 5 56.197 ± 2.194 ops/s
SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 10 thrpt 5 52.383 ± 2.795 ops/s
SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 100 thrpt 5 51.301 ± 3.219 ops/s
SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1000 thrpt 5 43.742 ± 2.847 ops/s
SchedulingBenchmarks.ValueConstraintMismatchSchedulingBenchmark.runBenchmark N/A thrpt 5 20417.531 ± 1305.747 ops/s
```
Thanks,
Maxim Khutornenko
Re: Review Request 37001: Centralizing offer/task matching in
TaskAssigner.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37001/#review93816
-----------------------------------------------------------
Ship it!
Master (a45a8ee) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On July 31, 2015, 11:03 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37001/
> -----------------------------------------------------------
>
> (Updated July 31, 2015, 11:03 p.m.)
>
>
> Review request for Aurora, Bill Farner and Zameer Manji.
>
>
> Bugs: AURORA-1416
> https://issues.apache.org/jira/browse/AURORA-1416
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This releaves `OfferManager` of task matching duties and encapsulates that responsibility entirely within `TaskAssigner`.
>
> Apologies for the large diff as there is no easy way to carve out multiple commits here. The bulk of yellow/green is in test though.
>
> Summary of changes:
> - OfferManager becomes a purely API-based solution managing and exposing offers
> - TaskAssigner inherits a simplified version of OfferManager.launchFirst() and is now fully responsible for requesting a task launch
> - AssignmentResult is gone along with Function<HostOffer, Assignment>
> - TaskScheduler does not need OfferManager and StateManager anymore
> - TaskSchedulerTest is gone as the coverage it provides has been split between OfferManagerImplTest and TaskSchedulerImplTest. TaskGroups coverage was added in https://reviews.apache.org/r/36994
>
>
> Diffs
> -----
>
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 5bc73d58c5914e0ece67f2c3845c62c50ea55222
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeOfferManager.java f41330129b221008012fe2bd3632621badb2e125
> src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java 14bf265a9260548fc0a26696970fa8a84ebf687f
> src/main/java/org/apache/aurora/scheduler/scheduling/SchedulingModule.java c7a1a46b52cfb60d595cd31d84390e85c657cf32
> src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java d4bd5299ddbdf108e2520b2ab8a7517cb4c6fff9
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 3acb45a9ffd88ccff0817acde602e133bf56ac4e
> src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 04be32e8bfe431f3b6a158753936e796b0cc906b
> src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java a2e2d4c918ad946f8bec256cb1be5e1ba8e2c40f
> src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerTest.java 9a91e63e5b7e72848ec7cac1dfadc4b6e9431216
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java f98818fe8649e5aa0104de197bc8573feeecd37c
>
> Diff: https://reviews.apache.org/r/37001/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
> Benchmark testing - no noticable difference in general apart from the limit constraint benchmark (the only dynamic group veto), which benefited from reduced complexity (no isStaticallyBanned() call on every match iteration).
>
> Before:
> ```
> Benchmark (numPendingTasks) Mode Cnt Score Error Units
> SchedulingBenchmarks.ClusterFullUtilizationBenchmark.runBenchmark N/A thrpt 5 201330.070 ± 13581.745 ops/s
> SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark N/A thrpt 5 28745.739 ± 3323.054 ops/s
> SchedulingBenchmarks.LimitConstraintMismatchSchedulingBenchmark.runBenchmark N/A thrpt 5 2789.896 ± 447.990 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1 thrpt 5 53.487 ± 4.865 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 10 thrpt 5 54.061 ± 3.724 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 100 thrpt 5 52.667 ± 5.017 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1000 thrpt 5 43.241 ± 1.809 ops/s
> SchedulingBenchmarks.ValueConstraintMismatchSchedulingBenchmark.runBenchmark N/A thrpt 5 20517.686 ± 1312.178 ops/s
> ```
>
> After:
> ```
> Benchmark (numPendingTasks) Mode Cnt Score Error Units
> SchedulingBenchmarks.ClusterFullUtilizationBenchmark.runBenchmark N/A thrpt 5 200476.140 ± 16369.588 ops/s
> SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark N/A thrpt 5 26137.553 ± 2704.818 ops/s
> SchedulingBenchmarks.LimitConstraintMismatchSchedulingBenchmark.runBenchmark N/A thrpt 5 14111.832 ± 79.047 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1 thrpt 5 56.197 ± 2.194 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 10 thrpt 5 52.383 ± 2.795 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 100 thrpt 5 51.301 ± 3.219 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1000 thrpt 5 43.742 ± 2.847 ops/s
> SchedulingBenchmarks.ValueConstraintMismatchSchedulingBenchmark.runBenchmark N/A thrpt 5 20417.531 ± 1305.747 ops/s
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 37001: Centralizing offer/task matching in
TaskAssigner.
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37001/#review93817
-----------------------------------------------------------
Ship it!
- Bill Farner
On July 31, 2015, 11:03 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37001/
> -----------------------------------------------------------
>
> (Updated July 31, 2015, 11:03 p.m.)
>
>
> Review request for Aurora, Bill Farner and Zameer Manji.
>
>
> Bugs: AURORA-1416
> https://issues.apache.org/jira/browse/AURORA-1416
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This releaves `OfferManager` of task matching duties and encapsulates that responsibility entirely within `TaskAssigner`.
>
> Apologies for the large diff as there is no easy way to carve out multiple commits here. The bulk of yellow/green is in test though.
>
> Summary of changes:
> - OfferManager becomes a purely API-based solution managing and exposing offers
> - TaskAssigner inherits a simplified version of OfferManager.launchFirst() and is now fully responsible for requesting a task launch
> - AssignmentResult is gone along with Function<HostOffer, Assignment>
> - TaskScheduler does not need OfferManager and StateManager anymore
> - TaskSchedulerTest is gone as the coverage it provides has been split between OfferManagerImplTest and TaskSchedulerImplTest. TaskGroups coverage was added in https://reviews.apache.org/r/36994
>
>
> Diffs
> -----
>
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 5bc73d58c5914e0ece67f2c3845c62c50ea55222
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeOfferManager.java f41330129b221008012fe2bd3632621badb2e125
> src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java 14bf265a9260548fc0a26696970fa8a84ebf687f
> src/main/java/org/apache/aurora/scheduler/scheduling/SchedulingModule.java c7a1a46b52cfb60d595cd31d84390e85c657cf32
> src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java d4bd5299ddbdf108e2520b2ab8a7517cb4c6fff9
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 3acb45a9ffd88ccff0817acde602e133bf56ac4e
> src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 04be32e8bfe431f3b6a158753936e796b0cc906b
> src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java a2e2d4c918ad946f8bec256cb1be5e1ba8e2c40f
> src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerTest.java 9a91e63e5b7e72848ec7cac1dfadc4b6e9431216
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java f98818fe8649e5aa0104de197bc8573feeecd37c
>
> Diff: https://reviews.apache.org/r/37001/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
> Benchmark testing - no noticable difference in general apart from the limit constraint benchmark (the only dynamic group veto), which benefited from reduced complexity (no isStaticallyBanned() call on every match iteration).
>
> Before:
> ```
> Benchmark (numPendingTasks) Mode Cnt Score Error Units
> SchedulingBenchmarks.ClusterFullUtilizationBenchmark.runBenchmark N/A thrpt 5 201330.070 ± 13581.745 ops/s
> SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark N/A thrpt 5 28745.739 ± 3323.054 ops/s
> SchedulingBenchmarks.LimitConstraintMismatchSchedulingBenchmark.runBenchmark N/A thrpt 5 2789.896 ± 447.990 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1 thrpt 5 53.487 ± 4.865 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 10 thrpt 5 54.061 ± 3.724 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 100 thrpt 5 52.667 ± 5.017 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1000 thrpt 5 43.241 ± 1.809 ops/s
> SchedulingBenchmarks.ValueConstraintMismatchSchedulingBenchmark.runBenchmark N/A thrpt 5 20517.686 ± 1312.178 ops/s
> ```
>
> After:
> ```
> Benchmark (numPendingTasks) Mode Cnt Score Error Units
> SchedulingBenchmarks.ClusterFullUtilizationBenchmark.runBenchmark N/A thrpt 5 200476.140 ± 16369.588 ops/s
> SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark N/A thrpt 5 26137.553 ± 2704.818 ops/s
> SchedulingBenchmarks.LimitConstraintMismatchSchedulingBenchmark.runBenchmark N/A thrpt 5 14111.832 ± 79.047 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1 thrpt 5 56.197 ± 2.194 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 10 thrpt 5 52.383 ± 2.795 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 100 thrpt 5 51.301 ± 3.219 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1000 thrpt 5 43.742 ± 2.847 ops/s
> SchedulingBenchmarks.ValueConstraintMismatchSchedulingBenchmark.runBenchmark N/A thrpt 5 20417.531 ± 1305.747 ops/s
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 37001: Centralizing offer/task matching in
TaskAssigner.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On Aug. 3, 2015, 5:38 p.m., Zameer Manji wrote:
> > src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java, line 111
> > <https://reviews.apache.org/r/37001/diff/1/?file=1026645#file1026645line111>
> >
> > nit: Was it needed to flip the BiCache type here? If not, it would be nice not to do it to minimize the diff.
Swapping key/value here allows getting rid of 2 `getByValue()` calls that return set-based results. This simplifies expectations and result handling quite a bit.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37001/#review93922
-----------------------------------------------------------
On July 31, 2015, 11:03 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37001/
> -----------------------------------------------------------
>
> (Updated July 31, 2015, 11:03 p.m.)
>
>
> Review request for Aurora, Bill Farner and Zameer Manji.
>
>
> Bugs: AURORA-1416
> https://issues.apache.org/jira/browse/AURORA-1416
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This releaves `OfferManager` of task matching duties and encapsulates that responsibility entirely within `TaskAssigner`.
>
> Apologies for the large diff as there is no easy way to carve out multiple commits here. The bulk of yellow/green is in test though.
>
> Summary of changes:
> - OfferManager becomes a purely API-based solution managing and exposing offers
> - TaskAssigner inherits a simplified version of OfferManager.launchFirst() and is now fully responsible for requesting a task launch
> - AssignmentResult is gone along with Function<HostOffer, Assignment>
> - TaskScheduler does not need OfferManager and StateManager anymore
> - TaskSchedulerTest is gone as the coverage it provides has been split between OfferManagerImplTest and TaskSchedulerImplTest. TaskGroups coverage was added in https://reviews.apache.org/r/36994
>
>
> Diffs
> -----
>
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 5bc73d58c5914e0ece67f2c3845c62c50ea55222
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeOfferManager.java f41330129b221008012fe2bd3632621badb2e125
> src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java 14bf265a9260548fc0a26696970fa8a84ebf687f
> src/main/java/org/apache/aurora/scheduler/scheduling/SchedulingModule.java c7a1a46b52cfb60d595cd31d84390e85c657cf32
> src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java d4bd5299ddbdf108e2520b2ab8a7517cb4c6fff9
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 3acb45a9ffd88ccff0817acde602e133bf56ac4e
> src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 04be32e8bfe431f3b6a158753936e796b0cc906b
> src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java a2e2d4c918ad946f8bec256cb1be5e1ba8e2c40f
> src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerTest.java 9a91e63e5b7e72848ec7cac1dfadc4b6e9431216
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java f98818fe8649e5aa0104de197bc8573feeecd37c
>
> Diff: https://reviews.apache.org/r/37001/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
> Benchmark testing - no noticable difference in general apart from the limit constraint benchmark (the only dynamic group veto), which benefited from reduced complexity (no isStaticallyBanned() call on every match iteration).
>
> Before:
> ```
> Benchmark (numPendingTasks) Mode Cnt Score Error Units
> SchedulingBenchmarks.ClusterFullUtilizationBenchmark.runBenchmark N/A thrpt 5 201330.070 ± 13581.745 ops/s
> SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark N/A thrpt 5 28745.739 ± 3323.054 ops/s
> SchedulingBenchmarks.LimitConstraintMismatchSchedulingBenchmark.runBenchmark N/A thrpt 5 2789.896 ± 447.990 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1 thrpt 5 53.487 ± 4.865 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 10 thrpt 5 54.061 ± 3.724 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 100 thrpt 5 52.667 ± 5.017 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1000 thrpt 5 43.241 ± 1.809 ops/s
> SchedulingBenchmarks.ValueConstraintMismatchSchedulingBenchmark.runBenchmark N/A thrpt 5 20517.686 ± 1312.178 ops/s
> ```
>
> After:
> ```
> Benchmark (numPendingTasks) Mode Cnt Score Error Units
> SchedulingBenchmarks.ClusterFullUtilizationBenchmark.runBenchmark N/A thrpt 5 200476.140 ± 16369.588 ops/s
> SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark N/A thrpt 5 26137.553 ± 2704.818 ops/s
> SchedulingBenchmarks.LimitConstraintMismatchSchedulingBenchmark.runBenchmark N/A thrpt 5 14111.832 ± 79.047 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1 thrpt 5 56.197 ± 2.194 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 10 thrpt 5 52.383 ± 2.795 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 100 thrpt 5 51.301 ± 3.219 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1000 thrpt 5 43.742 ± 2.847 ops/s
> SchedulingBenchmarks.ValueConstraintMismatchSchedulingBenchmark.runBenchmark N/A thrpt 5 20417.531 ± 1305.747 ops/s
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 37001: Centralizing offer/task matching in
TaskAssigner.
Posted by Zameer Manji <zm...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37001/#review93922
-----------------------------------------------------------
Ship it!
src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java (line 101)
<https://reviews.apache.org/r/37001/#comment148359>
nit: Was it needed to flip the BiCache type here? If not, it would be nice not to do it to minimize the diff.
- Zameer Manji
On July 31, 2015, 4:03 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37001/
> -----------------------------------------------------------
>
> (Updated July 31, 2015, 4:03 p.m.)
>
>
> Review request for Aurora, Bill Farner and Zameer Manji.
>
>
> Bugs: AURORA-1416
> https://issues.apache.org/jira/browse/AURORA-1416
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This releaves `OfferManager` of task matching duties and encapsulates that responsibility entirely within `TaskAssigner`.
>
> Apologies for the large diff as there is no easy way to carve out multiple commits here. The bulk of yellow/green is in test though.
>
> Summary of changes:
> - OfferManager becomes a purely API-based solution managing and exposing offers
> - TaskAssigner inherits a simplified version of OfferManager.launchFirst() and is now fully responsible for requesting a task launch
> - AssignmentResult is gone along with Function<HostOffer, Assignment>
> - TaskScheduler does not need OfferManager and StateManager anymore
> - TaskSchedulerTest is gone as the coverage it provides has been split between OfferManagerImplTest and TaskSchedulerImplTest. TaskGroups coverage was added in https://reviews.apache.org/r/36994
>
>
> Diffs
> -----
>
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 5bc73d58c5914e0ece67f2c3845c62c50ea55222
> src/jmh/java/org/apache/aurora/benchmark/fakes/FakeOfferManager.java f41330129b221008012fe2bd3632621badb2e125
> src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java 14bf265a9260548fc0a26696970fa8a84ebf687f
> src/main/java/org/apache/aurora/scheduler/scheduling/SchedulingModule.java c7a1a46b52cfb60d595cd31d84390e85c657cf32
> src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java d4bd5299ddbdf108e2520b2ab8a7517cb4c6fff9
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 3acb45a9ffd88ccff0817acde602e133bf56ac4e
> src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 04be32e8bfe431f3b6a158753936e796b0cc906b
> src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java a2e2d4c918ad946f8bec256cb1be5e1ba8e2c40f
> src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerTest.java 9a91e63e5b7e72848ec7cac1dfadc4b6e9431216
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java f98818fe8649e5aa0104de197bc8573feeecd37c
>
> Diff: https://reviews.apache.org/r/37001/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
> Benchmark testing - no noticable difference in general apart from the limit constraint benchmark (the only dynamic group veto), which benefited from reduced complexity (no isStaticallyBanned() call on every match iteration).
>
> Before:
> ```
> Benchmark (numPendingTasks) Mode Cnt Score Error Units
> SchedulingBenchmarks.ClusterFullUtilizationBenchmark.runBenchmark N/A thrpt 5 201330.070 ± 13581.745 ops/s
> SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark N/A thrpt 5 28745.739 ± 3323.054 ops/s
> SchedulingBenchmarks.LimitConstraintMismatchSchedulingBenchmark.runBenchmark N/A thrpt 5 2789.896 ± 447.990 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1 thrpt 5 53.487 ± 4.865 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 10 thrpt 5 54.061 ± 3.724 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 100 thrpt 5 52.667 ± 5.017 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1000 thrpt 5 43.241 ± 1.809 ops/s
> SchedulingBenchmarks.ValueConstraintMismatchSchedulingBenchmark.runBenchmark N/A thrpt 5 20517.686 ± 1312.178 ops/s
> ```
>
> After:
> ```
> Benchmark (numPendingTasks) Mode Cnt Score Error Units
> SchedulingBenchmarks.ClusterFullUtilizationBenchmark.runBenchmark N/A thrpt 5 200476.140 ± 16369.588 ops/s
> SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark N/A thrpt 5 26137.553 ± 2704.818 ops/s
> SchedulingBenchmarks.LimitConstraintMismatchSchedulingBenchmark.runBenchmark N/A thrpt 5 14111.832 ± 79.047 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1 thrpt 5 56.197 ± 2.194 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 10 thrpt 5 52.383 ± 2.795 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 100 thrpt 5 51.301 ± 3.219 ops/s
> SchedulingBenchmarks.PreemptorSlotSearchBenchmark.runBenchmark 1000 thrpt 5 43.742 ± 2.847 ops/s
> SchedulingBenchmarks.ValueConstraintMismatchSchedulingBenchmark.runBenchmark N/A thrpt 5 20417.531 ± 1305.747 ops/s
> ```
>
>
> Thanks,
>
> Maxim Khutornenko
>
>