You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by Maxim Khutornenko <ma...@apache.org> on 2014/12/03 02:15:24 UTC
Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/
-----------------------------------------------------------
Review request for Aurora, Kevin Sweeney and Bill Farner.
Bugs: AURORA-909
https://issues.apache.org/jira/browse/AURORA-909
Repository: aurora
Description
-------
Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
I will follow up with more accurate measurements (https://reviews.apache.org/r/28474/) but preliminary testing in vagrant (see pictures below) shows close to 50% improvement in task scheduling performance.
Diffs
-----
src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java ead9d28100673440168a32d114ecaa15874978a6
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java c2a342ce07bfb223193886038761f0da5230135d
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 1cb56f19c331508a1585077e9c4a98f52aac343b
src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 265c38d20136210e7639ac8ea915d307a4b72949
src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
Diff: https://reviews.apache.org/r/28617/diff/
Testing
-------
./gradlew -Pq build
Tested in vagrant
File Attachments
----------------
NoStaticVetoFiltering.png
https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
StaticVetoFiltering.png
https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
Thanks,
Maxim Khutornenko
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review64111
-----------------------------------------------------------
Ship it!
Master (40d2718) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Dec. 5, 2014, 10:57 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> -----------------------------------------------------------
>
> (Updated Dec. 5, 2014, 10:57 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
>
> Preliminary testing in vagrant (see pictures below) shows close to 50% improvement in task scheduling performance.
>
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and https://reviews.apache.org/r/28731/) shows over 97% better perf:
>
> ```
> Master with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 8291046.074 ± 145251.995 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 7522269.050 ± 142446.265 ns/op
>
> This RB with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 204171.046 ± 3800.124 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 215854.129 ± 8959.851 ns/op
> ```
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
> src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java ead9d28100673440168a32d114ecaa15874978a6
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java c2a342ce07bfb223193886038761f0da5230135d
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 1cb56f19c331508a1585077e9c4a98f52aac343b
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
> src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
> src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 265c38d20136210e7639ac8ea915d307a4b72949
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
>
> Diff: https://reviews.apache.org/r/28617/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
> Tested in vagrant
>
>
> File Attachments
> ----------------
>
> NoStaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On Dec. 8, 2014, 7:29 p.m., Kevin Sweeney wrote:
> > src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java, line 102
> > <https://reviews.apache.org/r/28617/diff/2/?file=784162#file784162line102>
> >
> > Can you add this as a member to the Veto interface instead? This will ensure we don't add new Veto types without updating this map.
> >
> > Like
> >
> > ```java
> > interface Veto {
> > VetoGroup getVetoGroup();
> > }
> > ```
There is no Veto interface and I am hesitant to create one just for this method. I have moved the group assignment into the VetoType instead. That also helped to get rid of a null check code not reachable from unit tests.
> On Dec. 8, 2014, 7:29 p.m., Kevin Sweeney wrote:
> > src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java, line 181
> > <https://reviews.apache.org/r/28617/diff/2/?file=784162#file784162line181>
> >
> > If you follow the comment above this block becomes unnecessary.
Gone.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review64259
-----------------------------------------------------------
On Dec. 5, 2014, 10:57 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> -----------------------------------------------------------
>
> (Updated Dec. 5, 2014, 10:57 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
>
> Preliminary testing in vagrant (see pictures below) shows close to 50% improvement in task scheduling performance.
>
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and https://reviews.apache.org/r/28731/) shows over 97% better perf:
>
> ```
> Master with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 8291046.074 ± 145251.995 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 7522269.050 ± 142446.265 ns/op
>
> This RB with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 204171.046 ± 3800.124 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 215854.129 ± 8959.851 ns/op
> ```
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
> src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java ead9d28100673440168a32d114ecaa15874978a6
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java c2a342ce07bfb223193886038761f0da5230135d
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 1cb56f19c331508a1585077e9c4a98f52aac343b
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
> src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
> src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 265c38d20136210e7639ac8ea915d307a4b72949
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
>
> Diff: https://reviews.apache.org/r/28617/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
> Tested in vagrant
>
>
> File Attachments
> ----------------
>
> NoStaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Kevin Sweeney <ke...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review64259
-----------------------------------------------------------
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java
<https://reviews.apache.org/r/28617/#comment106898>
Can you add this as a member to the Veto interface instead? This will ensure we don't add new Veto types without updating this map.
Like
```java
interface Veto {
VetoGroup getVetoGroup();
}
```
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java
<https://reviews.apache.org/r/28617/#comment106899>
If you follow the comment above this block becomes unnecessary.
- Kevin Sweeney
On Dec. 5, 2014, 2:57 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> -----------------------------------------------------------
>
> (Updated Dec. 5, 2014, 2:57 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
>
> Preliminary testing in vagrant (see pictures below) shows close to 50% improvement in task scheduling performance.
>
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and https://reviews.apache.org/r/28731/) shows over 97% better perf:
>
> ```
> Master with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 8291046.074 ± 145251.995 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 7522269.050 ± 142446.265 ns/op
>
> This RB with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 204171.046 ± 3800.124 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 215854.129 ± 8959.851 ns/op
> ```
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
> src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java ead9d28100673440168a32d114ecaa15874978a6
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java c2a342ce07bfb223193886038761f0da5230135d
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 1cb56f19c331508a1585077e9c4a98f52aac343b
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
> src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
> src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 265c38d20136210e7639ac8ea915d307a4b72949
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
>
> Diff: https://reviews.apache.org/r/28617/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
> Tested in vagrant
>
>
> File Attachments
> ----------------
>
> NoStaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review64314
-----------------------------------------------------------
Ship it!
Master (a9e1217) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Dec. 8, 2014, 10:57 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> -----------------------------------------------------------
>
> (Updated Dec. 8, 2014, 10:57 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
>
> Preliminary testing in vagrant (see pictures below) shows close to 50% improvement in task scheduling performance.
>
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and https://reviews.apache.org/r/28731/) shows over 97% better perf:
>
> ```
> Master with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 8291046.074 ± 145251.995 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 7522269.050 ± 142446.265 ns/op
>
> This RB with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 204171.046 ± 3800.124 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 215854.129 ± 8959.851 ns/op
> ```
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
> src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java ead9d28100673440168a32d114ecaa15874978a6
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java c2a342ce07bfb223193886038761f0da5230135d
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 1cb56f19c331508a1585077e9c4a98f52aac343b
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
> src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
> src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 265c38d20136210e7639ac8ea915d307a4b72949
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
>
> Diff: https://reviews.apache.org/r/28617/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
> Tested in vagrant
>
>
> File Attachments
> ----------------
>
> NoStaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review64951
-----------------------------------------------------------
Ship it!
Master (cbe9c05) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Dec. 12, 2014, 6:05 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> -----------------------------------------------------------
>
> (Updated Dec. 12, 2014, 6:05 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
>
> Preliminary testing in vagrant (see pictures below) shows close to 50% improvement in task scheduling performance.
>
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and https://reviews.apache.org/r/28731/) shows over 97% better perf:
>
> ```
> Master with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 8291046.074 ± 145251.995 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 7522269.050 ± 142446.265 ns/op
>
> This RB with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 204171.046 ± 3800.124 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 215854.129 ± 8959.851 ns/op
> ```
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
> src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java c2a342ce07bfb223193886038761f0da5230135d
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 1cb56f19c331508a1585077e9c4a98f52aac343b
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
> src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
> src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 265c38d20136210e7639ac8ea915d307a4b72949
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
>
> Diff: https://reviews.apache.org/r/28617/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
> Tested in vagrant
>
>
> File Attachments
> ----------------
>
> NoStaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Bill Farner <wf...@apache.org>.
> On Feb. 5, 2015, 12:01 a.m., Bill Farner wrote:
> > Can you see any opportunity to break this diff apart? As it stands i'm having a hard time giving a thoughtful review. Perhaps you can start by introducing the `Assignment` class?
>
> Maxim Khutornenko wrote:
> I'd really prefer keeping this diff as a whole. The Assignment class would not make sense without the entire picture in mind.
It's okay if you commit them back-to-back, but we really need to exercise a pattern for incrementally building features even if they appear as code islands in the commit history. Doing so makes it easier to review with confidence, and more likely that you get faster review turnaround.
- Bill
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review71079
-----------------------------------------------------------
On Feb. 4, 2015, 11:38 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> -----------------------------------------------------------
>
> (Updated Feb. 4, 2015, 11:38 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
>
> Real life testing in vagrant (see pictures) shows close to 50% improvement in task scheduling performance.
>
> Testing with JMH shows over 97% better perf when testing with disabled preemptor (1 scheduling loop):
> ```
> Master
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 8291046.074 ± 145251.995 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 7522269.050 ± 142446.265 ns/op
>
> This RB
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 204171.046 ± 3800.124 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 215854.129 ± 8959.851 ns/op
> ```
>
> Testing with preemptor enabled and no tasks eligible for preemption gives around 40% improvement (2 scheduling loops):
> ```
> Master
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 1767479.299 ± 26907.571 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 1538682.287 ± 119119.911 ns/op
>
> This RB
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 1105731.141 ± 10040.721 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 939230.662 ± 11091.505 ns/op
> ```
>
> Testing with preemptor enabled and running the worst case possible scenario (every slave is eligible and all tasks are victims) yields the least improvement 2-3% (3 scheduling loops).
> ```
> Master
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 11043701.243 ± 40550.259 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 10478631.055 ± 178833.158 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 116258653.000 ± 403080.017 ns/op
>
> This RB
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 10886116.889 ± 193934.324 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 10182572.955 ± 35740.891 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 113656994.000 ± 424163.759 ns/op
> ```
>
>
> Diffs
> -----
>
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 8c11ef8bd6609f3e4d97ca154d922898f8362446
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java 1a35f9ee9e8e76def0f9bf5454cf8cbdf6a89c25
> src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
> src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java ce47ff152e303fd2116bc3b9e91c0c1a8f76f258
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 6a43bcd1719e8aa32fd3fcb7387d0318c3c0b804
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java f06fdaeb92e154d0982bdabed5df93e7bcba9048
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
> src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
> src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 52ee7c1e3742d9315c7e7aaa77677121e1e9288d
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
>
> Diff: https://reviews.apache.org/r/28617/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
> Tested in vagrant
>
>
> File Attachments
> ----------------
>
> NoStaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On Feb. 5, 2015, 12:01 a.m., Bill Farner wrote:
> > Can you see any opportunity to break this diff apart? As it stands i'm having a hard time giving a thoughtful review. Perhaps you can start by introducing the `Assignment` class?
I'd really prefer keeping this diff as a whole. The Assignment class would not make sense without the entire picture in mind.
> On Feb. 5, 2015, 12:01 a.m., Bill Farner wrote:
> > src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java, line 210
> > <https://reviews.apache.org/r/28617/diff/6/?file=849086#file849086line210>
> >
> > What motivates you to supply these as overridden methods rather than a configuration object? The latter seems like better encapsulation.
Nothing in particular. It has grown organically this way. Perhaps it's time to refactor. I would like to address it separately though to avoid growing this diff any further. Will leave a TODO.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review71079
-----------------------------------------------------------
On Feb. 4, 2015, 11:38 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> -----------------------------------------------------------
>
> (Updated Feb. 4, 2015, 11:38 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
>
> Real life testing in vagrant (see pictures) shows close to 50% improvement in task scheduling performance.
>
> Testing with JMH shows over 97% better perf when testing with disabled preemptor (1 scheduling loop):
> ```
> Master
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 8291046.074 ± 145251.995 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 7522269.050 ± 142446.265 ns/op
>
> This RB
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 204171.046 ± 3800.124 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 215854.129 ± 8959.851 ns/op
> ```
>
> Testing with preemptor enabled and no tasks eligible for preemption gives around 40% improvement (2 scheduling loops):
> ```
> Master
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 1767479.299 ± 26907.571 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 1538682.287 ± 119119.911 ns/op
>
> This RB
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 1105731.141 ± 10040.721 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 939230.662 ± 11091.505 ns/op
> ```
>
> Testing with preemptor enabled and running the worst case possible scenario (every slave is eligible and all tasks are victims) yields the least improvement 2-3% (3 scheduling loops).
> ```
> Master
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 11043701.243 ± 40550.259 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 10478631.055 ± 178833.158 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 116258653.000 ± 403080.017 ns/op
>
> This RB
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 10886116.889 ± 193934.324 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 10182572.955 ± 35740.891 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 113656994.000 ± 424163.759 ns/op
> ```
>
>
> Diffs
> -----
>
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 8c11ef8bd6609f3e4d97ca154d922898f8362446
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java 1a35f9ee9e8e76def0f9bf5454cf8cbdf6a89c25
> src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
> src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java ce47ff152e303fd2116bc3b9e91c0c1a8f76f258
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 6a43bcd1719e8aa32fd3fcb7387d0318c3c0b804
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java f06fdaeb92e154d0982bdabed5df93e7bcba9048
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
> src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
> src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 52ee7c1e3742d9315c7e7aaa77677121e1e9288d
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
>
> Diff: https://reviews.apache.org/r/28617/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
> Tested in vagrant
>
>
> File Attachments
> ----------------
>
> NoStaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On Feb. 5, 2015, 12:01 a.m., Bill Farner wrote:
> > Can you see any opportunity to break this diff apart? As it stands i'm having a hard time giving a thoughtful review. Perhaps you can start by introducing the `Assignment` class?
>
> Maxim Khutornenko wrote:
> I'd really prefer keeping this diff as a whole. The Assignment class would not make sense without the entire picture in mind.
>
> Bill Farner wrote:
> It's okay if you commit them back-to-back, but we really need to exercise a pattern for incrementally building features even if they appear as code islands in the commit history. Doing so makes it easier to review with confidence, and more likely that you get faster review turnaround.
Splitting this RB into 4 parts for easier reviewing. Keeping it around for history though.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review71079
-----------------------------------------------------------
On Feb. 4, 2015, 11:38 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> -----------------------------------------------------------
>
> (Updated Feb. 4, 2015, 11:38 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
>
> Real life testing in vagrant (see pictures) shows close to 50% improvement in task scheduling performance.
>
> Testing with JMH shows over 97% better perf when testing with disabled preemptor (1 scheduling loop):
> ```
> Master
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 8291046.074 ± 145251.995 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 7522269.050 ± 142446.265 ns/op
>
> This RB
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 204171.046 ± 3800.124 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 215854.129 ± 8959.851 ns/op
> ```
>
> Testing with preemptor enabled and no tasks eligible for preemption gives around 40% improvement (2 scheduling loops):
> ```
> Master
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 1767479.299 ± 26907.571 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 1538682.287 ± 119119.911 ns/op
>
> This RB
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 1105731.141 ± 10040.721 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 939230.662 ± 11091.505 ns/op
> ```
>
> Testing with preemptor enabled and running the worst case possible scenario (every slave is eligible and all tasks are victims) yields the least improvement 2-3% (3 scheduling loops).
> ```
> Master
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 11043701.243 ± 40550.259 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 10478631.055 ± 178833.158 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 116258653.000 ± 403080.017 ns/op
>
> This RB
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 10886116.889 ± 193934.324 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 10182572.955 ± 35740.891 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 113656994.000 ± 424163.759 ns/op
> ```
>
>
> Diffs
> -----
>
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 8c11ef8bd6609f3e4d97ca154d922898f8362446
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java 1a35f9ee9e8e76def0f9bf5454cf8cbdf6a89c25
> src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
> src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java ce47ff152e303fd2116bc3b9e91c0c1a8f76f258
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 6a43bcd1719e8aa32fd3fcb7387d0318c3c0b804
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java f06fdaeb92e154d0982bdabed5df93e7bcba9048
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
> src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
> src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 52ee7c1e3742d9315c7e7aaa77677121e1e9288d
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
>
> Diff: https://reviews.apache.org/r/28617/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
> Tested in vagrant
>
>
> File Attachments
> ----------------
>
> NoStaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review71079
-----------------------------------------------------------
Can you see any opportunity to break this diff apart? As it stands i'm having a hard time giving a thoughtful review. Perhaps you can start by introducing the `Assignment` class?
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java
<https://reviews.apache.org/r/28617/#comment116667>
What motivates you to supply these as overridden methods rather than a configuration object? The latter seems like better encapsulation.
- Bill Farner
On Feb. 4, 2015, 11:38 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> -----------------------------------------------------------
>
> (Updated Feb. 4, 2015, 11:38 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
>
> Real life testing in vagrant (see pictures) shows close to 50% improvement in task scheduling performance.
>
> Testing with JMH shows over 97% better perf when testing with disabled preemptor (1 scheduling loop):
> ```
> Master
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 8291046.074 ± 145251.995 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 7522269.050 ± 142446.265 ns/op
>
> This RB
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 204171.046 ± 3800.124 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 215854.129 ± 8959.851 ns/op
> ```
>
> Testing with preemptor enabled and no tasks eligible for preemption gives around 40% improvement (2 scheduling loops):
> ```
> Master
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 1767479.299 ± 26907.571 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 1538682.287 ± 119119.911 ns/op
>
> This RB
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 1105731.141 ± 10040.721 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 939230.662 ± 11091.505 ns/op
> ```
>
> Testing with preemptor enabled and running the worst case possible scenario (every slave is eligible and all tasks are victims) yields the least improvement 2-3% (3 scheduling loops).
> ```
> Master
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 11043701.243 ± 40550.259 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 10478631.055 ± 178833.158 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 116258653.000 ± 403080.017 ns/op
>
> This RB
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 10886116.889 ± 193934.324 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 10182572.955 ± 35740.891 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 113656994.000 ± 424163.759 ns/op
> ```
>
>
> Diffs
> -----
>
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 8c11ef8bd6609f3e4d97ca154d922898f8362446
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java 1a35f9ee9e8e76def0f9bf5454cf8cbdf6a89c25
> src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
> src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java ce47ff152e303fd2116bc3b9e91c0c1a8f76f258
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 6a43bcd1719e8aa32fd3fcb7387d0318c3c0b804
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java f06fdaeb92e154d0982bdabed5df93e7bcba9048
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
> src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
> src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 52ee7c1e3742d9315c7e7aaa77677121e1e9288d
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
>
> Diff: https://reviews.apache.org/r/28617/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
> Tested in vagrant
>
>
> File Attachments
> ----------------
>
> NoStaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review71084
-----------------------------------------------------------
Master (b49e1a0) is red with this patch.
./build-support/jenkins/build.sh
src.test.python.apache.aurora.client.cli.config ..... SUCCESS
src.test.python.apache.aurora.client.cli.cron ..... SUCCESS
src.test.python.apache.aurora.client.cli.inspect ..... SUCCESS
src.test.python.apache.aurora.client.cli.job ..... SUCCESS
src.test.python.apache.aurora.client.cli.plugins ..... SUCCESS
src.test.python.apache.aurora.client.cli.quota ..... SUCCESS
src.test.python.apache.aurora.client.cli.sla ..... SUCCESS
src.test.python.apache.aurora.client.cli.supdate ..... SUCCESS
src.test.python.apache.aurora.client.cli.task ..... SUCCESS
src.test.python.apache.aurora.client.cli.update ..... SUCCESS
src.test.python.apache.aurora.client.cli.version ..... SUCCESS
src.test.python.apache.aurora.client.config ..... SUCCESS
src.test.python.apache.aurora.client.hooks.hooked_api ..... SUCCESS
src.test.python.apache.aurora.client.hooks.non_hooked_api ..... SUCCESS
src.test.python.apache.aurora.common.test_aurora_job_key ..... SUCCESS
src.test.python.apache.aurora.common.test_cluster ..... SUCCESS
src.test.python.apache.aurora.common.test_cluster_option ..... SUCCESS
src.test.python.apache.aurora.common.test_clusters ..... SUCCESS
src.test.python.apache.aurora.common.test_http_signaler ..... SUCCESS
src.test.python.apache.aurora.common.test_pex_version ..... SUCCESS
src.test.python.apache.aurora.common.test_shellify ..... SUCCESS
src.test.python.apache.aurora.common.test_transport ..... SUCCESS
src.test.python.apache.aurora.config.test_base ..... SUCCESS
src.test.python.apache.aurora.config.test_constraint_parsing ..... SUCCESS
src.test.python.apache.aurora.config.test_loader ..... SUCCESS
src.test.python.apache.aurora.config.test_thrift ..... SUCCESS
src.test.python.apache.aurora.executor.common.task_info ..... SUCCESS
src.test.python.apache.aurora.executor.executor_base ..... SUCCESS
src.test.python.apache.aurora.executor.executor_detector ..... SUCCESS
src.test.python.apache.aurora.executor.executor_vars ..... SUCCESS
src.test.python.apache.aurora.executor.status_manager ..... SUCCESS
src.test.python.apache.aurora.executor.thermos_task_runner ..... FAILURE
src.test.python.apache.thermos.common.test_pathspec ..... SUCCESS
src.test.python.apache.thermos.core.test_runner_integration ..... SUCCESS
src.test.python.apache.thermos.monitoring.test_disk ..... SUCCESS
FAILURE
[31m
FAILURE[0m
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Feb. 4, 2015, 11:38 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> -----------------------------------------------------------
>
> (Updated Feb. 4, 2015, 11:38 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
>
> Real life testing in vagrant (see pictures) shows close to 50% improvement in task scheduling performance.
>
> Testing with JMH shows over 97% better perf when testing with disabled preemptor (1 scheduling loop):
> ```
> Master
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 8291046.074 ± 145251.995 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 7522269.050 ± 142446.265 ns/op
>
> This RB
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 204171.046 ± 3800.124 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 215854.129 ± 8959.851 ns/op
> ```
>
> Testing with preemptor enabled and no tasks eligible for preemption gives around 40% improvement (2 scheduling loops):
> ```
> Master
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 1767479.299 ± 26907.571 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 1538682.287 ± 119119.911 ns/op
>
> This RB
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 1105731.141 ± 10040.721 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 939230.662 ± 11091.505 ns/op
> ```
>
> Testing with preemptor enabled and running the worst case possible scenario (every slave is eligible and all tasks are victims) yields the least improvement 2-3% (3 scheduling loops).
> ```
> Master
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 11043701.243 ± 40550.259 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 10478631.055 ± 178833.158 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 116258653.000 ± 403080.017 ns/op
>
> This RB
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 10886116.889 ± 193934.324 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 10182572.955 ± 35740.891 ns/op
> o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 113656994.000 ± 424163.759 ns/op
> ```
>
>
> Diffs
> -----
>
> src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 8c11ef8bd6609f3e4d97ca154d922898f8362446
> src/jmh/java/org/apache/aurora/benchmark/Tasks.java 1a35f9ee9e8e76def0f9bf5454cf8cbdf6a89c25
> src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
> src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java ce47ff152e303fd2116bc3b9e91c0c1a8f76f258
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 6a43bcd1719e8aa32fd3fcb7387d0318c3c0b804
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java f06fdaeb92e154d0982bdabed5df93e7bcba9048
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
> src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
> src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 52ee7c1e3742d9315c7e7aaa77677121e1e9288d
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
>
> Diff: https://reviews.apache.org/r/28617/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
> Tested in vagrant
>
>
> File Attachments
> ----------------
>
> NoStaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/
-----------------------------------------------------------
(Updated Feb. 4, 2015, 11:38 p.m.)
Review request for Aurora, Kevin Sweeney and Bill Farner.
Changes
-------
Rebased and modified benchmarks to support victim-less preemption run.
Bugs: AURORA-909
https://issues.apache.org/jira/browse/AURORA-909
Repository: aurora
Description (updated)
-------
Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
Real life testing in vagrant (see pictures) shows close to 50% improvement in task scheduling performance.
Testing with JMH shows over 97% better perf when testing with disabled preemptor (1 scheduling loop):
```
Master
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 8291046.074 ± 145251.995 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 7522269.050 ± 142446.265 ns/op
This RB
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 204171.046 ± 3800.124 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 215854.129 ± 8959.851 ns/op
```
Testing with preemptor enabled and no tasks eligible for preemption gives around 40% improvement (2 scheduling loops):
```
Master
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 1767479.299 ± 26907.571 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 1538682.287 ± 119119.911 ns/op
This RB
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 1105731.141 ± 10040.721 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 939230.662 ± 11091.505 ns/op
```
Testing with preemptor enabled and running the worst case possible scenario (every slave is eligible and all tasks are victims) yields the least improvement 2-3% (3 scheduling loops).
```
Master
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 11043701.243 ± 40550.259 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 10478631.055 ± 178833.158 ns/op
o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 116258653.000 ± 403080.017 ns/op
This RB
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 10886116.889 ± 193934.324 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 10182572.955 ± 35740.891 ns/op
o.a.a.b.SchedulingBenchmarks.PreemptorFallbackForLargeClusterBenchmark.runBenchmark avgt 100 113656994.000 ± 424163.759 ns/op
```
Diffs (updated)
-----
src/jmh/java/org/apache/aurora/benchmark/SchedulingBenchmarks.java 8c11ef8bd6609f3e4d97ca154d922898f8362446
src/jmh/java/org/apache/aurora/benchmark/Tasks.java 1a35f9ee9e8e76def0f9bf5454cf8cbdf6a89c25
src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java ce47ff152e303fd2116bc3b9e91c0c1a8f76f258
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 6a43bcd1719e8aa32fd3fcb7387d0318c3c0b804
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java f06fdaeb92e154d0982bdabed5df93e7bcba9048
src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 52ee7c1e3742d9315c7e7aaa77677121e1e9288d
src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
Diff: https://reviews.apache.org/r/28617/diff/
Testing
-------
./gradlew -Pq build
Tested in vagrant
File Attachments
----------------
NoStaticVetoFiltering.png
https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
StaticVetoFiltering.png
https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
Thanks,
Maxim Khutornenko
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review66964
-----------------------------------------------------------
Ship it!
Master (8c49029) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Jan. 7, 2015, 12:26 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> -----------------------------------------------------------
>
> (Updated Jan. 7, 2015, 12:26 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
>
> Preliminary testing in vagrant (see pictures below) shows close to 50% improvement in task scheduling performance.
>
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and https://reviews.apache.org/r/28731/) shows over 97% better perf when testing with disabled preemptor:
> ```
> Master with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 8291046.074 ± 145251.995 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 7522269.050 ± 142446.265 ns/op
>
> This RB with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 204171.046 ± 3800.124 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 215854.129 ± 8959.851 ns/op
> ```
>
> Testing with preemptor enabled and running the worst case possible scenario (every slave is eligible and has task victims to evaluate) is still 16-22% faster than the master.
> ```
> Master with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7170244.522 ± 230259.848 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 8158184.909 ± 465853.379 ns/op
>
> This RB with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 6050901.381 ± 182134.371 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6425953.419 ± 163741.064 ns/op
> ```
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
> src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java c2a342ce07bfb223193886038761f0da5230135d
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 1cb56f19c331508a1585077e9c4a98f52aac343b
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
> src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
> src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 265c38d20136210e7639ac8ea915d307a4b72949
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
>
> Diff: https://reviews.apache.org/r/28617/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
> Tested in vagrant
>
>
> File Attachments
> ----------------
>
> NoStaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/
-----------------------------------------------------------
(Updated Jan. 7, 2015, 12:26 a.m.)
Review request for Aurora, Kevin Sweeney and Bill Farner.
Changes
-------
Kevin's comments.
Bugs: AURORA-909
https://issues.apache.org/jira/browse/AURORA-909
Repository: aurora
Description
-------
Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
Preliminary testing in vagrant (see pictures below) shows close to 50% improvement in task scheduling performance.
Update:
Testing with JMH (https://reviews.apache.org/r/28710/ and https://reviews.apache.org/r/28731/) shows over 97% better perf when testing with disabled preemptor:
```
Master with cluster fillup 0.9:
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 8291046.074 ± 145251.995 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 7522269.050 ± 142446.265 ns/op
This RB with cluster fillup 0.9:
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 204171.046 ± 3800.124 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 215854.129 ± 8959.851 ns/op
```
Testing with preemptor enabled and running the worst case possible scenario (every slave is eligible and has task victims to evaluate) is still 16-22% faster than the master.
```
Master with cluster fillup 0.9:
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7170244.522 ± 230259.848 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 8158184.909 ± 465853.379 ns/op
This RB with cluster fillup 0.9:
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 6050901.381 ± 182134.371 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6425953.419 ± 163741.064 ns/op
```
Diffs (updated)
-----
src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java c2a342ce07bfb223193886038761f0da5230135d
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 1cb56f19c331508a1585077e9c4a98f52aac343b
src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 265c38d20136210e7639ac8ea915d307a4b72949
src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
Diff: https://reviews.apache.org/r/28617/diff/
Testing
-------
./gradlew -Pq build
Tested in vagrant
File Attachments
----------------
NoStaticVetoFiltering.png
https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
StaticVetoFiltering.png
https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
Thanks,
Maxim Khutornenko
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On Dec. 24, 2014, 12:26 a.m., Kevin Sweeney wrote:
> > src/main/java/org/apache/aurora/scheduler/TaskVars.java, line 79
> > <https://reviews.apache.org/r/28617/diff/4/?file=790574#file790574line79>
> >
> > How about either
> >
> > 1) Make this a member function of VetoGroup (so it's guaranteed by the compiler to be implemented if we add a new group type).
> >
> > 2) Make this a Function<VetoGroup, Optional<String>>.
> >
> > 3) Make this a Function<VetoGroup, String> with a default for a missing value.
> >
> > Right now we risk a runtime NullPointerException if we add a new VetoGroup implementation and forget to update this map in a different package.
#1 - metric name mapping does not logically fit into the VetoGroup
#2 - this would add extra heap churn (Optional creation) I am trying to avoid for this very high frequency call event handler
#3 - default string value would slip the build phase and manifest itself as an invalid metric name at runtime, which is arguably even worse.
How about a test case exhaustively covering all VetoGroup mappings instead? Added.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review65988
-----------------------------------------------------------
On Dec. 15, 2014, 11:12 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> -----------------------------------------------------------
>
> (Updated Dec. 15, 2014, 11:12 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
>
> Preliminary testing in vagrant (see pictures below) shows close to 50% improvement in task scheduling performance.
>
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and https://reviews.apache.org/r/28731/) shows over 97% better perf when testing with disabled preemptor:
> ```
> Master with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 8291046.074 ± 145251.995 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 7522269.050 ± 142446.265 ns/op
>
> This RB with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 204171.046 ± 3800.124 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 215854.129 ± 8959.851 ns/op
> ```
>
> Testing with preemptor enabled and running the worst case possible scenario (every slave is eligible and has task victims to evaluate) is still 16-22% faster than the master.
> ```
> Master with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7170244.522 ± 230259.848 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 8158184.909 ± 465853.379 ns/op
>
> This RB with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 6050901.381 ± 182134.371 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6425953.419 ± 163741.064 ns/op
> ```
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
> src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java c2a342ce07bfb223193886038761f0da5230135d
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 1cb56f19c331508a1585077e9c4a98f52aac343b
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
> src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
> src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 265c38d20136210e7639ac8ea915d307a4b72949
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
>
> Diff: https://reviews.apache.org/r/28617/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
> Tested in vagrant
>
>
> File Attachments
> ----------------
>
> NoStaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Kevin Sweeney <ke...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review65988
-----------------------------------------------------------
src/main/java/org/apache/aurora/scheduler/TaskVars.java
<https://reviews.apache.org/r/28617/#comment109291>
How about either
1) Make this a member function of VetoGroup (so it's guaranteed by the compiler to be implemented if we add a new group type).
2) Make this a Function<VetoGroup, Optional<String>>.
3) Make this a Function<VetoGroup, String> with a default for a missing value.
Right now we risk a runtime NullPointerException if we add a new VetoGroup implementation and forget to update this map in a different package.
- Kevin Sweeney
On Dec. 15, 2014, 3:12 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> -----------------------------------------------------------
>
> (Updated Dec. 15, 2014, 3:12 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
>
> Preliminary testing in vagrant (see pictures below) shows close to 50% improvement in task scheduling performance.
>
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and https://reviews.apache.org/r/28731/) shows over 97% better perf when testing with disabled preemptor:
> ```
> Master with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 8291046.074 ± 145251.995 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 7522269.050 ± 142446.265 ns/op
>
> This RB with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 204171.046 ± 3800.124 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 215854.129 ± 8959.851 ns/op
> ```
>
> Testing with preemptor enabled and running the worst case possible scenario (every slave is eligible and has task victims to evaluate) is still 16-22% faster than the master.
> ```
> Master with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7170244.522 ± 230259.848 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 8158184.909 ± 465853.379 ns/op
>
> This RB with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 6050901.381 ± 182134.371 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6425953.419 ± 163741.064 ns/op
> ```
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
> src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java c2a342ce07bfb223193886038761f0da5230135d
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 1cb56f19c331508a1585077e9c4a98f52aac343b
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
> src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
> src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 265c38d20136210e7639ac8ea915d307a4b72949
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
>
> Diff: https://reviews.apache.org/r/28617/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
> Tested in vagrant
>
>
> File Attachments
> ----------------
>
> NoStaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/
-----------------------------------------------------------
(Updated Dec. 15, 2014, 11:12 p.m.)
Review request for Aurora, Kevin Sweeney and Bill Farner.
Changes
-------
Updated test results.
Bugs: AURORA-909
https://issues.apache.org/jira/browse/AURORA-909
Repository: aurora
Description (updated)
-------
Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
Preliminary testing in vagrant (see pictures below) shows close to 50% improvement in task scheduling performance.
Update:
Testing with JMH (https://reviews.apache.org/r/28710/ and https://reviews.apache.org/r/28731/) shows over 97% better perf when testing with disabled preemptor:
```
Master with cluster fillup 0.9:
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 8291046.074 ± 145251.995 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 7522269.050 ± 142446.265 ns/op
This RB with cluster fillup 0.9:
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 204171.046 ± 3800.124 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 215854.129 ± 8959.851 ns/op
```
Testing with preemptor enabled and running the worst case possible scenario (every slave is eligible and has task victims to evaluate) is still 16-22% faster than the master.
```
Master with cluster fillup 0.9:
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 7170244.522 ± 230259.848 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 8158184.909 ± 465853.379 ns/op
This RB with cluster fillup 0.9:
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.runBenchmark avgt 100 6050901.381 ± 182134.371 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark avgt 100 6425953.419 ± 163741.064 ns/op
```
Diffs
-----
src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java c2a342ce07bfb223193886038761f0da5230135d
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 1cb56f19c331508a1585077e9c4a98f52aac343b
src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 265c38d20136210e7639ac8ea915d307a4b72949
src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
Diff: https://reviews.apache.org/r/28617/diff/
Testing
-------
./gradlew -Pq build
Tested in vagrant
File Attachments
----------------
NoStaticVetoFiltering.png
https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
StaticVetoFiltering.png
https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
Thanks,
Maxim Khutornenko
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/
-----------------------------------------------------------
(Updated Dec. 12, 2014, 6:05 p.m.)
Review request for Aurora, Kevin Sweeney and Bill Farner.
Changes
-------
CR comments.
Bugs: AURORA-909
https://issues.apache.org/jira/browse/AURORA-909
Repository: aurora
Description
-------
Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
Preliminary testing in vagrant (see pictures below) shows close to 50% improvement in task scheduling performance.
Update:
Testing with JMH (https://reviews.apache.org/r/28710/ and https://reviews.apache.org/r/28731/) shows over 97% better perf:
```
Master with cluster fillup 0.9:
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 8291046.074 ± 145251.995 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 7522269.050 ± 142446.265 ns/op
This RB with cluster fillup 0.9:
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 204171.046 ± 3800.124 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 215854.129 ± 8959.851 ns/op
```
Diffs (updated)
-----
src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java c2a342ce07bfb223193886038761f0da5230135d
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 1cb56f19c331508a1585077e9c4a98f52aac343b
src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 265c38d20136210e7639ac8ea915d307a4b72949
src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
Diff: https://reviews.apache.org/r/28617/diff/
Testing
-------
./gradlew -Pq build
Tested in vagrant
File Attachments
----------------
NoStaticVetoFiltering.png
https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
StaticVetoFiltering.png
https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
Thanks,
Maxim Khutornenko
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review64609
-----------------------------------------------------------
Heads up - i'm focusing on https://reviews.apache.org/r/28731/ before this review.
- Bill Farner
On Dec. 8, 2014, 10:57 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> -----------------------------------------------------------
>
> (Updated Dec. 8, 2014, 10:57 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
>
> Preliminary testing in vagrant (see pictures below) shows close to 50% improvement in task scheduling performance.
>
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and https://reviews.apache.org/r/28731/) shows over 97% better perf:
>
> ```
> Master with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 8291046.074 ± 145251.995 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 7522269.050 ± 142446.265 ns/op
>
> This RB with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 204171.046 ± 3800.124 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 215854.129 ± 8959.851 ns/op
> ```
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
> src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java ead9d28100673440168a32d114ecaa15874978a6
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java c2a342ce07bfb223193886038761f0da5230135d
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 1cb56f19c331508a1585077e9c4a98f52aac343b
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
> src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
> src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 265c38d20136210e7639ac8ea915d307a4b72949
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
>
> Diff: https://reviews.apache.org/r/28617/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
> Tested in vagrant
>
>
> File Attachments
> ----------------
>
> NoStaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On Dec. 12, 2014, 10:28 a.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java, line 295
> > <https://reviews.apache.org/r/28617/diff/3/?file=786034#file786034line295>
> >
> > staticallyBannedOffers is only filled but never cleared. Looks like we will be leaking memory.
Thanks for catching this! Added tests to validate cleanup.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review64890
-----------------------------------------------------------
On Dec. 12, 2014, 6:05 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> -----------------------------------------------------------
>
> (Updated Dec. 12, 2014, 6:05 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
>
> Preliminary testing in vagrant (see pictures below) shows close to 50% improvement in task scheduling performance.
>
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and https://reviews.apache.org/r/28731/) shows over 97% better perf:
>
> ```
> Master with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 8291046.074 ± 145251.995 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 7522269.050 ± 142446.265 ns/op
>
> This RB with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 204171.046 ± 3800.124 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 215854.129 ± 8959.851 ns/op
> ```
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
> src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java b6402ae42e3c7e4dca1c120fa6ef82d2d69e69d5
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java c2a342ce07bfb223193886038761f0da5230135d
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 1cb56f19c331508a1585077e9c4a98f52aac343b
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
> src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
> src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 265c38d20136210e7639ac8ea915d307a4b72949
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
>
> Diff: https://reviews.apache.org/r/28617/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
> Tested in vagrant
>
>
> File Attachments
> ----------------
>
> NoStaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Stephan Erb <ap...@stephanerb.eu>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review64890
-----------------------------------------------------------
src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java
<https://reviews.apache.org/r/28617/#comment107722>
staticallyBannedOffers is only filled but never cleared. Looks like we will be leaking memory.
- Stephan Erb
On Dec. 8, 2014, 10:57 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> -----------------------------------------------------------
>
> (Updated Dec. 8, 2014, 10:57 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
>
> Preliminary testing in vagrant (see pictures below) shows close to 50% improvement in task scheduling performance.
>
> Update:
> Testing with JMH (https://reviews.apache.org/r/28710/ and https://reviews.apache.org/r/28731/) shows over 97% better perf:
>
> ```
> Master with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 8291046.074 ± 145251.995 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 7522269.050 ± 142446.265 ns/op
>
> This RB with cluster fillup 0.9:
> Benchmark Mode Samples Score Error Units
> o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 204171.046 ± 3800.124 ns/op
> o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 215854.129 ± 8959.851 ns/op
> ```
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
> src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java ead9d28100673440168a32d114ecaa15874978a6
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java c2a342ce07bfb223193886038761f0da5230135d
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 1cb56f19c331508a1585077e9c4a98f52aac343b
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
> src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
> src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 265c38d20136210e7639ac8ea915d307a4b72949
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
>
> Diff: https://reviews.apache.org/r/28617/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
> Tested in vagrant
>
>
> File Attachments
> ----------------
>
> NoStaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/
-----------------------------------------------------------
(Updated Dec. 8, 2014, 10:57 p.m.)
Review request for Aurora, Kevin Sweeney and Bill Farner.
Changes
-------
CR comments.
Bugs: AURORA-909
https://issues.apache.org/jira/browse/AURORA-909
Repository: aurora
Description
-------
Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
Preliminary testing in vagrant (see pictures below) shows close to 50% improvement in task scheduling performance.
Update:
Testing with JMH (https://reviews.apache.org/r/28710/ and https://reviews.apache.org/r/28731/) shows over 97% better perf:
```
Master with cluster fillup 0.9:
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 8291046.074 ± 145251.995 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 7522269.050 ± 142446.265 ns/op
This RB with cluster fillup 0.9:
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 204171.046 ± 3800.124 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 215854.129 ± 8959.851 ns/op
```
Diffs (updated)
-----
src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java ead9d28100673440168a32d114ecaa15874978a6
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java c2a342ce07bfb223193886038761f0da5230135d
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 1cb56f19c331508a1585077e9c4a98f52aac343b
src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 265c38d20136210e7639ac8ea915d307a4b72949
src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
Diff: https://reviews.apache.org/r/28617/diff/
Testing
-------
./gradlew -Pq build
Tested in vagrant
File Attachments
----------------
NoStaticVetoFiltering.png
https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
StaticVetoFiltering.png
https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
Thanks,
Maxim Khutornenko
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/
-----------------------------------------------------------
(Updated Dec. 5, 2014, 10:57 p.m.)
Review request for Aurora, Kevin Sweeney and Bill Farner.
Changes
-------
Improving banned offer lookup performance.
Bugs: AURORA-909
https://issues.apache.org/jira/browse/AURORA-909
Repository: aurora
Description (updated)
-------
Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
Preliminary testing in vagrant (see pictures below) shows close to 50% improvement in task scheduling performance.
Update:
Testing with JMH (https://reviews.apache.org/r/28710/ and https://reviews.apache.org/r/28731/) shows over 97% better perf:
```
Master with cluster fillup 0.9:
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 8291046.074 ± 145251.995 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 7522269.050 ± 142446.265 ns/op
This RB with cluster fillup 0.9:
Benchmark Mode Samples Score Error Units
o.a.a.b.SchedulingBenchmarks.ConstraintMismatchsSchedulingBenchmark.example avgt 100 204171.046 ± 3800.124 ns/op
o.a.a.b.SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.example avgt 100 215854.129 ± 8959.851 ns/op
```
Diffs (updated)
-----
src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java ead9d28100673440168a32d114ecaa15874978a6
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java c2a342ce07bfb223193886038761f0da5230135d
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 1cb56f19c331508a1585077e9c4a98f52aac343b
src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 265c38d20136210e7639ac8ea915d307a4b72949
src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
Diff: https://reviews.apache.org/r/28617/diff/
Testing
-------
./gradlew -Pq build
Tested in vagrant
File Attachments
----------------
NoStaticVetoFiltering.png
https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
StaticVetoFiltering.png
https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
Thanks,
Maxim Khutornenko
Re: Review Request 28617: Implemented offer filtering for tasks with
static vetoes.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28617/#review63627
-----------------------------------------------------------
Ship it!
Master (552ae93) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Dec. 3, 2014, 1:15 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28617/
> -----------------------------------------------------------
>
> (Updated Dec. 3, 2014, 1:15 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-909
> https://issues.apache.org/jira/browse/AURORA-909
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Modified the task offer/task matching logic to skip offer matching for tasks previously vetoed statically.
>
> I will follow up with more accurate measurements (https://reviews.apache.org/r/28474/) but preliminary testing in vagrant (see pictures below) shows close to 50% improvement in task scheduling performance.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/TaskVars.java f017cdd26ca40138a7e141f21613ed567314c399
> src/main/java/org/apache/aurora/scheduler/async/OfferQueue.java f66383830140e5eaba436f35ebb5192eee65947a
> src/main/java/org/apache/aurora/scheduler/async/TaskScheduler.java ead9d28100673440168a32d114ecaa15874978a6
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java c2a342ce07bfb223193886038761f0da5230135d
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java 1cb56f19c331508a1585077e9c4a98f52aac343b
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e1c29747c9854cf75bf63f6f085cf40ca68989af
> src/test/java/org/apache/aurora/scheduler/TaskVarsTest.java 4e7efb3c1214c3d193afd61f162713490eb8effb
> src/test/java/org/apache/aurora/scheduler/async/OfferQueueImplTest.java 4cf602ad32b972c18eb5a81e9b2f59c67859bdb2
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerImplTest.java 5647349854a5e04de749c4d809684a0066d4da06
> src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 6cc13231560996b144101eba36577f49017aba06
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 265c38d20136210e7639ac8ea915d307a4b72949
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 411a55a8d85f60bb2703468f2d69b64b2736eee4
>
> Diff: https://reviews.apache.org/r/28617/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
> Tested in vagrant
>
>
> File Attachments
> ----------------
>
> NoStaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/7945c60b-4135-4016-a9bf-8d4815a4a573__NoStaticVetoFiltering.png
> StaticVetoFiltering.png
> https://reviews.apache.org/media/uploaded/files/2014/12/03/2f73b94a-5ba9-43b6-922e-e9e4ec18d0bb__StaticVetoFiltering.png
>
>
> Thanks,
>
> Maxim Khutornenko
>
>