You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by Maxim Khutornenko <ma...@apache.org> on 2015/02/27 04:19:34 UTC
Review Request 31525: Improving NearestFit reporting accuracy.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31525/
-----------------------------------------------------------
Review request for Aurora, Kevin Sweeney and Bill Farner.
Bugs: AURORA-1148
https://issues.apache.org/jira/browse/AURORA-1148
Repository: aurora
Description
-------
Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with "break early" rule in SchedulingFilter, this ensures "heavier" vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance.
Diffs
-----
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147
src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b
src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424
src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02
Diff: https://reviews.apache.org/r/31525/diff/
Testing
-------
./gradlew -Pq build
Thanks,
Maxim Khutornenko
Re: Review Request 31525: Improving NearestFit reporting accuracy.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On Feb. 27, 2015, 7:30 p.m., Kevin Sweeney wrote:
> > src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java, lines 110-113
> > <https://reviews.apache.org/r/31525/diff/1/?file=879537#file879537line110>
> >
> > How about making score an EnumSet<VetoType> so that you can do a typesafe bitwise OR?
These scores did not match directly to VetoType. However, your comment got me thinking it's quite easy to fix that. Created a new DEDICATED_CONSTRAINT_MISMATCH and moved all scores definitions into the VetoType enum.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31525/#review74554
-----------------------------------------------------------
On Feb. 27, 2015, 3:19 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31525/
> -----------------------------------------------------------
>
> (Updated Feb. 27, 2015, 3:19 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-1148
> https://issues.apache.org/jira/browse/AURORA-1148
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with "break early" rule in SchedulingFilter, this ensures "heavier" vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147
> src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424
> src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02
>
> Diff: https://reviews.apache.org/r/31525/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 31525: Improving NearestFit reporting accuracy.
Posted by Kevin Sweeney <ke...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31525/#review74554
-----------------------------------------------------------
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java
<https://reviews.apache.org/r/31525/#comment121148>
How about making score an EnumSet<VetoType> so that you can do a typesafe bitwise OR?
- Kevin Sweeney
On Feb. 26, 2015, 7:19 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31525/
> -----------------------------------------------------------
>
> (Updated Feb. 26, 2015, 7:19 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-1148
> https://issues.apache.org/jira/browse/AURORA-1148
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with "break early" rule in SchedulingFilter, this ensures "heavier" vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147
> src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424
> src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02
>
> Diff: https://reviews.apache.org/r/31525/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 31525: Improving NearestFit reporting accuracy.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31525/#review74594
-----------------------------------------------------------
Ship it!
Master (e8a0a9f) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Feb. 27, 2015, 9:05 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31525/
> -----------------------------------------------------------
>
> (Updated Feb. 27, 2015, 9:05 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-1148
> https://issues.apache.org/jira/browse/AURORA-1148
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with "break early" rule in SchedulingFilter, this ensures "heavier" vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147
> src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424
> src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02
>
> Diff: https://reviews.apache.org/r/31525/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 31525: Improving NearestFit reporting accuracy.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On March 2, 2015, 7:40 p.m., Bill Farner wrote:
> > Can you elaborate on how this change would have addressed this scenario described in AURORA-1148? Seems like the confusion of only seeing 'Insufficient RAM' is not resolved by this change alone.
Correct, it's an incremental quality improvement rather than a comprehensive fix (which I am not even sure is possible given our current NearestFit approach). The idea is to NOT emit lower score (aka "better fit") vetoes until the higher score vetoes are dismissed. Here is the most likely execution sequence in AURORA-1148 case:
1. non-decicated hosts are filtered out (veto reason: dedicated constraint mismatch)
2. dedicated hosts are evaluated wrt diversity constraints (veto reason: host limit constraint mismatch)
3. dedicated host resources are evaluated (veto reason: insufficient RAM/CPU/DISK)
Given that every VetoType triggered short-circuits further veto analysis, users would *more* likely see a host limit constraint.
What this change does not solve is that in case there are some offers satisfying diversity analysis they would "pollute" veto reason with "Insufficient resource" veto thus humpering investigation. Given our current 10 minute veto reason expiration and default offer hold time (5 minutes), users would likely see pending reason occasionally flipping from "host limit mismatch" to "insufficient resource".
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31525/#review74801
-----------------------------------------------------------
On Feb. 27, 2015, 9:05 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31525/
> -----------------------------------------------------------
>
> (Updated Feb. 27, 2015, 9:05 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-1148
> https://issues.apache.org/jira/browse/AURORA-1148
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with "break early" rule in SchedulingFilter, this ensures "heavier" vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147
> src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424
> src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02
>
> Diff: https://reviews.apache.org/r/31525/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 31525: Improving NearestFit reporting accuracy.
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31525/#review74801
-----------------------------------------------------------
Can you elaborate on how this change would have addressed this scenario described in AURORA-1148? Seems like the confusion of only seeing 'Insufficient RAM' is not resolved by this change alone.
- Bill Farner
On Feb. 27, 2015, 9:05 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31525/
> -----------------------------------------------------------
>
> (Updated Feb. 27, 2015, 9:05 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-1148
> https://issues.apache.org/jira/browse/AURORA-1148
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with "break early" rule in SchedulingFilter, this ensures "heavier" vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147
> src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424
> src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02
>
> Diff: https://reviews.apache.org/r/31525/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 31525: Improving NearestFit reporting accuracy.
Posted by Maxim Khutornenko <ma...@apache.org>.
> On March 2, 2015, 7:14 p.m., Kevin Sweeney wrote:
> > src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java, line 64
> > <https://reviews.apache.org/r/31525/diff/2/?file=880724#file880724line64>
> >
> > I don't think the Math.pow approach is necessary - why not make score an EnumMap<VetoType, Integer> (or just an EnumSet<VetoType>). A veto increments the counter in the bucket for its type (or sets the bit for it). Still very efficient and more type-safe.
Not sure I understand the proposal. The score buckets, as currently defined, flatten out the VetoType hierarchy via non-intersecting int ranges. This abstracts out the internal VetoType relationship and simplifies the consuming side (NearestFit). The sum of all previous level veto scores should never exceed the next level score. I don't see how we can get by with EnumSet<VetoType> here without making NearestFit aware of VetoType hierarchy.
- Maxim
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31525/#review74793
-----------------------------------------------------------
On Feb. 27, 2015, 9:05 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31525/
> -----------------------------------------------------------
>
> (Updated Feb. 27, 2015, 9:05 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-1148
> https://issues.apache.org/jira/browse/AURORA-1148
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with "break early" rule in SchedulingFilter, this ensures "heavier" vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147
> src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424
> src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02
>
> Diff: https://reviews.apache.org/r/31525/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 31525: Improving NearestFit reporting accuracy.
Posted by Kevin Sweeney <ke...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31525/#review74793
-----------------------------------------------------------
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java
<https://reviews.apache.org/r/31525/#comment121550>
I don't think the Math.pow approach is necessary - why not make score an EnumMap<VetoType, Integer> (or just an EnumSet<VetoType>). A veto increments the counter in the bucket for its type (or sets the bit for it). Still very efficient and more type-safe.
- Kevin Sweeney
On Feb. 27, 2015, 1:05 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31525/
> -----------------------------------------------------------
>
> (Updated Feb. 27, 2015, 1:05 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-1148
> https://issues.apache.org/jira/browse/AURORA-1148
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with "break early" rule in SchedulingFilter, this ensures "heavier" vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147
> src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424
> src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02
>
> Diff: https://reviews.apache.org/r/31525/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 31525: Improving NearestFit reporting accuracy.
Posted by Kevin Sweeney <ke...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31525/#review75591
-----------------------------------------------------------
Ship it!
Ship It!
- Kevin Sweeney
On March 2, 2015, 3:40 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31525/
> -----------------------------------------------------------
>
> (Updated March 2, 2015, 3:40 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-1159
> https://issues.apache.org/jira/browse/AURORA-1159
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with "break early" rule in SchedulingFilter, this ensures "heavier" vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147
> src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424
> src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02
>
> Diff: https://reviews.apache.org/r/31525/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 31525: Improving NearestFit reporting accuracy.
Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31525/#review75103
-----------------------------------------------------------
Ship it!
Ship It!
- Bill Farner
On March 2, 2015, 11:40 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31525/
> -----------------------------------------------------------
>
> (Updated March 2, 2015, 11:40 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-1159
> https://issues.apache.org/jira/browse/AURORA-1159
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with "break early" rule in SchedulingFilter, this ensures "heavier" vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147
> src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424
> src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02
>
> Diff: https://reviews.apache.org/r/31525/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 31525: Improving NearestFit reporting accuracy.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31525/#review75801
-----------------------------------------------------------
Master (e55113d) is red with this patch.
./build-support/jenkins/build.sh
+ date
Mon Mar 9 22:35:54 UTC 2015
+ ./gradlew -Pq clean build
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0xaf11a52c, pid=16787, tid=4136401728
#
# JRE version: 7.0_25-b15
# Java VM: Java HotSpot(TM) Server VM (23.25-b01 mixed mode linux-x86 )
# Problematic frame:
# C [libnet.so+0x352c] _init+0x620
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /home/jenkins/jenkins-slave/workspace/AuroraBot/hs_err_pid16787.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.sun.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
./build-support/jenkins/build.sh: line 21: 16787 Aborted ./gradlew -Pq clean build
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On March 9, 2015, 9:59 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31525/
> -----------------------------------------------------------
>
> (Updated March 9, 2015, 9:59 p.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-1159
> https://issues.apache.org/jira/browse/AURORA-1159
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with "break early" rule in SchedulingFilter, this ensures "heavier" vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147
> src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java 785b5742301eec6930a431585256fefce7ec776d
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424
> src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java e0f9d8b4c3d80b70faa10612b82a034e3dae9112
>
> Diff: https://reviews.apache.org/r/31525/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
>
> Thanks,
>
> Maxim Khutornenko
>
>
Re: Review Request 31525: Improving NearestFit reporting accuracy.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31525/
-----------------------------------------------------------
(Updated March 9, 2015, 9:59 p.m.)
Review request for Aurora, Kevin Sweeney and Bill Farner.
Changes
-------
Rebased.
Bugs: AURORA-1159
https://issues.apache.org/jira/browse/AURORA-1159
Repository: aurora
Description
-------
Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with "break early" rule in SchedulingFilter, this ensures "heavier" vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance.
Diffs (updated)
-----
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147
src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java 785b5742301eec6930a431585256fefce7ec776d
src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424
src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java e0f9d8b4c3d80b70faa10612b82a034e3dae9112
Diff: https://reviews.apache.org/r/31525/diff/
Testing
-------
./gradlew -Pq build
Thanks,
Maxim Khutornenko
Re: Review Request 31525: Improving NearestFit reporting accuracy.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31525/
-----------------------------------------------------------
(Updated March 2, 2015, 11:40 p.m.)
Review request for Aurora, Kevin Sweeney and Bill Farner.
Changes
-------
Updating ticket number
Bugs: AURORA-1159
https://issues.apache.org/jira/browse/AURORA-1159
Repository: aurora
Description
-------
Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with "break early" rule in SchedulingFilter, this ensures "heavier" vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance.
Diffs
-----
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147
src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b
src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424
src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02
Diff: https://reviews.apache.org/r/31525/diff/
Testing
-------
./gradlew -Pq build
Thanks,
Maxim Khutornenko
Re: Review Request 31525: Improving NearestFit reporting accuracy.
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31525/
-----------------------------------------------------------
(Updated Feb. 27, 2015, 9:05 p.m.)
Review request for Aurora, Kevin Sweeney and Bill Farner.
Changes
-------
Kevin's comments.
Bugs: AURORA-1148
https://issues.apache.org/jira/browse/AURORA-1148
Repository: aurora
Description
-------
Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with "break early" rule in SchedulingFilter, this ensures "heavier" vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance.
Diffs (updated)
-----
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963
src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147
src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b
src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424
src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02
Diff: https://reviews.apache.org/r/31525/diff/
Testing
-------
./gradlew -Pq build
Thanks,
Maxim Khutornenko
Re: Review Request 31525: Improving NearestFit reporting accuracy.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31525/#review74434
-----------------------------------------------------------
Master (766d1c9) is red with this patch.
./build-support/jenkins/build.sh
src.test.python.apache.aurora.client.cli.update ..... SUCCESS
src.test.python.apache.aurora.client.cli.version ..... SUCCESS
src.test.python.apache.aurora.client.config ..... SUCCESS
src.test.python.apache.aurora.client.factory ..... SUCCESS
src.test.python.apache.aurora.client.hooks.hooked_api ..... SUCCESS
src.test.python.apache.aurora.client.hooks.non_hooked_api ..... SUCCESS
src.test.python.apache.aurora.common.test_aurora_job_key ..... SUCCESS
src.test.python.apache.aurora.common.test_cluster ..... SUCCESS
src.test.python.apache.aurora.common.test_cluster_option ..... SUCCESS
src.test.python.apache.aurora.common.test_clusters ..... SUCCESS
src.test.python.apache.aurora.common.test_http_signaler ..... SUCCESS
src.test.python.apache.aurora.common.test_pex_version ..... SUCCESS
src.test.python.apache.aurora.common.test_shellify ..... SUCCESS
src.test.python.apache.aurora.common.test_transport ..... SUCCESS
src.test.python.apache.aurora.config.test_base ..... SUCCESS
src.test.python.apache.aurora.config.test_constraint_parsing ..... SUCCESS
src.test.python.apache.aurora.config.test_loader ..... SUCCESS
src.test.python.apache.aurora.config.test_thrift ..... SUCCESS
src.test.python.apache.aurora.executor.common.announcer ..... SUCCESS
src.test.python.apache.aurora.executor.common.directory_sandbox ..... SUCCESS
src.test.python.apache.aurora.executor.common.executor_detector ..... SUCCESS
src.test.python.apache.aurora.executor.common.executor_timeout ..... SUCCESS
src.test.python.apache.aurora.executor.common.health_checker ..... SUCCESS
src.test.python.apache.aurora.executor.common.kill_manager ..... SUCCESS
src.test.python.apache.aurora.executor.common.path_detector ..... SUCCESS
src.test.python.apache.aurora.executor.common.status_checker ..... SUCCESS
src.test.python.apache.aurora.executor.common.task_info ..... SUCCESS
src.test.python.apache.aurora.executor.executor_base ..... SUCCESS
src.test.python.apache.aurora.executor.executor_vars ..... SUCCESS
src.test.python.apache.aurora.executor.gc_executor ..... FAILURE
src.test.python.apache.aurora.executor.status_manager ..... SUCCESS
src.test.python.apache.aurora.executor.thermos_task_runner ..... SUCCESS
src.test.python.apache.thermos.common.test_pathspec ..... SUCCESS
src.test.python.apache.thermos.core.test_runner_integration ..... SUCCESS
src.test.python.apache.thermos.monitoring.test_disk ..... SUCCESS
FAILURE
[31m
FAILURE[0m
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Feb. 27, 2015, 3:19 a.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31525/
> -----------------------------------------------------------
>
> (Updated Feb. 27, 2015, 3:19 a.m.)
>
>
> Review request for Aurora, Kevin Sweeney and Bill Farner.
>
>
> Bugs: AURORA-1148
> https://issues.apache.org/jira/browse/AURORA-1148
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Instead of relying on a fixed MAX_SCORE, vetoes are now graded by their relative weight (severity) with a dedicated constraint mismatch ranked higest and an insufficient resource mismatch - lowest. Together with "break early" rule in SchedulingFilter, this ensures "heavier" vetoes are given proper attention in the NearestFit. This simplifies `NearestFit` logic while also improving pending reason reporting accuracy and scheduling performance.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java 3313bd2f9ed5a88375d88715dea5da1e9ad8b963
> src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilterImpl.java a020ce50d578fba7f32b370f448a49eb1c284147
> src/main/java/org/apache/aurora/scheduler/metadata/NearestFit.java c3097e49c0f6588ea765aa4fab69dd35e3d90e8b
> src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java b00668423a53a8cf6f4da3456bce3354f1fd2424
> src/test/java/org/apache/aurora/scheduler/metadata/NearestFitTest.java 78a236c0f9074692b67ce18e6e03f18fe4529e02
>
> Diff: https://reviews.apache.org/r/31525/diff/
>
>
> Testing
> -------
>
> ./gradlew -Pq build
>
>
> Thanks,
>
> Maxim Khutornenko
>
>