You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by Bill Farner <wf...@apache.org> on 2017/10/18 21:25:03 UTC
Re: Review Request 62956: When scheduling,
skip offers with no CPU and no mem
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62956/
-----------------------------------------------------------
(Updated Oct. 18, 2017, 2:25 p.m.)
Review request for Aurora, David McLaughlin and Jordan Ly.
Changes
-------
Changed the approach in this patch to avoid disrupting other parts of the scheduler. In particular, preemption would have been impacted by rejecting 0 CPU/mem offers.
Summary (updated)
-----------------
When scheduling, skip offers with no CPU and no mem
Repository: aurora
Description
-------
There's no reason for us to evaluate offers with no CPUs or memory, so reject them early in the offer lifecycle.
This is an incremental performance optimization, but it may net significant improvements based on observations in some very large clusters.
Diffs (updated)
-----
src/main/java/org/apache/aurora/scheduler/HostOffer.java bc40d0798f40003cab5bf6efe607217e4d5de9f1
src/main/java/org/apache/aurora/scheduler/http/Utilization.java 3c77e2983ce00f897f3d5ed106b779cd7f7f0940
src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java 1b1239753f40d7d46d91724def6c25037eb79f1c
src/main/java/org/apache/aurora/scheduler/resources/ResourceBag.java d5db81b88a0369d0b26c8fbf70efab3886ad7695
src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e35720fca903cdcd27fff50f12ece9cc615f5d54
src/main/java/org/apache/aurora/scheduler/stats/TaskStatCalculator.java b98aaaf48ae60afef19a368ee96abc897300f8fa
src/test/java/org/apache/aurora/scheduler/HostOfferTest.java PRE-CREATION
src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 2cfdc090ff75a63111ae146c9fe7b3542e7ac83f
src/test/java/org/apache/aurora/scheduler/offers/Offers.java 129b4437315c6ad4ea47ca75d4ae6e28cadd7911
src/test/java/org/apache/aurora/scheduler/resources/ResourceTestUtil.java 765a527acb96997989c920be8b69dfa1113dc302
src/test/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerTest.java a74efbdc55383cd748f52cd55e1a94551d7d2ba1
Diff: https://reviews.apache.org/r/62956/diff/4/
Changes: https://reviews.apache.org/r/62956/diff/3-4/
Testing
-------
Thanks,
Bill Farner
Re: Review Request 62956: When scheduling,
skip offers with no CPU and no mem
Posted by Jordan Ly <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62956/#review188588
-----------------------------------------------------------
Ship it!
Ship It!
- Jordan Ly
On Oct. 18, 2017, 9:25 p.m., Bill Farner wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62956/
> -----------------------------------------------------------
>
> (Updated Oct. 18, 2017, 9:25 p.m.)
>
>
> Review request for Aurora, David McLaughlin and Jordan Ly.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> There's no reason for us to evaluate offers with no CPUs or memory, so reject them early in the offer lifecycle.
>
> This is an incremental performance optimization, but it may net significant improvements based on observations in some very large clusters.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/HostOffer.java bc40d0798f40003cab5bf6efe607217e4d5de9f1
> src/main/java/org/apache/aurora/scheduler/http/Utilization.java 3c77e2983ce00f897f3d5ed106b779cd7f7f0940
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java 1b1239753f40d7d46d91724def6c25037eb79f1c
> src/main/java/org/apache/aurora/scheduler/resources/ResourceBag.java d5db81b88a0369d0b26c8fbf70efab3886ad7695
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e35720fca903cdcd27fff50f12ece9cc615f5d54
> src/main/java/org/apache/aurora/scheduler/stats/TaskStatCalculator.java b98aaaf48ae60afef19a368ee96abc897300f8fa
> src/test/java/org/apache/aurora/scheduler/HostOfferTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 2cfdc090ff75a63111ae146c9fe7b3542e7ac83f
> src/test/java/org/apache/aurora/scheduler/offers/Offers.java 129b4437315c6ad4ea47ca75d4ae6e28cadd7911
> src/test/java/org/apache/aurora/scheduler/resources/ResourceTestUtil.java 765a527acb96997989c920be8b69dfa1113dc302
> src/test/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerTest.java a74efbdc55383cd748f52cd55e1a94551d7d2ba1
>
>
> Diff: https://reviews.apache.org/r/62956/diff/4/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bill Farner
>
>
Re: Review Request 62956: When scheduling,
skip offers with no CPU and no mem
Posted by Santhosh Kumar Shanmugham <sa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62956/#review188589
-----------------------------------------------------------
Ship it!
Ship It!
- Santhosh Kumar Shanmugham
On Oct. 18, 2017, 2:25 p.m., Bill Farner wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62956/
> -----------------------------------------------------------
>
> (Updated Oct. 18, 2017, 2:25 p.m.)
>
>
> Review request for Aurora, David McLaughlin and Jordan Ly.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> There's no reason for us to evaluate offers with no CPUs or memory, so reject them early in the offer lifecycle.
>
> This is an incremental performance optimization, but it may net significant improvements based on observations in some very large clusters.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/HostOffer.java bc40d0798f40003cab5bf6efe607217e4d5de9f1
> src/main/java/org/apache/aurora/scheduler/http/Utilization.java 3c77e2983ce00f897f3d5ed106b779cd7f7f0940
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java 1b1239753f40d7d46d91724def6c25037eb79f1c
> src/main/java/org/apache/aurora/scheduler/resources/ResourceBag.java d5db81b88a0369d0b26c8fbf70efab3886ad7695
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e35720fca903cdcd27fff50f12ece9cc615f5d54
> src/main/java/org/apache/aurora/scheduler/stats/TaskStatCalculator.java b98aaaf48ae60afef19a368ee96abc897300f8fa
> src/test/java/org/apache/aurora/scheduler/HostOfferTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 2cfdc090ff75a63111ae146c9fe7b3542e7ac83f
> src/test/java/org/apache/aurora/scheduler/offers/Offers.java 129b4437315c6ad4ea47ca75d4ae6e28cadd7911
> src/test/java/org/apache/aurora/scheduler/resources/ResourceTestUtil.java 765a527acb96997989c920be8b69dfa1113dc302
> src/test/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerTest.java a74efbdc55383cd748f52cd55e1a94551d7d2ba1
>
>
> Diff: https://reviews.apache.org/r/62956/diff/4/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bill Farner
>
>
Re: Review Request 62956: When scheduling,
skip offers with no CPU and no mem
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62956/#review188593
-----------------------------------------------------------
Master (ff39120) is red with this patch.
./build-support/jenkins/build.sh
1 error, 0 warnings potentially fixable with the `--fix` option.
npm ERR! Linux 4.4.0-43-generic
npm ERR! argv "/home/jenkins/jenkins-slave/workspace/AuroraBot/ui/.gradle/nodejs/node-v6.9.1-linux-x64/bin/node" "/home/jenkins/jenkins-slave/workspace/AuroraBot/ui/.gradle/nodejs/node-v6.9.1-linux-x64/bin/npm" "run" "lint"
npm ERR! node v6.9.1
npm ERR! npm v3.10.8
npm ERR! code ELIFECYCLE
npm ERR! apache-aurora@1.0.0 lint: `eslint src/main/js --ext .js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the apache-aurora@1.0.0 lint script 'eslint src/main/js --ext .js'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the apache-aurora package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! eslint src/main/js --ext .js
npm ERR! You can get information on how to open an issue for this project with:
npm ERR! npm bugs apache-aurora
npm ERR! Or if that isn't available, you can get their info via:
npm ERR! npm owner ls apache-aurora
npm ERR! There is likely additional logging output above.
npm ERR! Please include the following file with any support request:
npm ERR! /home/jenkins/jenkins-slave/workspace/AuroraBot/ui/npm-debug.log
:ui:lint FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':ui:lint'.
> Process 'command '/home/jenkins/jenkins-slave/workspace/AuroraBot/ui/.gradle/nodejs/node-v6.9.1-linux-x64/bin/npm'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
BUILD FAILED in 6m 40s
58 actionable tasks: 47 executed, 11 up-to-date
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Oct. 18, 2017, 9:25 p.m., Bill Farner wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62956/
> -----------------------------------------------------------
>
> (Updated Oct. 18, 2017, 9:25 p.m.)
>
>
> Review request for Aurora, David McLaughlin and Jordan Ly.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> There's no reason for us to evaluate offers with no CPUs or memory, so reject them early in the offer lifecycle.
>
> This is an incremental performance optimization, but it may net significant improvements based on observations in some very large clusters.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/HostOffer.java bc40d0798f40003cab5bf6efe607217e4d5de9f1
> src/main/java/org/apache/aurora/scheduler/http/Utilization.java 3c77e2983ce00f897f3d5ed106b779cd7f7f0940
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java 1b1239753f40d7d46d91724def6c25037eb79f1c
> src/main/java/org/apache/aurora/scheduler/resources/ResourceBag.java d5db81b88a0369d0b26c8fbf70efab3886ad7695
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e35720fca903cdcd27fff50f12ece9cc615f5d54
> src/main/java/org/apache/aurora/scheduler/stats/TaskStatCalculator.java b98aaaf48ae60afef19a368ee96abc897300f8fa
> src/test/java/org/apache/aurora/scheduler/HostOfferTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 2cfdc090ff75a63111ae146c9fe7b3542e7ac83f
> src/test/java/org/apache/aurora/scheduler/offers/Offers.java 129b4437315c6ad4ea47ca75d4ae6e28cadd7911
> src/test/java/org/apache/aurora/scheduler/resources/ResourceTestUtil.java 765a527acb96997989c920be8b69dfa1113dc302
> src/test/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerTest.java a74efbdc55383cd748f52cd55e1a94551d7d2ba1
>
>
> Diff: https://reviews.apache.org/r/62956/diff/4/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bill Farner
>
>
Re: Review Request 62956: When scheduling,
skip offers with no CPU and no mem
Posted by David McLaughlin <da...@dmclaughlin.com>.
> On Oct. 18, 2017, 9:29 p.m., David McLaughlin wrote:
> > src/main/java/org/apache/aurora/scheduler/HostOffer.java
> > Lines 64-65 (patched)
> > <https://reviews.apache.org/r/62956/diff/4/?file=1862825#file1862825line64>
> >
> > Shouldn't this be or (||)?
Oh, derp, nm. I read it backwards :)
- David
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62956/#review188582
-----------------------------------------------------------
On Oct. 18, 2017, 9:25 p.m., Bill Farner wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62956/
> -----------------------------------------------------------
>
> (Updated Oct. 18, 2017, 9:25 p.m.)
>
>
> Review request for Aurora, David McLaughlin and Jordan Ly.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> There's no reason for us to evaluate offers with no CPUs or memory, so reject them early in the offer lifecycle.
>
> This is an incremental performance optimization, but it may net significant improvements based on observations in some very large clusters.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/HostOffer.java bc40d0798f40003cab5bf6efe607217e4d5de9f1
> src/main/java/org/apache/aurora/scheduler/http/Utilization.java 3c77e2983ce00f897f3d5ed106b779cd7f7f0940
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java 1b1239753f40d7d46d91724def6c25037eb79f1c
> src/main/java/org/apache/aurora/scheduler/resources/ResourceBag.java d5db81b88a0369d0b26c8fbf70efab3886ad7695
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e35720fca903cdcd27fff50f12ece9cc615f5d54
> src/main/java/org/apache/aurora/scheduler/stats/TaskStatCalculator.java b98aaaf48ae60afef19a368ee96abc897300f8fa
> src/test/java/org/apache/aurora/scheduler/HostOfferTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 2cfdc090ff75a63111ae146c9fe7b3542e7ac83f
> src/test/java/org/apache/aurora/scheduler/offers/Offers.java 129b4437315c6ad4ea47ca75d4ae6e28cadd7911
> src/test/java/org/apache/aurora/scheduler/resources/ResourceTestUtil.java 765a527acb96997989c920be8b69dfa1113dc302
> src/test/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerTest.java a74efbdc55383cd748f52cd55e1a94551d7d2ba1
>
>
> Diff: https://reviews.apache.org/r/62956/diff/4/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bill Farner
>
>
Re: Review Request 62956: When scheduling,
skip offers with no CPU and no mem
Posted by David McLaughlin <da...@dmclaughlin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62956/#review188582
-----------------------------------------------------------
src/main/java/org/apache/aurora/scheduler/HostOffer.java
Lines 64-65 (patched)
<https://reviews.apache.org/r/62956/#comment265576>
Shouldn't this be or (||)?
- David McLaughlin
On Oct. 18, 2017, 9:25 p.m., Bill Farner wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62956/
> -----------------------------------------------------------
>
> (Updated Oct. 18, 2017, 9:25 p.m.)
>
>
> Review request for Aurora, David McLaughlin and Jordan Ly.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> There's no reason for us to evaluate offers with no CPUs or memory, so reject them early in the offer lifecycle.
>
> This is an incremental performance optimization, but it may net significant improvements based on observations in some very large clusters.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/HostOffer.java bc40d0798f40003cab5bf6efe607217e4d5de9f1
> src/main/java/org/apache/aurora/scheduler/http/Utilization.java 3c77e2983ce00f897f3d5ed106b779cd7f7f0940
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java 1b1239753f40d7d46d91724def6c25037eb79f1c
> src/main/java/org/apache/aurora/scheduler/resources/ResourceBag.java d5db81b88a0369d0b26c8fbf70efab3886ad7695
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e35720fca903cdcd27fff50f12ece9cc615f5d54
> src/main/java/org/apache/aurora/scheduler/stats/TaskStatCalculator.java b98aaaf48ae60afef19a368ee96abc897300f8fa
> src/test/java/org/apache/aurora/scheduler/HostOfferTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 2cfdc090ff75a63111ae146c9fe7b3542e7ac83f
> src/test/java/org/apache/aurora/scheduler/offers/Offers.java 129b4437315c6ad4ea47ca75d4ae6e28cadd7911
> src/test/java/org/apache/aurora/scheduler/resources/ResourceTestUtil.java 765a527acb96997989c920be8b69dfa1113dc302
> src/test/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerTest.java a74efbdc55383cd748f52cd55e1a94551d7d2ba1
>
>
> Diff: https://reviews.apache.org/r/62956/diff/4/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bill Farner
>
>
Re: Review Request 62956: When scheduling,
skip offers with no CPU and no mem
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62956/#review188623
-----------------------------------------------------------
Ship it!
Master (c638877) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Oct. 18, 2017, 9:25 p.m., Bill Farner wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62956/
> -----------------------------------------------------------
>
> (Updated Oct. 18, 2017, 9:25 p.m.)
>
>
> Review request for Aurora, David McLaughlin and Jordan Ly.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> There's no reason for us to evaluate offers with no CPUs or memory, so reject them early in the offer lifecycle.
>
> This is an incremental performance optimization, but it may net significant improvements based on observations in some very large clusters.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/HostOffer.java bc40d0798f40003cab5bf6efe607217e4d5de9f1
> src/main/java/org/apache/aurora/scheduler/http/Utilization.java 3c77e2983ce00f897f3d5ed106b779cd7f7f0940
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java 1b1239753f40d7d46d91724def6c25037eb79f1c
> src/main/java/org/apache/aurora/scheduler/resources/ResourceBag.java d5db81b88a0369d0b26c8fbf70efab3886ad7695
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e35720fca903cdcd27fff50f12ece9cc615f5d54
> src/main/java/org/apache/aurora/scheduler/stats/TaskStatCalculator.java b98aaaf48ae60afef19a368ee96abc897300f8fa
> src/test/java/org/apache/aurora/scheduler/HostOfferTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 2cfdc090ff75a63111ae146c9fe7b3542e7ac83f
> src/test/java/org/apache/aurora/scheduler/offers/Offers.java 129b4437315c6ad4ea47ca75d4ae6e28cadd7911
> src/test/java/org/apache/aurora/scheduler/resources/ResourceTestUtil.java 765a527acb96997989c920be8b69dfa1113dc302
> src/test/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerTest.java a74efbdc55383cd748f52cd55e1a94551d7d2ba1
>
>
> Diff: https://reviews.apache.org/r/62956/diff/5/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bill Farner
>
>
Re: Review Request 62956: When scheduling,
skip offers with no CPU and no mem
Posted by David McLaughlin <da...@dmclaughlin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62956/#review188585
-----------------------------------------------------------
Ship it!
Ship It!
- David McLaughlin
On Oct. 18, 2017, 9:25 p.m., Bill Farner wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62956/
> -----------------------------------------------------------
>
> (Updated Oct. 18, 2017, 9:25 p.m.)
>
>
> Review request for Aurora, David McLaughlin and Jordan Ly.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> There's no reason for us to evaluate offers with no CPUs or memory, so reject them early in the offer lifecycle.
>
> This is an incremental performance optimization, but it may net significant improvements based on observations in some very large clusters.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/HostOffer.java bc40d0798f40003cab5bf6efe607217e4d5de9f1
> src/main/java/org/apache/aurora/scheduler/http/Utilization.java 3c77e2983ce00f897f3d5ed106b779cd7f7f0940
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java 1b1239753f40d7d46d91724def6c25037eb79f1c
> src/main/java/org/apache/aurora/scheduler/resources/ResourceBag.java d5db81b88a0369d0b26c8fbf70efab3886ad7695
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e35720fca903cdcd27fff50f12ece9cc615f5d54
> src/main/java/org/apache/aurora/scheduler/stats/TaskStatCalculator.java b98aaaf48ae60afef19a368ee96abc897300f8fa
> src/test/java/org/apache/aurora/scheduler/HostOfferTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 2cfdc090ff75a63111ae146c9fe7b3542e7ac83f
> src/test/java/org/apache/aurora/scheduler/offers/Offers.java 129b4437315c6ad4ea47ca75d4ae6e28cadd7911
> src/test/java/org/apache/aurora/scheduler/resources/ResourceTestUtil.java 765a527acb96997989c920be8b69dfa1113dc302
> src/test/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerTest.java a74efbdc55383cd748f52cd55e1a94551d7d2ba1
>
>
> Diff: https://reviews.apache.org/r/62956/diff/4/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bill Farner
>
>
Re: Review Request 62956: When scheduling,
skip offers with no CPU and no mem
Posted by Bill Farner <wf...@apache.org>.
> On Oct. 18, 2017, 2:48 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/HostOffer.java
> > Lines 44-47 (patched)
> > <https://reviews.apache.org/r/62956/diff/4/?file=1862825#file1862825line44>
> >
> > Please extend this comment a bit and indicate that this checks for the absence of revocable and non-revocable resources.
Done.
> On Oct. 18, 2017, 2:48 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/HostOffer.java
> > Lines 62-66 (patched)
> > <https://reviews.apache.org/r/62956/diff/4/?file=1862825#file1862825line62>
> >
> > I have had a short peek at our clusters and the number of offers matching this criteria would be very low.
> >
> > It could be increased if we'd use a higher bound like 0.1 CPU and 64mb memory as those would already be needed as executor overhead.
> >
> > I suspect for Twitter even using 0 as abound is significant though? Correct?
>
> David McLaughlin wrote:
> For Twitter is is quite large, but it would be large (as a %) for anyone who enables any type of bin-packing.
I'd like to start with the zero-config patch with little room for doubt. Though i welcome the follow-up patch/discussion on a higher threshold if you think it would benefit you, Stephan!
> On Oct. 18, 2017, 2:48 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java
> > Lines 295-296 (patched)
> > <https://reviews.apache.org/r/62956/diff/4/?file=1862829#file1862829line295>
> >
> > For consistency with the other early aborts below, write this in form of an `if` with a `continue` statement?
> >
> > (I don't feel strongly about this though. It just feels a little bit weird that this is handled differently here)
I agree, this is poor consistency. Fixed.
**Note**: i am also going to change the meaning of the `assigner_evaluated_offers` stat. It currently counts these early aborts, which i believe will complicate any kind of performance analysis related to optimizations like this.
- Bill
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62956/#review188586
-----------------------------------------------------------
On Oct. 18, 2017, 2:25 p.m., Bill Farner wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62956/
> -----------------------------------------------------------
>
> (Updated Oct. 18, 2017, 2:25 p.m.)
>
>
> Review request for Aurora, David McLaughlin and Jordan Ly.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> There's no reason for us to evaluate offers with no CPUs or memory, so reject them early in the offer lifecycle.
>
> This is an incremental performance optimization, but it may net significant improvements based on observations in some very large clusters.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/HostOffer.java bc40d0798f40003cab5bf6efe607217e4d5de9f1
> src/main/java/org/apache/aurora/scheduler/http/Utilization.java 3c77e2983ce00f897f3d5ed106b779cd7f7f0940
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java 1b1239753f40d7d46d91724def6c25037eb79f1c
> src/main/java/org/apache/aurora/scheduler/resources/ResourceBag.java d5db81b88a0369d0b26c8fbf70efab3886ad7695
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e35720fca903cdcd27fff50f12ece9cc615f5d54
> src/main/java/org/apache/aurora/scheduler/stats/TaskStatCalculator.java b98aaaf48ae60afef19a368ee96abc897300f8fa
> src/test/java/org/apache/aurora/scheduler/HostOfferTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 2cfdc090ff75a63111ae146c9fe7b3542e7ac83f
> src/test/java/org/apache/aurora/scheduler/offers/Offers.java 129b4437315c6ad4ea47ca75d4ae6e28cadd7911
> src/test/java/org/apache/aurora/scheduler/resources/ResourceTestUtil.java 765a527acb96997989c920be8b69dfa1113dc302
> src/test/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerTest.java a74efbdc55383cd748f52cd55e1a94551d7d2ba1
>
>
> Diff: https://reviews.apache.org/r/62956/diff/4/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bill Farner
>
>
Re: Review Request 62956: When scheduling,
skip offers with no CPU and no mem
Posted by David McLaughlin <da...@dmclaughlin.com>.
> On Oct. 18, 2017, 9:48 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/HostOffer.java
> > Lines 62-66 (patched)
> > <https://reviews.apache.org/r/62956/diff/4/?file=1862825#file1862825line62>
> >
> > I have had a short peek at our clusters and the number of offers matching this criteria would be very low.
> >
> > It could be increased if we'd use a higher bound like 0.1 CPU and 64mb memory as those would already be needed as executor overhead.
> >
> > I suspect for Twitter even using 0 as abound is significant though? Correct?
For Twitter is is quite large, but it would be large (as a %) for anyone who enables any type of bin-packing.
- David
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62956/#review188586
-----------------------------------------------------------
On Oct. 18, 2017, 9:25 p.m., Bill Farner wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62956/
> -----------------------------------------------------------
>
> (Updated Oct. 18, 2017, 9:25 p.m.)
>
>
> Review request for Aurora, David McLaughlin and Jordan Ly.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> There's no reason for us to evaluate offers with no CPUs or memory, so reject them early in the offer lifecycle.
>
> This is an incremental performance optimization, but it may net significant improvements based on observations in some very large clusters.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/HostOffer.java bc40d0798f40003cab5bf6efe607217e4d5de9f1
> src/main/java/org/apache/aurora/scheduler/http/Utilization.java 3c77e2983ce00f897f3d5ed106b779cd7f7f0940
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java 1b1239753f40d7d46d91724def6c25037eb79f1c
> src/main/java/org/apache/aurora/scheduler/resources/ResourceBag.java d5db81b88a0369d0b26c8fbf70efab3886ad7695
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e35720fca903cdcd27fff50f12ece9cc615f5d54
> src/main/java/org/apache/aurora/scheduler/stats/TaskStatCalculator.java b98aaaf48ae60afef19a368ee96abc897300f8fa
> src/test/java/org/apache/aurora/scheduler/HostOfferTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 2cfdc090ff75a63111ae146c9fe7b3542e7ac83f
> src/test/java/org/apache/aurora/scheduler/offers/Offers.java 129b4437315c6ad4ea47ca75d4ae6e28cadd7911
> src/test/java/org/apache/aurora/scheduler/resources/ResourceTestUtil.java 765a527acb96997989c920be8b69dfa1113dc302
> src/test/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerTest.java a74efbdc55383cd748f52cd55e1a94551d7d2ba1
>
>
> Diff: https://reviews.apache.org/r/62956/diff/4/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bill Farner
>
>
Re: Review Request 62956: When scheduling,
skip offers with no CPU and no mem
Posted by Stephan Erb <se...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62956/#review188586
-----------------------------------------------------------
Ship it!
src/main/java/org/apache/aurora/scheduler/HostOffer.java
Lines 44-47 (patched)
<https://reviews.apache.org/r/62956/#comment265579>
Please extend this comment a bit and indicate that this checks for the absence of revocable and non-revocable resources.
src/main/java/org/apache/aurora/scheduler/HostOffer.java
Lines 62-66 (patched)
<https://reviews.apache.org/r/62956/#comment265583>
I have had a short peek at our clusters and the number of offers matching this criteria would be very low.
It could be increased if we'd use a higher bound like 0.1 CPU and 64mb memory as those would already be needed as executor overhead.
I suspect for Twitter even using 0 as abound is significant though? Correct?
src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java
Lines 295-296 (patched)
<https://reviews.apache.org/r/62956/#comment265580>
For consistency with the other early aborts below, write this in form of an `if` with a `continue` statement?
(I don't feel strongly about this though. It just feels a little bit weird that this is handled differently here)
- Stephan Erb
On Oct. 18, 2017, 11:25 p.m., Bill Farner wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62956/
> -----------------------------------------------------------
>
> (Updated Oct. 18, 2017, 11:25 p.m.)
>
>
> Review request for Aurora, David McLaughlin and Jordan Ly.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> There's no reason for us to evaluate offers with no CPUs or memory, so reject them early in the offer lifecycle.
>
> This is an incremental performance optimization, but it may net significant improvements based on observations in some very large clusters.
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/HostOffer.java bc40d0798f40003cab5bf6efe607217e4d5de9f1
> src/main/java/org/apache/aurora/scheduler/http/Utilization.java 3c77e2983ce00f897f3d5ed106b779cd7f7f0940
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java 1b1239753f40d7d46d91724def6c25037eb79f1c
> src/main/java/org/apache/aurora/scheduler/resources/ResourceBag.java d5db81b88a0369d0b26c8fbf70efab3886ad7695
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java e35720fca903cdcd27fff50f12ece9cc615f5d54
> src/main/java/org/apache/aurora/scheduler/stats/TaskStatCalculator.java b98aaaf48ae60afef19a368ee96abc897300f8fa
> src/test/java/org/apache/aurora/scheduler/HostOfferTest.java PRE-CREATION
> src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 2cfdc090ff75a63111ae146c9fe7b3542e7ac83f
> src/test/java/org/apache/aurora/scheduler/offers/Offers.java 129b4437315c6ad4ea47ca75d4ae6e28cadd7911
> src/test/java/org/apache/aurora/scheduler/resources/ResourceTestUtil.java 765a527acb96997989c920be8b69dfa1113dc302
> src/test/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerTest.java a74efbdc55383cd748f52cd55e1a94551d7d2ba1
>
>
> Diff: https://reviews.apache.org/r/62956/diff/4/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Bill Farner
>
>