You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by David McLaughlin <da...@dmclaughlin.com> on 2017/05/06 16:52:51 UTC
Review Request 59039: Add the ability to customize scheduling logic.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59039/
-----------------------------------------------------------
Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
Bugs: AURORA-1920
https://issues.apache.org/jira/browse/AURORA-1920
Repository: aurora
Description
-------
Add the ability to replace the first-fit scheduling algorithm and associated first-fit preemption logic.
See design/proposal document here: https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit?usp=sharing
Diffs
-----
src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorModule.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterModule.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java 66d20290d381c041ac2f28903d6ba1468b352172
src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/state/StateModule.java 0186484208965c5b8942ec76eb078b30190c7d57
src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java da064e3589951d4f4cbff96d1ed89d9d7e4d3882
Diff: https://reviews.apache.org/r/59039/diff/1/
Testing
-------
./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
Before shipping this, I'm also going to migrate out internal fork of TaskAssigner over to using this approach to verify everything looks good.
Thanks,
David McLaughlin
Re: Review Request 59039: Add the ability to customize scheduling
logic.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59039/#review174117
-----------------------------------------------------------
Ship it!
Master (85fed6b) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On May 6, 2017, 4:52 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59039/
> -----------------------------------------------------------
>
> (Updated May 6, 2017, 4:52 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Bugs: AURORA-1920
> https://issues.apache.org/jira/browse/AURORA-1920
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Add the ability to replace the first-fit scheduling algorithm and associated first-fit preemption logic.
>
> See design/proposal document here: https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit?usp=sharing
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java 66d20290d381c041ac2f28903d6ba1468b352172
> src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/state/StateModule.java 0186484208965c5b8942ec76eb078b30190c7d57
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java da064e3589951d4f4cbff96d1ed89d9d7e4d3882
>
>
> Diff: https://reviews.apache.org/r/59039/diff/1/
>
>
> Testing
> -------
>
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Before shipping this, I'm also going to migrate out internal fork of TaskAssigner over to using this approach to verify everything looks good.
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 59039: Add the ability to customize scheduling
logic.
Posted by Santhosh Kumar Shanmugham <sa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59039/#review174422
-----------------------------------------------------------
I went through the code a few times. I am slighlty confused about the reason for exposing `PreemptionVictimFilter`, can you expand on it?
Otherwise the change looks good to me. Please update release and operator notes calling out that this is an experimental feature.
- Santhosh Kumar Shanmugham
On May 9, 2017, 2:42 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59039/
> -----------------------------------------------------------
>
> (Updated May 9, 2017, 2:42 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Bugs: AURORA-1920
> https://issues.apache.org/jira/browse/AURORA-1920
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Add the ability to replace the first-fit scheduling algorithm and associated first-fit preemption logic.
>
> See design/proposal document here: https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit?usp=sharing
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java 66d20290d381c041ac2f28903d6ba1468b352172
> src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/state/StateModule.java 0186484208965c5b8942ec76eb078b30190c7d57
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java da064e3589951d4f4cbff96d1ed89d9d7e4d3882
>
>
> Diff: https://reviews.apache.org/r/59039/diff/1/
>
>
> Testing
> -------
>
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
> I've used this in one of our large-scale test clusters to plug in a custom task assigner. We were able to remove our modifications to the OSS code-base and use this mechanism to do what we need. Here is a rough gist of what it looks like:
> https://gist.github.com/DavidMcLaughlin/5d5472cde15343aef8705478f644ddb9
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 59039: Add the ability to customize scheduling
logic.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59039/#review174736
-----------------------------------------------------------
Ship it!
Master (b0d24f4) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On May 11, 2017, 2:46 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59039/
> -----------------------------------------------------------
>
> (Updated May 11, 2017, 2:46 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Bugs: AURORA-1920
> https://issues.apache.org/jira/browse/AURORA-1920
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Add the ability to replace the first-fit scheduling algorithm and associated first-fit preemption logic.
>
> See design/proposal document here: https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit?usp=sharing
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md 4e930fb1c5b87bca3cf4d8de804d69301f013f07
> docs/development/design-documents.md c942643d6220cb61eb97289b35d92facae06a682
> src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java 66d20290d381c041ac2f28903d6ba1468b352172
> src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/state/StateModule.java 0186484208965c5b8942ec76eb078b30190c7d57
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java da064e3589951d4f4cbff96d1ed89d9d7e4d3882
>
>
> Diff: https://reviews.apache.org/r/59039/diff/2/
>
>
> Testing
> -------
>
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
> I've used this in one of our large-scale test clusters to plug in a custom task assigner. We were able to remove our modifications to the OSS code-base and use this mechanism to do what we need. Here is a rough gist of what it looks like:
> https://gist.github.com/DavidMcLaughlin/5d5472cde15343aef8705478f644ddb9
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 59039: Add the ability to customize scheduling
logic.
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59039/#review176044
-----------------------------------------------------------
Ship it!
Master (1fbf773) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On May 25, 2017, 5:50 a.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59039/
> -----------------------------------------------------------
>
> (Updated May 25, 2017, 5:50 a.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Bugs: AURORA-1920
> https://issues.apache.org/jira/browse/AURORA-1920
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Add the ability to replace the first-fit scheduling algorithm and associated first-fit preemption logic.
>
> See design/proposal document here: https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit?usp=sharing
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md 77376e438bd7af74c364dcd5d1b3e3f1ece2adbf
> docs/development/design-documents.md c942643d6220cb61eb97289b35d92facae06a682
> docs/reference/scheduler-configuration.md 3e3d7991970fc75907d07c5a8a31d508c26eda78
> src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java 92087eb60cb81e1c64eaa7d1e2da7f6bbd51500c
> src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/state/StateModule.java 0186484208965c5b8942ec76eb078b30190c7d57
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 7c531afee880022af5db0fa860267a96bc735dde
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java da064e3589951d4f4cbff96d1ed89d9d7e4d3882
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 11835dc21ffd649b04867fcbf14e5003b6d7f7aa
>
>
> Diff: https://reviews.apache.org/r/59039/diff/3/
>
>
> Testing
> -------
>
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
> I've used this in one of our large-scale test clusters to plug in a custom task assigner. We were able to remove our modifications to the OSS code-base and use this mechanism to do what we need. Here is a rough gist of what it looks like:
> https://gist.github.com/DavidMcLaughlin/5d5472cde15343aef8705478f644ddb9
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 59039: Add the ability to customize scheduling
logic.
Posted by David McLaughlin <da...@dmclaughlin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59039/
-----------------------------------------------------------
(Updated May 25, 2017, 5:50 a.m.)
Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
Changes
-------
feedback.
Bugs: AURORA-1920
https://issues.apache.org/jira/browse/AURORA-1920
Repository: aurora
Description
-------
Add the ability to replace the first-fit scheduling algorithm and associated first-fit preemption logic.
See design/proposal document here: https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit?usp=sharing
Diffs (updated)
-----
RELEASE-NOTES.md 77376e438bd7af74c364dcd5d1b3e3f1ece2adbf
docs/development/design-documents.md c942643d6220cb61eb97289b35d92facae06a682
docs/reference/scheduler-configuration.md 3e3d7991970fc75907d07c5a8a31d508c26eda78
src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorModule.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterModule.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java 92087eb60cb81e1c64eaa7d1e2da7f6bbd51500c
src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/state/StateModule.java 0186484208965c5b8942ec76eb078b30190c7d57
src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 7c531afee880022af5db0fa860267a96bc735dde
src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java da064e3589951d4f4cbff96d1ed89d9d7e4d3882
src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 11835dc21ffd649b04867fcbf14e5003b6d7f7aa
Diff: https://reviews.apache.org/r/59039/diff/3/
Changes: https://reviews.apache.org/r/59039/diff/2-3/
Testing
-------
./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
I've used this in one of our large-scale test clusters to plug in a custom task assigner. We were able to remove our modifications to the OSS code-base and use this mechanism to do what we need. Here is a rough gist of what it looks like:
https://gist.github.com/DavidMcLaughlin/5d5472cde15343aef8705478f644ddb9
Thanks,
David McLaughlin
Re: Review Request 59039: Add the ability to customize scheduling
logic.
Posted by David McLaughlin <da...@dmclaughlin.com>.
> On May 12, 2017, 6:37 a.m., Santhosh Kumar Shanmugham wrote:
> > Need to update the configuration documents.
> >
> > https://github.com/apache/aurora/blob/master/docs/reference/scheduler-configuration.md
Done.
- David
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59039/#review174762
-----------------------------------------------------------
On May 25, 2017, 5:50 a.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59039/
> -----------------------------------------------------------
>
> (Updated May 25, 2017, 5:50 a.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Bugs: AURORA-1920
> https://issues.apache.org/jira/browse/AURORA-1920
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Add the ability to replace the first-fit scheduling algorithm and associated first-fit preemption logic.
>
> See design/proposal document here: https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit?usp=sharing
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md 77376e438bd7af74c364dcd5d1b3e3f1ece2adbf
> docs/development/design-documents.md c942643d6220cb61eb97289b35d92facae06a682
> docs/reference/scheduler-configuration.md 3e3d7991970fc75907d07c5a8a31d508c26eda78
> src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java 92087eb60cb81e1c64eaa7d1e2da7f6bbd51500c
> src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/state/StateModule.java 0186484208965c5b8942ec76eb078b30190c7d57
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 7c531afee880022af5db0fa860267a96bc735dde
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java da064e3589951d4f4cbff96d1ed89d9d7e4d3882
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 11835dc21ffd649b04867fcbf14e5003b6d7f7aa
>
>
> Diff: https://reviews.apache.org/r/59039/diff/3/
>
>
> Testing
> -------
>
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
> I've used this in one of our large-scale test clusters to plug in a custom task assigner. We were able to remove our modifications to the OSS code-base and use this mechanism to do what we need. Here is a rough gist of what it looks like:
> https://gist.github.com/DavidMcLaughlin/5d5472cde15343aef8705478f644ddb9
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 59039: Add the ability to customize scheduling
logic.
Posted by Santhosh Kumar Shanmugham <sa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59039/#review174762
-----------------------------------------------------------
Need to update the configuration documents.
https://github.com/apache/aurora/blob/master/docs/reference/scheduler-configuration.md
- Santhosh Kumar Shanmugham
On May 11, 2017, 2:46 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59039/
> -----------------------------------------------------------
>
> (Updated May 11, 2017, 2:46 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Bugs: AURORA-1920
> https://issues.apache.org/jira/browse/AURORA-1920
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Add the ability to replace the first-fit scheduling algorithm and associated first-fit preemption logic.
>
> See design/proposal document here: https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit?usp=sharing
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md 4e930fb1c5b87bca3cf4d8de804d69301f013f07
> docs/development/design-documents.md c942643d6220cb61eb97289b35d92facae06a682
> src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java 66d20290d381c041ac2f28903d6ba1468b352172
> src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/state/StateModule.java 0186484208965c5b8942ec76eb078b30190c7d57
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java da064e3589951d4f4cbff96d1ed89d9d7e4d3882
>
>
> Diff: https://reviews.apache.org/r/59039/diff/2/
>
>
> Testing
> -------
>
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
> I've used this in one of our large-scale test clusters to plug in a custom task assigner. We were able to remove our modifications to the OSS code-base and use this mechanism to do what we need. Here is a rough gist of what it looks like:
> https://gist.github.com/DavidMcLaughlin/5d5472cde15343aef8705478f644ddb9
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 59039: Add the ability to customize scheduling
logic.
Posted by Joshua Cohen <jc...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59039/#review174807
-----------------------------------------------------------
Ship it!
My main concern is the potential for breaking changes, but since it's clearly labeled as alpha, caveat emptor~.
- Joshua Cohen
On May 11, 2017, 9:46 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59039/
> -----------------------------------------------------------
>
> (Updated May 11, 2017, 9:46 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Bugs: AURORA-1920
> https://issues.apache.org/jira/browse/AURORA-1920
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Add the ability to replace the first-fit scheduling algorithm and associated first-fit preemption logic.
>
> See design/proposal document here: https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit?usp=sharing
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md 4e930fb1c5b87bca3cf4d8de804d69301f013f07
> docs/development/design-documents.md c942643d6220cb61eb97289b35d92facae06a682
> src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java 66d20290d381c041ac2f28903d6ba1468b352172
> src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/state/StateModule.java 0186484208965c5b8942ec76eb078b30190c7d57
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java da064e3589951d4f4cbff96d1ed89d9d7e4d3882
>
>
> Diff: https://reviews.apache.org/r/59039/diff/2/
>
>
> Testing
> -------
>
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
> I've used this in one of our large-scale test clusters to plug in a custom task assigner. We were able to remove our modifications to the OSS code-base and use this mechanism to do what we need. Here is a rough gist of what it looks like:
> https://gist.github.com/DavidMcLaughlin/5d5472cde15343aef8705478f644ddb9
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 59039: Add the ability to customize scheduling
logic.
Posted by Stephan Erb <se...@apache.org>.
> On May 12, 2017, 4:20 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java
> > Line 130 (original), 174 (patched)
> > <https://reviews.apache.org/r/59039/diff/2/?file=1716093#file1716093line178>
> >
> > Changes like these and others in this patch could potentially break our benchmarking scripts.
> >
> > Could you please make sure that at least the `SchedulingBenchmarks` continue to run?
> >
> > We already have several broken the benchmarks suites `ThriftApiBenchmarks`, `SnapshotBenchmarks` `StatusUpdateBenchmark` and `ThriftApiBenchmarks`. Would be a pitty if we break even more.
>
> Joshua Cohen wrote:
> Not directly related here, but should we add something that regularly runs the benchmarks and alerts if they stop working (or if there's a major regression in terms of perf)?
Good idea! We might need some tweaking of the benchmark footprint but that should be feasible.
- Stephan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59039/#review174790
-----------------------------------------------------------
On May 11, 2017, 11:46 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59039/
> -----------------------------------------------------------
>
> (Updated May 11, 2017, 11:46 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Bugs: AURORA-1920
> https://issues.apache.org/jira/browse/AURORA-1920
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Add the ability to replace the first-fit scheduling algorithm and associated first-fit preemption logic.
>
> See design/proposal document here: https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit?usp=sharing
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md 4e930fb1c5b87bca3cf4d8de804d69301f013f07
> docs/development/design-documents.md c942643d6220cb61eb97289b35d92facae06a682
> src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java 66d20290d381c041ac2f28903d6ba1468b352172
> src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/state/StateModule.java 0186484208965c5b8942ec76eb078b30190c7d57
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java da064e3589951d4f4cbff96d1ed89d9d7e4d3882
>
>
> Diff: https://reviews.apache.org/r/59039/diff/2/
>
>
> Testing
> -------
>
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
> I've used this in one of our large-scale test clusters to plug in a custom task assigner. We were able to remove our modifications to the OSS code-base and use this mechanism to do what we need. Here is a rough gist of what it looks like:
> https://gist.github.com/DavidMcLaughlin/5d5472cde15343aef8705478f644ddb9
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 59039: Add the ability to customize scheduling
logic.
Posted by David McLaughlin <da...@dmclaughlin.com>.
> On May 12, 2017, 2:20 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java
> > Line 130 (original), 174 (patched)
> > <https://reviews.apache.org/r/59039/diff/2/?file=1716093#file1716093line178>
> >
> > Changes like these and others in this patch could potentially break our benchmarking scripts.
> >
> > Could you please make sure that at least the `SchedulingBenchmarks` continue to run?
> >
> > We already have several broken the benchmarks suites `ThriftApiBenchmarks`, `SnapshotBenchmarks` `StatusUpdateBenchmark` and `ThriftApiBenchmarks`. Would be a pitty if we break even more.
>
> Joshua Cohen wrote:
> Not directly related here, but should we add something that regularly runs the benchmarks and alerts if they stop working (or if there's a major regression in terms of perf)?
>
> Stephan Erb wrote:
> Good idea! We might need some tweaking of the benchmark footprint but that should be feasible.
>
> David McLaughlin wrote:
> Do the SchedulingBenchmarks really pass for you on master? I spent a while trying to debug the broken scheduling benchmarks.. but going back a month of commits they seem broken then too. This issue is making it really difficult to figure out the missing guice bindings: https://github.com/google/guice/issues/757
AFAICT, they've been broken since this commit: https://github.com/apache/aurora/commit/c32f14c75e7f8620aedb12d38aee8ca1d2427724
I'll try and fix them in a separate patch.
- David
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59039/#review174790
-----------------------------------------------------------
On May 11, 2017, 9:46 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59039/
> -----------------------------------------------------------
>
> (Updated May 11, 2017, 9:46 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Bugs: AURORA-1920
> https://issues.apache.org/jira/browse/AURORA-1920
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Add the ability to replace the first-fit scheduling algorithm and associated first-fit preemption logic.
>
> See design/proposal document here: https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit?usp=sharing
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md 4e930fb1c5b87bca3cf4d8de804d69301f013f07
> docs/development/design-documents.md c942643d6220cb61eb97289b35d92facae06a682
> src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java 66d20290d381c041ac2f28903d6ba1468b352172
> src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/state/StateModule.java 0186484208965c5b8942ec76eb078b30190c7d57
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java da064e3589951d4f4cbff96d1ed89d9d7e4d3882
>
>
> Diff: https://reviews.apache.org/r/59039/diff/2/
>
>
> Testing
> -------
>
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
> I've used this in one of our large-scale test clusters to plug in a custom task assigner. We were able to remove our modifications to the OSS code-base and use this mechanism to do what we need. Here is a rough gist of what it looks like:
> https://gist.github.com/DavidMcLaughlin/5d5472cde15343aef8705478f644ddb9
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 59039: Add the ability to customize scheduling
logic.
Posted by David McLaughlin <da...@dmclaughlin.com>.
> On May 12, 2017, 2:20 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java
> > Lines 81 (patched)
> > <https://reviews.apache.org/r/59039/diff/2/?file=1716093#file1716093line81>
> >
> > How does this look like on the commandline? Do I have to provide a fully-specified class name here?
Yup.
> On May 12, 2017, 2:20 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java
> > Line 130 (original), 174 (patched)
> > <https://reviews.apache.org/r/59039/diff/2/?file=1716093#file1716093line178>
> >
> > Changes like these and others in this patch could potentially break our benchmarking scripts.
> >
> > Could you please make sure that at least the `SchedulingBenchmarks` continue to run?
> >
> > We already have several broken the benchmarks suites `ThriftApiBenchmarks`, `SnapshotBenchmarks` `StatusUpdateBenchmark` and `ThriftApiBenchmarks`. Would be a pitty if we break even more.
>
> Joshua Cohen wrote:
> Not directly related here, but should we add something that regularly runs the benchmarks and alerts if they stop working (or if there's a major regression in terms of perf)?
>
> Stephan Erb wrote:
> Good idea! We might need some tweaking of the benchmark footprint but that should be feasible.
>
> David McLaughlin wrote:
> Do the SchedulingBenchmarks really pass for you on master? I spent a while trying to debug the broken scheduling benchmarks.. but going back a month of commits they seem broken then too. This issue is making it really difficult to figure out the missing guice bindings: https://github.com/google/guice/issues/757
>
> David McLaughlin wrote:
> AFAICT, they've been broken since this commit: https://github.com/apache/aurora/commit/c32f14c75e7f8620aedb12d38aee8ca1d2427724
>
> I'll try and fix them in a separate patch.
These are now running.
> On May 12, 2017, 2:20 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java
> > Lines 28 (patched)
> > <https://reviews.apache.org/r/59039/diff/2/?file=1716094#file1716094line28>
> >
> > For consistency how about renaming `TaskAssignerImpl` to `FirstFitTaskAssignerImpl`?
Done.
- David
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59039/#review174790
-----------------------------------------------------------
On May 25, 2017, 5:50 a.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59039/
> -----------------------------------------------------------
>
> (Updated May 25, 2017, 5:50 a.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Bugs: AURORA-1920
> https://issues.apache.org/jira/browse/AURORA-1920
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Add the ability to replace the first-fit scheduling algorithm and associated first-fit preemption logic.
>
> See design/proposal document here: https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit?usp=sharing
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md 77376e438bd7af74c364dcd5d1b3e3f1ece2adbf
> docs/development/design-documents.md c942643d6220cb61eb97289b35d92facae06a682
> docs/reference/scheduler-configuration.md 3e3d7991970fc75907d07c5a8a31d508c26eda78
> src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java 92087eb60cb81e1c64eaa7d1e2da7f6bbd51500c
> src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/state/StateModule.java 0186484208965c5b8942ec76eb078b30190c7d57
> src/main/java/org/apache/aurora/scheduler/state/TaskAssigner.java 7c531afee880022af5db0fa860267a96bc735dde
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java da064e3589951d4f4cbff96d1ed89d9d7e4d3882
> src/test/java/org/apache/aurora/scheduler/state/TaskAssignerImplTest.java 11835dc21ffd649b04867fcbf14e5003b6d7f7aa
>
>
> Diff: https://reviews.apache.org/r/59039/diff/3/
>
>
> Testing
> -------
>
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
> I've used this in one of our large-scale test clusters to plug in a custom task assigner. We were able to remove our modifications to the OSS code-base and use this mechanism to do what we need. Here is a rough gist of what it looks like:
> https://gist.github.com/DavidMcLaughlin/5d5472cde15343aef8705478f644ddb9
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 59039: Add the ability to customize scheduling
logic.
Posted by David McLaughlin <da...@dmclaughlin.com>.
> On May 12, 2017, 2:20 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java
> > Line 130 (original), 174 (patched)
> > <https://reviews.apache.org/r/59039/diff/2/?file=1716093#file1716093line178>
> >
> > Changes like these and others in this patch could potentially break our benchmarking scripts.
> >
> > Could you please make sure that at least the `SchedulingBenchmarks` continue to run?
> >
> > We already have several broken the benchmarks suites `ThriftApiBenchmarks`, `SnapshotBenchmarks` `StatusUpdateBenchmark` and `ThriftApiBenchmarks`. Would be a pitty if we break even more.
>
> Joshua Cohen wrote:
> Not directly related here, but should we add something that regularly runs the benchmarks and alerts if they stop working (or if there's a major regression in terms of perf)?
>
> Stephan Erb wrote:
> Good idea! We might need some tweaking of the benchmark footprint but that should be feasible.
Do the SchedulingBenchmarks really pass for you on master? I spent a while trying to debug the broken scheduling benchmarks.. but going back a month of commits they seem broken then too. This issue is making it really difficult to figure out the missing guice bindings: https://github.com/google/guice/issues/757
- David
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59039/#review174790
-----------------------------------------------------------
On May 11, 2017, 9:46 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59039/
> -----------------------------------------------------------
>
> (Updated May 11, 2017, 9:46 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Bugs: AURORA-1920
> https://issues.apache.org/jira/browse/AURORA-1920
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Add the ability to replace the first-fit scheduling algorithm and associated first-fit preemption logic.
>
> See design/proposal document here: https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit?usp=sharing
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md 4e930fb1c5b87bca3cf4d8de804d69301f013f07
> docs/development/design-documents.md c942643d6220cb61eb97289b35d92facae06a682
> src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java 66d20290d381c041ac2f28903d6ba1468b352172
> src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/state/StateModule.java 0186484208965c5b8942ec76eb078b30190c7d57
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java da064e3589951d4f4cbff96d1ed89d9d7e4d3882
>
>
> Diff: https://reviews.apache.org/r/59039/diff/2/
>
>
> Testing
> -------
>
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
> I've used this in one of our large-scale test clusters to plug in a custom task assigner. We were able to remove our modifications to the OSS code-base and use this mechanism to do what we need. Here is a rough gist of what it looks like:
> https://gist.github.com/DavidMcLaughlin/5d5472cde15343aef8705478f644ddb9
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 59039: Add the ability to customize scheduling
logic.
Posted by Joshua Cohen <jc...@apache.org>.
> On May 12, 2017, 2:20 p.m., Stephan Erb wrote:
> > src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java
> > Line 130 (original), 174 (patched)
> > <https://reviews.apache.org/r/59039/diff/2/?file=1716093#file1716093line178>
> >
> > Changes like these and others in this patch could potentially break our benchmarking scripts.
> >
> > Could you please make sure that at least the `SchedulingBenchmarks` continue to run?
> >
> > We already have several broken the benchmarks suites `ThriftApiBenchmarks`, `SnapshotBenchmarks` `StatusUpdateBenchmark` and `ThriftApiBenchmarks`. Would be a pitty if we break even more.
Not directly related here, but should we add something that regularly runs the benchmarks and alerts if they stop working (or if there's a major regression in terms of perf)?
- Joshua
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59039/#review174790
-----------------------------------------------------------
On May 11, 2017, 9:46 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59039/
> -----------------------------------------------------------
>
> (Updated May 11, 2017, 9:46 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Bugs: AURORA-1920
> https://issues.apache.org/jira/browse/AURORA-1920
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Add the ability to replace the first-fit scheduling algorithm and associated first-fit preemption logic.
>
> See design/proposal document here: https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit?usp=sharing
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md 4e930fb1c5b87bca3cf4d8de804d69301f013f07
> docs/development/design-documents.md c942643d6220cb61eb97289b35d92facae06a682
> src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java 66d20290d381c041ac2f28903d6ba1468b352172
> src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/state/StateModule.java 0186484208965c5b8942ec76eb078b30190c7d57
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java da064e3589951d4f4cbff96d1ed89d9d7e4d3882
>
>
> Diff: https://reviews.apache.org/r/59039/diff/2/
>
>
> Testing
> -------
>
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
> I've used this in one of our large-scale test clusters to plug in a custom task assigner. We were able to remove our modifications to the OSS code-base and use this mechanism to do what we need. Here is a rough gist of what it looks like:
> https://gist.github.com/DavidMcLaughlin/5d5472cde15343aef8705478f644ddb9
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 59039: Add the ability to customize scheduling
logic.
Posted by Stephan Erb <se...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59039/#review174790
-----------------------------------------------------------
Fix it, then Ship it!
The patch looks good. However, my guice knowledge is spotty at best.
src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java
Lines 81 (patched)
<https://reviews.apache.org/r/59039/#comment248015>
How does this look like on the commandline? Do I have to provide a fully-specified class name here?
src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java
Line 130 (original), 174 (patched)
<https://reviews.apache.org/r/59039/#comment248013>
Changes like these and others in this patch could potentially break our benchmarking scripts.
Could you please make sure that at least the `SchedulingBenchmarks` continue to run?
We already have several broken the benchmarks suites `ThriftApiBenchmarks`, `SnapshotBenchmarks` `StatusUpdateBenchmark` and `ThriftApiBenchmarks`. Would be a pitty if we break even more.
src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java
Lines 28 (patched)
<https://reviews.apache.org/r/59039/#comment248012>
For consistency how about renaming `TaskAssignerImpl` to `FirstFitTaskAssignerImpl`?
- Stephan Erb
On May 11, 2017, 11:46 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59039/
> -----------------------------------------------------------
>
> (Updated May 11, 2017, 11:46 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Bugs: AURORA-1920
> https://issues.apache.org/jira/browse/AURORA-1920
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Add the ability to replace the first-fit scheduling algorithm and associated first-fit preemption logic.
>
> See design/proposal document here: https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit?usp=sharing
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md 4e930fb1c5b87bca3cf4d8de804d69301f013f07
> docs/development/design-documents.md c942643d6220cb61eb97289b35d92facae06a682
> src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java 66d20290d381c041ac2f28903d6ba1468b352172
> src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/state/StateModule.java 0186484208965c5b8942ec76eb078b30190c7d57
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java da064e3589951d4f4cbff96d1ed89d9d7e4d3882
>
>
> Diff: https://reviews.apache.org/r/59039/diff/2/
>
>
> Testing
> -------
>
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
> I've used this in one of our large-scale test clusters to plug in a custom task assigner. We were able to remove our modifications to the OSS code-base and use this mechanism to do what we need. Here is a rough gist of what it looks like:
> https://gist.github.com/DavidMcLaughlin/5d5472cde15343aef8705478f644ddb9
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 59039: Add the ability to customize scheduling
logic.
Posted by David McLaughlin <da...@dmclaughlin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59039/
-----------------------------------------------------------
(Updated May 11, 2017, 9:46 p.m.)
Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
Changes
-------
Add mention of feature in release notes. Add doc to design docs.
Bugs: AURORA-1920
https://issues.apache.org/jira/browse/AURORA-1920
Repository: aurora
Description
-------
Add the ability to replace the first-fit scheduling algorithm and associated first-fit preemption logic.
See design/proposal document here: https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit?usp=sharing
Diffs (updated)
-----
RELEASE-NOTES.md 4e930fb1c5b87bca3cf4d8de804d69301f013f07
docs/development/design-documents.md c942643d6220cb61eb97289b35d92facae06a682
src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorModule.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterModule.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java 66d20290d381c041ac2f28903d6ba1468b352172
src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/state/StateModule.java 0186484208965c5b8942ec76eb078b30190c7d57
src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java da064e3589951d4f4cbff96d1ed89d9d7e4d3882
Diff: https://reviews.apache.org/r/59039/diff/2/
Changes: https://reviews.apache.org/r/59039/diff/1-2/
Testing
-------
./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
I've used this in one of our large-scale test clusters to plug in a custom task assigner. We were able to remove our modifications to the OSS code-base and use this mechanism to do what we need. Here is a rough gist of what it looks like:
https://gist.github.com/DavidMcLaughlin/5d5472cde15343aef8705478f644ddb9
Thanks,
David McLaughlin
Re: Review Request 59039: Add the ability to customize scheduling
logic.
Posted by Santhosh Kumar Shanmugham <sa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59039/#review174475
-----------------------------------------------------------
Ship it!
Ship It!
- Santhosh Kumar Shanmugham
On May 9, 2017, 2:42 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59039/
> -----------------------------------------------------------
>
> (Updated May 9, 2017, 2:42 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Bugs: AURORA-1920
> https://issues.apache.org/jira/browse/AURORA-1920
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Add the ability to replace the first-fit scheduling algorithm and associated first-fit preemption logic.
>
> See design/proposal document here: https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit?usp=sharing
>
>
> Diffs
> -----
>
> src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java 66d20290d381c041ac2f28903d6ba1468b352172
> src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java PRE-CREATION
> src/main/java/org/apache/aurora/scheduler/state/StateModule.java 0186484208965c5b8942ec76eb078b30190c7d57
> src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java da064e3589951d4f4cbff96d1ed89d9d7e4d3882
>
>
> Diff: https://reviews.apache.org/r/59039/diff/1/
>
>
> Testing
> -------
>
> ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
> I've used this in one of our large-scale test clusters to plug in a custom task assigner. We were able to remove our modifications to the OSS code-base and use this mechanism to do what we need. Here is a rough gist of what it looks like:
> https://gist.github.com/DavidMcLaughlin/5d5472cde15343aef8705478f644ddb9
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 59039: Add the ability to customize scheduling
logic.
Posted by David McLaughlin <da...@dmclaughlin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59039/
-----------------------------------------------------------
(Updated May 9, 2017, 9:42 p.m.)
Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
Changes
-------
Added code sample of how this can be used.
Bugs: AURORA-1920
https://issues.apache.org/jira/browse/AURORA-1920
Repository: aurora
Description
-------
Add the ability to replace the first-fit scheduling algorithm and associated first-fit preemption logic.
See design/proposal document here: https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit?usp=sharing
Diffs
-----
src/main/java/org/apache/aurora/scheduler/preemptor/PendingTaskProcessorModule.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterModule.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/preemptor/PreemptorModule.java 66d20290d381c041ac2f28903d6ba1468b352172
src/main/java/org/apache/aurora/scheduler/state/FirstFitTaskAssignerModule.java PRE-CREATION
src/main/java/org/apache/aurora/scheduler/state/StateModule.java 0186484208965c5b8942ec76eb078b30190c7d57
src/test/java/org/apache/aurora/scheduler/preemptor/PreemptorModuleTest.java da064e3589951d4f4cbff96d1ed89d9d7e4d3882
Diff: https://reviews.apache.org/r/59039/diff/1/
Testing (updated)
-------
./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
I've used this in one of our large-scale test clusters to plug in a custom task assigner. We were able to remove our modifications to the OSS code-base and use this mechanism to do what we need. Here is a rough gist of what it looks like:
https://gist.github.com/DavidMcLaughlin/5d5472cde15343aef8705478f644ddb9
Thanks,
David McLaughlin