You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by Bill Farner <wf...@apache.org> on 2018/01/04 19:08:26 UTC

Review Request 64954: Refactor scheduling code to split matching and assigning phases

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64954/
-----------------------------------------------------------

Review request for Aurora, Daniel Knightly and Jordan Ly.


Repository: aurora


Description
-------

This patch sets the stage for performing the bulk of scheduling work in a separate call path, without holding the write lock.

Also included is a mechanical refactor pushing the `revocable` flag into `ResourceRequest` (which was ~always needed as a sibling parameter).


Diffs
-----

  commons/src/main/java/org/apache/aurora/common/testing/easymock/EasyMockTest.java c5500b3658437d0a940f953957c880559f5f1b61 
  src/jmh/java/org/apache/aurora/benchmark/fakes/FakeOfferManager.java f0dacd4e90bc142df8b6489ffb33d7265ec03706 
  src/main/java/org/apache/aurora/scheduler/TierManager.java c6ad2b1c48673ca2c14ddd308684d81ce536beca 
  src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 22d5a642687c1ddc91071c27abda7d61726b28b9 
  src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorSettings.java 5c987fd051728486172c8afd34219e86d56f00d5 
  src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java bd415909ac1abdb670fde57ead0e391de3e5238b 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java cb288bb12d577330888c6a66ddd2baaff4b71e8e 
  src/main/java/org/apache/aurora/scheduler/offers/HostOffers.java a01c0a87856a63b1734b4ea336bbbf5fa74b8636 
  src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java e90de3eec43a3df9ff019830e927ecff195d1747 
  src/main/java/org/apache/aurora/scheduler/offers/OfferManagerImpl.java 8e806b705d8bee4381ed8afd792d777420405195 
  src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictim.java 69b686639de1eb7d896096e2fadb357fe9ef3141 
  src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java cf6d348d645d78ab77fcd305307df8a94fea114c 
  src/main/java/org/apache/aurora/scheduler/resources/ResourceManager.java d093753b715ffc9ff37448f0266903d6778af024 
  src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssigner.java 87619b55dae0c49029d4f804a52fdcbf9f16a50a 
  src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImpl.java 54bd1770274dcedecacccba697aee65b8dd48347 
  src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java 3c38f953ed830a2e3627705aa6493e9b37dae155 
  src/main/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImpl.java cff4ab196451fc846827cfc1f48897e645ac3d5d 
  src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 1219215d610e0dbe1f4e78bb8ea57504de0fa329 
  src/main/java/org/apache/aurora/scheduler/storage/durability/WriteRecorder.java dea8e690d8325596c70f31212029ddcfdf8e4451 
  src/main/java/org/apache/aurora/scheduler/updater/UpdateAgentReserver.java b6909a692b7ea590768ecead08b59277b5577866 
  src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 82e40d509d84c37a19b6a9ef942283d908833840 
  src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java 64d7a44eaeb627daf7b7bee75ac0be19594a9b25 
  src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 0de90d70afa1bfa00c93875a325b8a3a7ed7b03d 
  src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java c27a662a2a03ee15098ff66229f86da831177a52 
  src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 1e9532bef39591fcc61a2b41a04f2290960d8906 
  src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java 348386e938ce910c73f1e2e6e2b39f3654a85e7e 
  src/test/java/org/apache/aurora/scheduler/scheduling/FirstFitOfferSelectorTest.java 5e2fdcb646a3362043f171bfc201b83d76524d78 
  src/test/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImplTest.java 78e12691be0bf20e868ad202aaa306ef29bf5971 
  src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java ecf2987bf9fa124a4de3a93ffb20b4a807d79b3e 
  src/test/java/org/apache/aurora/scheduler/updater/NullAgentReserverTest.java d8563b86747a7887be9262a621fd761fea832b8c 
  src/test/java/org/apache/aurora/scheduler/updater/UpdateAgentReserverImplTest.java 7f17be0427e3baf12aaa1d176414f7009baeea94 


Diff: https://reviews.apache.org/r/64954/diff/1/


Testing
-------


Thanks,

Bill Farner


Re: Review Request 64954: Refactor scheduling code to split matching and assigning phases

Posted by Jordan Ly <jo...@gmail.com>.

> On Jan. 4, 2018, 7:42 p.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/TierManager.java
> > Line 105 (original), 106 (patched)
> > <https://reviews.apache.org/r/64954/diff/1/?file=1930809#file1930809line106>
> >
> >     This was a potential bug - `TierManager` relied on the caller to inject the default tier, but would return `null` here if an unknown tier was requested.

Does the above `checkArgument` prevent unknown tiers from ever being gotten?
```
      checkArgument(
          !taskConfig.isSetTier() || tierConfig.tiers.containsKey(taskConfig.getTier()),
          "Invalid tier '%s' in TaskConfig.", taskConfig.getTier());
```
Seems like it should be an `&&` as opposed to an `||` though (and `!` removed). However, not entirely sure what this is checking... If there is no tier, this will still pass.


- Jordan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64954/#review194775
-----------------------------------------------------------


On Jan. 4, 2018, 7:08 p.m., Bill Farner wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64954/
> -----------------------------------------------------------
> 
> (Updated Jan. 4, 2018, 7:08 p.m.)
> 
> 
> Review request for Aurora, Daniel Knightly and Jordan Ly.
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> This patch sets the stage for performing the bulk of scheduling work in a separate call path, without holding the write lock.
> 
> Also included is a mechanical refactor pushing the `revocable` flag into `ResourceRequest` (which was ~always needed as a sibling parameter).
> 
> 
> Diffs
> -----
> 
>   commons/src/main/java/org/apache/aurora/common/testing/easymock/EasyMockTest.java c5500b3658437d0a940f953957c880559f5f1b61 
>   src/jmh/java/org/apache/aurora/benchmark/fakes/FakeOfferManager.java f0dacd4e90bc142df8b6489ffb33d7265ec03706 
>   src/main/java/org/apache/aurora/scheduler/TierManager.java c6ad2b1c48673ca2c14ddd308684d81ce536beca 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 22d5a642687c1ddc91071c27abda7d61726b28b9 
>   src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorSettings.java 5c987fd051728486172c8afd34219e86d56f00d5 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java bd415909ac1abdb670fde57ead0e391de3e5238b 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java cb288bb12d577330888c6a66ddd2baaff4b71e8e 
>   src/main/java/org/apache/aurora/scheduler/offers/HostOffers.java a01c0a87856a63b1734b4ea336bbbf5fa74b8636 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java e90de3eec43a3df9ff019830e927ecff195d1747 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManagerImpl.java 8e806b705d8bee4381ed8afd792d777420405195 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictim.java 69b686639de1eb7d896096e2fadb357fe9ef3141 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java cf6d348d645d78ab77fcd305307df8a94fea114c 
>   src/main/java/org/apache/aurora/scheduler/resources/ResourceManager.java d093753b715ffc9ff37448f0266903d6778af024 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssigner.java 87619b55dae0c49029d4f804a52fdcbf9f16a50a 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImpl.java 54bd1770274dcedecacccba697aee65b8dd48347 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java 3c38f953ed830a2e3627705aa6493e9b37dae155 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImpl.java cff4ab196451fc846827cfc1f48897e645ac3d5d 
>   src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 1219215d610e0dbe1f4e78bb8ea57504de0fa329 
>   src/main/java/org/apache/aurora/scheduler/storage/durability/WriteRecorder.java dea8e690d8325596c70f31212029ddcfdf8e4451 
>   src/main/java/org/apache/aurora/scheduler/updater/UpdateAgentReserver.java b6909a692b7ea590768ecead08b59277b5577866 
>   src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 82e40d509d84c37a19b6a9ef942283d908833840 
>   src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java 64d7a44eaeb627daf7b7bee75ac0be19594a9b25 
>   src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 0de90d70afa1bfa00c93875a325b8a3a7ed7b03d 
>   src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java c27a662a2a03ee15098ff66229f86da831177a52 
>   src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 1e9532bef39591fcc61a2b41a04f2290960d8906 
>   src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java 348386e938ce910c73f1e2e6e2b39f3654a85e7e 
>   src/test/java/org/apache/aurora/scheduler/scheduling/FirstFitOfferSelectorTest.java 5e2fdcb646a3362043f171bfc201b83d76524d78 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImplTest.java 78e12691be0bf20e868ad202aaa306ef29bf5971 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java ecf2987bf9fa124a4de3a93ffb20b4a807d79b3e 
>   src/test/java/org/apache/aurora/scheduler/updater/NullAgentReserverTest.java d8563b86747a7887be9262a621fd761fea832b8c 
>   src/test/java/org/apache/aurora/scheduler/updater/UpdateAgentReserverImplTest.java 7f17be0427e3baf12aaa1d176414f7009baeea94 
> 
> 
> Diff: https://reviews.apache.org/r/64954/diff/1/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Bill Farner
> 
>


Re: Review Request 64954: Refactor scheduling code to split matching and assigning phases

Posted by Bill Farner <wf...@apache.org>.

> On Jan. 4, 2018, 11:42 a.m., Bill Farner wrote:
> > src/main/java/org/apache/aurora/scheduler/TierManager.java
> > Line 105 (original), 106 (patched)
> > <https://reviews.apache.org/r/64954/diff/1/?file=1930809#file1930809line106>
> >
> >     This was a potential bug - `TierManager` relied on the caller to inject the default tier, but would return `null` here if an unknown tier was requested.
> 
> Jordan Ly wrote:
>     Does the above `checkArgument` prevent unknown tiers from ever being gotten?
>     ```
>           checkArgument(
>               !taskConfig.isSetTier() || tierConfig.tiers.containsKey(taskConfig.getTier()),
>               "Invalid tier '%s' in TaskConfig.", taskConfig.getTier());
>     ```
>     Seems like it should be an `&&` as opposed to an `||` though (and `!` removed). However, not entirely sure what this is checking... If there is no tier, this will still pass.

Ah yes, that's it - it allowed an unset tier (`taskConfig.isSetTier()` == false), which would return `null` from this method.


- Bill


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64954/#review194775
-----------------------------------------------------------


On Jan. 4, 2018, 11:08 a.m., Bill Farner wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64954/
> -----------------------------------------------------------
> 
> (Updated Jan. 4, 2018, 11:08 a.m.)
> 
> 
> Review request for Aurora, Daniel Knightly and Jordan Ly.
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> This patch sets the stage for performing the bulk of scheduling work in a separate call path, without holding the write lock.
> 
> Also included is a mechanical refactor pushing the `revocable` flag into `ResourceRequest` (which was ~always needed as a sibling parameter).
> 
> 
> Diffs
> -----
> 
>   commons/src/main/java/org/apache/aurora/common/testing/easymock/EasyMockTest.java c5500b3658437d0a940f953957c880559f5f1b61 
>   src/jmh/java/org/apache/aurora/benchmark/fakes/FakeOfferManager.java f0dacd4e90bc142df8b6489ffb33d7265ec03706 
>   src/main/java/org/apache/aurora/scheduler/TierManager.java c6ad2b1c48673ca2c14ddd308684d81ce536beca 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 22d5a642687c1ddc91071c27abda7d61726b28b9 
>   src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorSettings.java 5c987fd051728486172c8afd34219e86d56f00d5 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java bd415909ac1abdb670fde57ead0e391de3e5238b 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java cb288bb12d577330888c6a66ddd2baaff4b71e8e 
>   src/main/java/org/apache/aurora/scheduler/offers/HostOffers.java a01c0a87856a63b1734b4ea336bbbf5fa74b8636 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java e90de3eec43a3df9ff019830e927ecff195d1747 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManagerImpl.java 8e806b705d8bee4381ed8afd792d777420405195 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictim.java 69b686639de1eb7d896096e2fadb357fe9ef3141 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java cf6d348d645d78ab77fcd305307df8a94fea114c 
>   src/main/java/org/apache/aurora/scheduler/resources/ResourceManager.java d093753b715ffc9ff37448f0266903d6778af024 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssigner.java 87619b55dae0c49029d4f804a52fdcbf9f16a50a 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImpl.java 54bd1770274dcedecacccba697aee65b8dd48347 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java 3c38f953ed830a2e3627705aa6493e9b37dae155 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImpl.java cff4ab196451fc846827cfc1f48897e645ac3d5d 
>   src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 1219215d610e0dbe1f4e78bb8ea57504de0fa329 
>   src/main/java/org/apache/aurora/scheduler/storage/durability/WriteRecorder.java dea8e690d8325596c70f31212029ddcfdf8e4451 
>   src/main/java/org/apache/aurora/scheduler/updater/UpdateAgentReserver.java b6909a692b7ea590768ecead08b59277b5577866 
>   src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 82e40d509d84c37a19b6a9ef942283d908833840 
>   src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java 64d7a44eaeb627daf7b7bee75ac0be19594a9b25 
>   src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 0de90d70afa1bfa00c93875a325b8a3a7ed7b03d 
>   src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java c27a662a2a03ee15098ff66229f86da831177a52 
>   src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 1e9532bef39591fcc61a2b41a04f2290960d8906 
>   src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java 348386e938ce910c73f1e2e6e2b39f3654a85e7e 
>   src/test/java/org/apache/aurora/scheduler/scheduling/FirstFitOfferSelectorTest.java 5e2fdcb646a3362043f171bfc201b83d76524d78 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImplTest.java 78e12691be0bf20e868ad202aaa306ef29bf5971 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java ecf2987bf9fa124a4de3a93ffb20b4a807d79b3e 
>   src/test/java/org/apache/aurora/scheduler/updater/NullAgentReserverTest.java d8563b86747a7887be9262a621fd761fea832b8c 
>   src/test/java/org/apache/aurora/scheduler/updater/UpdateAgentReserverImplTest.java 7f17be0427e3baf12aaa1d176414f7009baeea94 
> 
> 
> Diff: https://reviews.apache.org/r/64954/diff/1/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Bill Farner
> 
>


Re: Review Request 64954: Refactor scheduling code to split matching and assigning phases

Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64954/#review194775
-----------------------------------------------------------



Reviewer notes


commons/src/main/java/org/apache/aurora/common/testing/easymock/EasyMockTest.java
Lines 48-49 (patched)
<https://reviews.apache.org/r/64954/#comment273754>

    Unnecessary for the scheduling refactor, but scratching an itch.



src/main/java/org/apache/aurora/scheduler/TierManager.java
Line 105 (original), 106 (patched)
<https://reviews.apache.org/r/64954/#comment273755>

    This was a potential bug - `TierManager` relied on the caller to inject the default tier, but would return `null` here if an unknown tier was requested.



src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorSettings.java
Lines 55-58 (patched)
<https://reviews.apache.org/r/64954/#comment273756>

    This logic was previously duplicated.



src/main/java/org/apache/aurora/scheduler/offers/HostOffers.java
Lines 161 (patched)
<https://reviews.apache.org/r/64954/#comment273757>

    No logic change, just simplified.



src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImpl.java
Lines 153-156 (original), 145-148 (patched)
<https://reviews.apache.org/r/64954/#comment273758>

    Captures the tri-state logic to represent reservations.  I believe the factory and accessor methods sufficiently self-document.



src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImpl.java
Lines 175 (patched)
<https://reviews.apache.org/r/64954/#comment273761>

    ReviewBoard's diff algorithm is a bit aggressive here, making it difficult to comprehend the changes as rendered.
    
    Just wanted to call out that this is split into a helper method.



src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImpl.java
Line 240 (original), 225 (patched)
<https://reviews.apache.org/r/64954/#comment273762>

    This is the primary objective of the change - we now have an isolated body of code that collects scheduling matches.



src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImpl.java
Lines 244-250 (original)
<https://reviews.apache.org/r/64954/#comment273763>

    Rather than "first try to match all tasks against reservations", the patch treats a reservation as an alternate way to acquire a `SchedulingMatch`.



src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java
Line 34 (original), 34 (patched)
<https://reviews.apache.org/r/64954/#comment273764>

    `Iterable` type forced an unnecessary `ImmutableSet.copyOf()` in the implementation.  No good reason to keep `Iterable`.



src/main/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImpl.java
Lines 132-138 (original)
<https://reviews.apache.org/r/64954/#comment273765>

    This check failure becomes a failure in `Iterables.getOnlyElement()` on :147.



src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java
Line 50 (original), 51 (patched)
<https://reviews.apache.org/r/64954/#comment273767>

    Allow extra hurdles to use `size()`, `isEmpty()`, and `stream()`.



src/main/java/org/apache/aurora/scheduler/updater/UpdateAgentReserver.java
Line 67 (original), 64 (patched)
<https://reviews.apache.org/r/64954/#comment273768>

    Slimmed this interface, the extra return detail was not used in practice.



src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java
Line 122 (original)
<https://reviews.apache.org/r/64954/#comment273770>

    Simplify with mock -> fake.  I can't think of a good reason to mock `TierManager`, as it's just a map with a default.


- Bill Farner


On Jan. 4, 2018, 11:08 a.m., Bill Farner wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64954/
> -----------------------------------------------------------
> 
> (Updated Jan. 4, 2018, 11:08 a.m.)
> 
> 
> Review request for Aurora, Daniel Knightly and Jordan Ly.
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> This patch sets the stage for performing the bulk of scheduling work in a separate call path, without holding the write lock.
> 
> Also included is a mechanical refactor pushing the `revocable` flag into `ResourceRequest` (which was ~always needed as a sibling parameter).
> 
> 
> Diffs
> -----
> 
>   commons/src/main/java/org/apache/aurora/common/testing/easymock/EasyMockTest.java c5500b3658437d0a940f953957c880559f5f1b61 
>   src/jmh/java/org/apache/aurora/benchmark/fakes/FakeOfferManager.java f0dacd4e90bc142df8b6489ffb33d7265ec03706 
>   src/main/java/org/apache/aurora/scheduler/TierManager.java c6ad2b1c48673ca2c14ddd308684d81ce536beca 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 22d5a642687c1ddc91071c27abda7d61726b28b9 
>   src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorSettings.java 5c987fd051728486172c8afd34219e86d56f00d5 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java bd415909ac1abdb670fde57ead0e391de3e5238b 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java cb288bb12d577330888c6a66ddd2baaff4b71e8e 
>   src/main/java/org/apache/aurora/scheduler/offers/HostOffers.java a01c0a87856a63b1734b4ea336bbbf5fa74b8636 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java e90de3eec43a3df9ff019830e927ecff195d1747 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManagerImpl.java 8e806b705d8bee4381ed8afd792d777420405195 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictim.java 69b686639de1eb7d896096e2fadb357fe9ef3141 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java cf6d348d645d78ab77fcd305307df8a94fea114c 
>   src/main/java/org/apache/aurora/scheduler/resources/ResourceManager.java d093753b715ffc9ff37448f0266903d6778af024 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssigner.java 87619b55dae0c49029d4f804a52fdcbf9f16a50a 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImpl.java 54bd1770274dcedecacccba697aee65b8dd48347 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java 3c38f953ed830a2e3627705aa6493e9b37dae155 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImpl.java cff4ab196451fc846827cfc1f48897e645ac3d5d 
>   src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 1219215d610e0dbe1f4e78bb8ea57504de0fa329 
>   src/main/java/org/apache/aurora/scheduler/storage/durability/WriteRecorder.java dea8e690d8325596c70f31212029ddcfdf8e4451 
>   src/main/java/org/apache/aurora/scheduler/updater/UpdateAgentReserver.java b6909a692b7ea590768ecead08b59277b5577866 
>   src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 82e40d509d84c37a19b6a9ef942283d908833840 
>   src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java 64d7a44eaeb627daf7b7bee75ac0be19594a9b25 
>   src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 0de90d70afa1bfa00c93875a325b8a3a7ed7b03d 
>   src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java c27a662a2a03ee15098ff66229f86da831177a52 
>   src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 1e9532bef39591fcc61a2b41a04f2290960d8906 
>   src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java 348386e938ce910c73f1e2e6e2b39f3654a85e7e 
>   src/test/java/org/apache/aurora/scheduler/scheduling/FirstFitOfferSelectorTest.java 5e2fdcb646a3362043f171bfc201b83d76524d78 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImplTest.java 78e12691be0bf20e868ad202aaa306ef29bf5971 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java ecf2987bf9fa124a4de3a93ffb20b4a807d79b3e 
>   src/test/java/org/apache/aurora/scheduler/updater/NullAgentReserverTest.java d8563b86747a7887be9262a621fd761fea832b8c 
>   src/test/java/org/apache/aurora/scheduler/updater/UpdateAgentReserverImplTest.java 7f17be0427e3baf12aaa1d176414f7009baeea94 
> 
> 
> Diff: https://reviews.apache.org/r/64954/diff/1/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Bill Farner
> 
>


Re: Review Request 64954: Refactor scheduling code to split matching and assigning phases

Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64954/#review194779
-----------------------------------------------------------


Ship it!




Master (8e5e08e) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot retry"

- Aurora ReviewBot


On Jan. 4, 2018, 9:08 p.m., Bill Farner wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64954/
> -----------------------------------------------------------
> 
> (Updated Jan. 4, 2018, 9:08 p.m.)
> 
> 
> Review request for Aurora, Daniel Knightly and Jordan Ly.
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> This patch sets the stage for performing the bulk of scheduling work in a separate call path, without holding the write lock.
> 
> Also included is a mechanical refactor pushing the `revocable` flag into `ResourceRequest` (which was ~always needed as a sibling parameter).
> 
> 
> Diffs
> -----
> 
>   commons/src/main/java/org/apache/aurora/common/testing/easymock/EasyMockTest.java c5500b3658437d0a940f953957c880559f5f1b61 
>   src/jmh/java/org/apache/aurora/benchmark/fakes/FakeOfferManager.java f0dacd4e90bc142df8b6489ffb33d7265ec03706 
>   src/main/java/org/apache/aurora/scheduler/TierManager.java c6ad2b1c48673ca2c14ddd308684d81ce536beca 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 22d5a642687c1ddc91071c27abda7d61726b28b9 
>   src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorSettings.java 5c987fd051728486172c8afd34219e86d56f00d5 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java bd415909ac1abdb670fde57ead0e391de3e5238b 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java cb288bb12d577330888c6a66ddd2baaff4b71e8e 
>   src/main/java/org/apache/aurora/scheduler/offers/HostOffers.java a01c0a87856a63b1734b4ea336bbbf5fa74b8636 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java e90de3eec43a3df9ff019830e927ecff195d1747 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManagerImpl.java 8e806b705d8bee4381ed8afd792d777420405195 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictim.java 69b686639de1eb7d896096e2fadb357fe9ef3141 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java cf6d348d645d78ab77fcd305307df8a94fea114c 
>   src/main/java/org/apache/aurora/scheduler/resources/ResourceManager.java d093753b715ffc9ff37448f0266903d6778af024 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssigner.java 87619b55dae0c49029d4f804a52fdcbf9f16a50a 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImpl.java 54bd1770274dcedecacccba697aee65b8dd48347 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java 3c38f953ed830a2e3627705aa6493e9b37dae155 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImpl.java cff4ab196451fc846827cfc1f48897e645ac3d5d 
>   src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 1219215d610e0dbe1f4e78bb8ea57504de0fa329 
>   src/main/java/org/apache/aurora/scheduler/storage/durability/WriteRecorder.java dea8e690d8325596c70f31212029ddcfdf8e4451 
>   src/main/java/org/apache/aurora/scheduler/updater/UpdateAgentReserver.java b6909a692b7ea590768ecead08b59277b5577866 
>   src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 82e40d509d84c37a19b6a9ef942283d908833840 
>   src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java 64d7a44eaeb627daf7b7bee75ac0be19594a9b25 
>   src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 0de90d70afa1bfa00c93875a325b8a3a7ed7b03d 
>   src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java c27a662a2a03ee15098ff66229f86da831177a52 
>   src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 1e9532bef39591fcc61a2b41a04f2290960d8906 
>   src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java 348386e938ce910c73f1e2e6e2b39f3654a85e7e 
>   src/test/java/org/apache/aurora/scheduler/scheduling/FirstFitOfferSelectorTest.java 5e2fdcb646a3362043f171bfc201b83d76524d78 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImplTest.java 78e12691be0bf20e868ad202aaa306ef29bf5971 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java ecf2987bf9fa124a4de3a93ffb20b4a807d79b3e 
>   src/test/java/org/apache/aurora/scheduler/updater/NullAgentReserverTest.java d8563b86747a7887be9262a621fd761fea832b8c 
>   src/test/java/org/apache/aurora/scheduler/updater/UpdateAgentReserverImplTest.java 7f17be0427e3baf12aaa1d176414f7009baeea94 
> 
> 
> Diff: https://reviews.apache.org/r/64954/diff/1/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Bill Farner
> 
>


Re: Review Request 64954: Refactor scheduling code to split matching and assigning phases

Posted by Bill Farner <wf...@apache.org>.

> On Jan. 4, 2018, 4:46 p.m., Jordan Ly wrote:
> > src/main/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImpl.java
> > Lines 152-158 (original), 141-144 (patched)
> > <https://reviews.apache.org/r/64954/diff/1/?file=1930823#file1930823line159>
> >
> >     Is this check effectively the same as the check in `fetchTasks`? Unless `ids` can be empty...

The checks do overlap in the case where none of the tasks are found.  With this refactor, this exceptional case will result in two `LOG.warn` lines while previously there was only one.  I've discarded the redundant `Failed to look up all...` log to remedy, but the check is useful to short-circuit and simplify the rest of the function.


- Bill


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64954/#review194810
-----------------------------------------------------------


On Jan. 4, 2018, 11:08 a.m., Bill Farner wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64954/
> -----------------------------------------------------------
> 
> (Updated Jan. 4, 2018, 11:08 a.m.)
> 
> 
> Review request for Aurora, Daniel Knightly and Jordan Ly.
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> This patch sets the stage for performing the bulk of scheduling work in a separate call path, without holding the write lock.
> 
> Also included is a mechanical refactor pushing the `revocable` flag into `ResourceRequest` (which was ~always needed as a sibling parameter).
> 
> 
> Diffs
> -----
> 
>   commons/src/main/java/org/apache/aurora/common/testing/easymock/EasyMockTest.java c5500b3658437d0a940f953957c880559f5f1b61 
>   src/jmh/java/org/apache/aurora/benchmark/fakes/FakeOfferManager.java f0dacd4e90bc142df8b6489ffb33d7265ec03706 
>   src/main/java/org/apache/aurora/scheduler/TierManager.java c6ad2b1c48673ca2c14ddd308684d81ce536beca 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 22d5a642687c1ddc91071c27abda7d61726b28b9 
>   src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorSettings.java 5c987fd051728486172c8afd34219e86d56f00d5 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java bd415909ac1abdb670fde57ead0e391de3e5238b 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java cb288bb12d577330888c6a66ddd2baaff4b71e8e 
>   src/main/java/org/apache/aurora/scheduler/offers/HostOffers.java a01c0a87856a63b1734b4ea336bbbf5fa74b8636 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java e90de3eec43a3df9ff019830e927ecff195d1747 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManagerImpl.java 8e806b705d8bee4381ed8afd792d777420405195 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictim.java 69b686639de1eb7d896096e2fadb357fe9ef3141 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java cf6d348d645d78ab77fcd305307df8a94fea114c 
>   src/main/java/org/apache/aurora/scheduler/resources/ResourceManager.java d093753b715ffc9ff37448f0266903d6778af024 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssigner.java 87619b55dae0c49029d4f804a52fdcbf9f16a50a 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImpl.java 54bd1770274dcedecacccba697aee65b8dd48347 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java 3c38f953ed830a2e3627705aa6493e9b37dae155 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImpl.java cff4ab196451fc846827cfc1f48897e645ac3d5d 
>   src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 1219215d610e0dbe1f4e78bb8ea57504de0fa329 
>   src/main/java/org/apache/aurora/scheduler/storage/durability/WriteRecorder.java dea8e690d8325596c70f31212029ddcfdf8e4451 
>   src/main/java/org/apache/aurora/scheduler/updater/UpdateAgentReserver.java b6909a692b7ea590768ecead08b59277b5577866 
>   src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 82e40d509d84c37a19b6a9ef942283d908833840 
>   src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java 64d7a44eaeb627daf7b7bee75ac0be19594a9b25 
>   src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 0de90d70afa1bfa00c93875a325b8a3a7ed7b03d 
>   src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java c27a662a2a03ee15098ff66229f86da831177a52 
>   src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 1e9532bef39591fcc61a2b41a04f2290960d8906 
>   src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java 348386e938ce910c73f1e2e6e2b39f3654a85e7e 
>   src/test/java/org/apache/aurora/scheduler/scheduling/FirstFitOfferSelectorTest.java 5e2fdcb646a3362043f171bfc201b83d76524d78 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImplTest.java 78e12691be0bf20e868ad202aaa306ef29bf5971 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java ecf2987bf9fa124a4de3a93ffb20b4a807d79b3e 
>   src/test/java/org/apache/aurora/scheduler/updater/NullAgentReserverTest.java d8563b86747a7887be9262a621fd761fea832b8c 
>   src/test/java/org/apache/aurora/scheduler/updater/UpdateAgentReserverImplTest.java 7f17be0427e3baf12aaa1d176414f7009baeea94 
> 
> 
> Diff: https://reviews.apache.org/r/64954/diff/1/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Bill Farner
> 
>


Re: Review Request 64954: Refactor scheduling code to split matching and assigning phases

Posted by Bill Farner <wf...@apache.org>.

> On Jan. 4, 2018, 4:46 p.m., Jordan Ly wrote:
> > Nice patch! Do you have any benchmarks comparing this refactor to the current master?

Scheduling benchmarks suggest that this does not affect performance.  While i expected as much, it's good to see there are no surprises!

master:
```
Benchmark                                                                      Mode  Cnt       Score      Error  Units
SchedulingBenchmarks.ClusterFullUtilizationBenchmark.runBenchmark             thrpt   10  152290.606 ± 5044.112  ops/s
SchedulingBenchmarks.FillClusterBenchmark.runBenchmark                        thrpt   10    1238.857 ± 1329.612  ops/s
SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark    thrpt   10   11388.429 ±   79.748  ops/s
SchedulingBenchmarks.LimitConstraintMismatchSchedulingBenchmark.runBenchmark  thrpt   10    1196.417 ±   37.682  ops/s
SchedulingBenchmarks.ValueConstraintMismatchSchedulingBenchmark.runBenchmark  thrpt   10   11557.389 ±  397.894  ops/s
```

this patch:
```
Benchmark                                                                      Mode  Cnt       Score      Error  Units
SchedulingBenchmarks.ClusterFullUtilizationBenchmark.runBenchmark             thrpt   10  173674.535 ± 7602.886  ops/s
SchedulingBenchmarks.FillClusterBenchmark.runBenchmark                        thrpt   10    1398.264 ± 1424.987  ops/s
SchedulingBenchmarks.InsufficientResourcesSchedulingBenchmark.runBenchmark    thrpt   10   11635.548 ±  203.560  ops/s
SchedulingBenchmarks.LimitConstraintMismatchSchedulingBenchmark.runBenchmark  thrpt   10    1235.561 ±   57.993  ops/s
SchedulingBenchmarks.ValueConstraintMismatchSchedulingBenchmark.runBenchmark  thrpt   10   11467.795 ±  269.553  ops/s
```


- Bill


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64954/#review194810
-----------------------------------------------------------


On Jan. 4, 2018, 11:08 a.m., Bill Farner wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64954/
> -----------------------------------------------------------
> 
> (Updated Jan. 4, 2018, 11:08 a.m.)
> 
> 
> Review request for Aurora, Daniel Knightly and Jordan Ly.
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> This patch sets the stage for performing the bulk of scheduling work in a separate call path, without holding the write lock.
> 
> Also included is a mechanical refactor pushing the `revocable` flag into `ResourceRequest` (which was ~always needed as a sibling parameter).
> 
> 
> Diffs
> -----
> 
>   commons/src/main/java/org/apache/aurora/common/testing/easymock/EasyMockTest.java c5500b3658437d0a940f953957c880559f5f1b61 
>   src/jmh/java/org/apache/aurora/benchmark/fakes/FakeOfferManager.java f0dacd4e90bc142df8b6489ffb33d7265ec03706 
>   src/main/java/org/apache/aurora/scheduler/TierManager.java c6ad2b1c48673ca2c14ddd308684d81ce536beca 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 22d5a642687c1ddc91071c27abda7d61726b28b9 
>   src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorSettings.java 5c987fd051728486172c8afd34219e86d56f00d5 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java bd415909ac1abdb670fde57ead0e391de3e5238b 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java cb288bb12d577330888c6a66ddd2baaff4b71e8e 
>   src/main/java/org/apache/aurora/scheduler/offers/HostOffers.java a01c0a87856a63b1734b4ea336bbbf5fa74b8636 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java e90de3eec43a3df9ff019830e927ecff195d1747 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManagerImpl.java 8e806b705d8bee4381ed8afd792d777420405195 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictim.java 69b686639de1eb7d896096e2fadb357fe9ef3141 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java cf6d348d645d78ab77fcd305307df8a94fea114c 
>   src/main/java/org/apache/aurora/scheduler/resources/ResourceManager.java d093753b715ffc9ff37448f0266903d6778af024 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssigner.java 87619b55dae0c49029d4f804a52fdcbf9f16a50a 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImpl.java 54bd1770274dcedecacccba697aee65b8dd48347 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java 3c38f953ed830a2e3627705aa6493e9b37dae155 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImpl.java cff4ab196451fc846827cfc1f48897e645ac3d5d 
>   src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 1219215d610e0dbe1f4e78bb8ea57504de0fa329 
>   src/main/java/org/apache/aurora/scheduler/storage/durability/WriteRecorder.java dea8e690d8325596c70f31212029ddcfdf8e4451 
>   src/main/java/org/apache/aurora/scheduler/updater/UpdateAgentReserver.java b6909a692b7ea590768ecead08b59277b5577866 
>   src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 82e40d509d84c37a19b6a9ef942283d908833840 
>   src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java 64d7a44eaeb627daf7b7bee75ac0be19594a9b25 
>   src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 0de90d70afa1bfa00c93875a325b8a3a7ed7b03d 
>   src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java c27a662a2a03ee15098ff66229f86da831177a52 
>   src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 1e9532bef39591fcc61a2b41a04f2290960d8906 
>   src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java 348386e938ce910c73f1e2e6e2b39f3654a85e7e 
>   src/test/java/org/apache/aurora/scheduler/scheduling/FirstFitOfferSelectorTest.java 5e2fdcb646a3362043f171bfc201b83d76524d78 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImplTest.java 78e12691be0bf20e868ad202aaa306ef29bf5971 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java ecf2987bf9fa124a4de3a93ffb20b4a807d79b3e 
>   src/test/java/org/apache/aurora/scheduler/updater/NullAgentReserverTest.java d8563b86747a7887be9262a621fd761fea832b8c 
>   src/test/java/org/apache/aurora/scheduler/updater/UpdateAgentReserverImplTest.java 7f17be0427e3baf12aaa1d176414f7009baeea94 
> 
> 
> Diff: https://reviews.apache.org/r/64954/diff/1/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Bill Farner
> 
>


Re: Review Request 64954: Refactor scheduling code to split matching and assigning phases

Posted by Jordan Ly <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64954/#review194810
-----------------------------------------------------------



Nice patch! Do you have any benchmarks comparing this refactor to the current master?


src/main/java/org/apache/aurora/scheduler/offers/HostOffers.java
Lines 168-170 (original), 161-168 (patched)
<https://reviews.apache.org/r/64954/#comment273868>

    I would use `Optional.filter` here.



src/main/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImpl.java
Lines 152-158 (original), 141-144 (patched)
<https://reviews.apache.org/r/64954/#comment273872>

    Is this check effectively the same as the check in `fetchTasks`? Unless `ids` can be empty...


- Jordan Ly


On Jan. 4, 2018, 7:08 p.m., Bill Farner wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64954/
> -----------------------------------------------------------
> 
> (Updated Jan. 4, 2018, 7:08 p.m.)
> 
> 
> Review request for Aurora, Daniel Knightly and Jordan Ly.
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> This patch sets the stage for performing the bulk of scheduling work in a separate call path, without holding the write lock.
> 
> Also included is a mechanical refactor pushing the `revocable` flag into `ResourceRequest` (which was ~always needed as a sibling parameter).
> 
> 
> Diffs
> -----
> 
>   commons/src/main/java/org/apache/aurora/common/testing/easymock/EasyMockTest.java c5500b3658437d0a940f953957c880559f5f1b61 
>   src/jmh/java/org/apache/aurora/benchmark/fakes/FakeOfferManager.java f0dacd4e90bc142df8b6489ffb33d7265ec03706 
>   src/main/java/org/apache/aurora/scheduler/TierManager.java c6ad2b1c48673ca2c14ddd308684d81ce536beca 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 22d5a642687c1ddc91071c27abda7d61726b28b9 
>   src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorSettings.java 5c987fd051728486172c8afd34219e86d56f00d5 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java bd415909ac1abdb670fde57ead0e391de3e5238b 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java cb288bb12d577330888c6a66ddd2baaff4b71e8e 
>   src/main/java/org/apache/aurora/scheduler/offers/HostOffers.java a01c0a87856a63b1734b4ea336bbbf5fa74b8636 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java e90de3eec43a3df9ff019830e927ecff195d1747 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManagerImpl.java 8e806b705d8bee4381ed8afd792d777420405195 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictim.java 69b686639de1eb7d896096e2fadb357fe9ef3141 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java cf6d348d645d78ab77fcd305307df8a94fea114c 
>   src/main/java/org/apache/aurora/scheduler/resources/ResourceManager.java d093753b715ffc9ff37448f0266903d6778af024 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssigner.java 87619b55dae0c49029d4f804a52fdcbf9f16a50a 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImpl.java 54bd1770274dcedecacccba697aee65b8dd48347 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java 3c38f953ed830a2e3627705aa6493e9b37dae155 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImpl.java cff4ab196451fc846827cfc1f48897e645ac3d5d 
>   src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 1219215d610e0dbe1f4e78bb8ea57504de0fa329 
>   src/main/java/org/apache/aurora/scheduler/storage/durability/WriteRecorder.java dea8e690d8325596c70f31212029ddcfdf8e4451 
>   src/main/java/org/apache/aurora/scheduler/updater/UpdateAgentReserver.java b6909a692b7ea590768ecead08b59277b5577866 
>   src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 82e40d509d84c37a19b6a9ef942283d908833840 
>   src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java 64d7a44eaeb627daf7b7bee75ac0be19594a9b25 
>   src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 0de90d70afa1bfa00c93875a325b8a3a7ed7b03d 
>   src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java c27a662a2a03ee15098ff66229f86da831177a52 
>   src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 1e9532bef39591fcc61a2b41a04f2290960d8906 
>   src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java 348386e938ce910c73f1e2e6e2b39f3654a85e7e 
>   src/test/java/org/apache/aurora/scheduler/scheduling/FirstFitOfferSelectorTest.java 5e2fdcb646a3362043f171bfc201b83d76524d78 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImplTest.java 78e12691be0bf20e868ad202aaa306ef29bf5971 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java ecf2987bf9fa124a4de3a93ffb20b4a807d79b3e 
>   src/test/java/org/apache/aurora/scheduler/updater/NullAgentReserverTest.java d8563b86747a7887be9262a621fd761fea832b8c 
>   src/test/java/org/apache/aurora/scheduler/updater/UpdateAgentReserverImplTest.java 7f17be0427e3baf12aaa1d176414f7009baeea94 
> 
> 
> Diff: https://reviews.apache.org/r/64954/diff/1/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Bill Farner
> 
>


Re: Review Request 64954: Refactor scheduling code to split matching and assigning phases

Posted by Bill Farner <wf...@apache.org>.

> On Jan. 9, 2018, 10:44 a.m., Jordan Ly wrote:
> > src/test/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImplTest.java
> > Lines 369-372 (original), 356-358 (patched)
> > <https://reviews.apache.org/r/64954/diff/1/?file=1930834#file1930834line378>
> >
> >     nit: can you just expect 0

Unfortunately that is disallowed:
```
java.lang.IllegalArgumentException: maximum must be >= 1

	at org.easymock.internal.MocksControl.times(MocksControl.java:345)
```


- Bill


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64954/#review195054
-----------------------------------------------------------


On Jan. 9, 2018, 10:32 a.m., Bill Farner wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64954/
> -----------------------------------------------------------
> 
> (Updated Jan. 9, 2018, 10:32 a.m.)
> 
> 
> Review request for Aurora, Daniel Knightly and Jordan Ly.
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> This patch sets the stage for performing the bulk of scheduling work in a separate call path, without holding the write lock.
> 
> Also included is a mechanical refactor pushing the `revocable` flag into `ResourceRequest` (which was ~always needed as a sibling parameter).
> 
> 
> Diffs
> -----
> 
>   commons/src/main/java/org/apache/aurora/common/testing/easymock/EasyMockTest.java c5500b3658437d0a940f953957c880559f5f1b61 
>   src/jmh/java/org/apache/aurora/benchmark/fakes/FakeOfferManager.java f0dacd4e90bc142df8b6489ffb33d7265ec03706 
>   src/main/java/org/apache/aurora/scheduler/TierManager.java c6ad2b1c48673ca2c14ddd308684d81ce536beca 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 22d5a642687c1ddc91071c27abda7d61726b28b9 
>   src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorSettings.java 5c987fd051728486172c8afd34219e86d56f00d5 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java bd415909ac1abdb670fde57ead0e391de3e5238b 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java cb288bb12d577330888c6a66ddd2baaff4b71e8e 
>   src/main/java/org/apache/aurora/scheduler/offers/HostOffers.java a01c0a87856a63b1734b4ea336bbbf5fa74b8636 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java e90de3eec43a3df9ff019830e927ecff195d1747 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManagerImpl.java 8e806b705d8bee4381ed8afd792d777420405195 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictim.java 69b686639de1eb7d896096e2fadb357fe9ef3141 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java cf6d348d645d78ab77fcd305307df8a94fea114c 
>   src/main/java/org/apache/aurora/scheduler/resources/ResourceManager.java d093753b715ffc9ff37448f0266903d6778af024 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssigner.java 87619b55dae0c49029d4f804a52fdcbf9f16a50a 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImpl.java 54bd1770274dcedecacccba697aee65b8dd48347 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java 3c38f953ed830a2e3627705aa6493e9b37dae155 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImpl.java cff4ab196451fc846827cfc1f48897e645ac3d5d 
>   src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 1219215d610e0dbe1f4e78bb8ea57504de0fa329 
>   src/main/java/org/apache/aurora/scheduler/storage/durability/WriteRecorder.java dea8e690d8325596c70f31212029ddcfdf8e4451 
>   src/main/java/org/apache/aurora/scheduler/updater/UpdateAgentReserver.java b6909a692b7ea590768ecead08b59277b5577866 
>   src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 82e40d509d84c37a19b6a9ef942283d908833840 
>   src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java 64d7a44eaeb627daf7b7bee75ac0be19594a9b25 
>   src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 0de90d70afa1bfa00c93875a325b8a3a7ed7b03d 
>   src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java c27a662a2a03ee15098ff66229f86da831177a52 
>   src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 1e9532bef39591fcc61a2b41a04f2290960d8906 
>   src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java 348386e938ce910c73f1e2e6e2b39f3654a85e7e 
>   src/test/java/org/apache/aurora/scheduler/scheduling/FirstFitOfferSelectorTest.java 5e2fdcb646a3362043f171bfc201b83d76524d78 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImplTest.java 78e12691be0bf20e868ad202aaa306ef29bf5971 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java ecf2987bf9fa124a4de3a93ffb20b4a807d79b3e 
>   src/test/java/org/apache/aurora/scheduler/updater/NullAgentReserverTest.java d8563b86747a7887be9262a621fd761fea832b8c 
>   src/test/java/org/apache/aurora/scheduler/updater/UpdateAgentReserverImplTest.java 7f17be0427e3baf12aaa1d176414f7009baeea94 
> 
> 
> Diff: https://reviews.apache.org/r/64954/diff/2/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Bill Farner
> 
>


Re: Review Request 64954: Refactor scheduling code to split matching and assigning phases

Posted by Jordan Ly <jo...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64954/#review195054
-----------------------------------------------------------


Ship it!




Thanks for the benchmarks! LGTM


src/test/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImplTest.java
Lines 369-372 (original), 356-358 (patched)
<https://reviews.apache.org/r/64954/#comment274180>

    nit: can you just expect 0


- Jordan Ly


On Jan. 9, 2018, 6:32 p.m., Bill Farner wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64954/
> -----------------------------------------------------------
> 
> (Updated Jan. 9, 2018, 6:32 p.m.)
> 
> 
> Review request for Aurora, Daniel Knightly and Jordan Ly.
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> This patch sets the stage for performing the bulk of scheduling work in a separate call path, without holding the write lock.
> 
> Also included is a mechanical refactor pushing the `revocable` flag into `ResourceRequest` (which was ~always needed as a sibling parameter).
> 
> 
> Diffs
> -----
> 
>   commons/src/main/java/org/apache/aurora/common/testing/easymock/EasyMockTest.java c5500b3658437d0a940f953957c880559f5f1b61 
>   src/jmh/java/org/apache/aurora/benchmark/fakes/FakeOfferManager.java f0dacd4e90bc142df8b6489ffb33d7265ec03706 
>   src/main/java/org/apache/aurora/scheduler/TierManager.java c6ad2b1c48673ca2c14ddd308684d81ce536beca 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 22d5a642687c1ddc91071c27abda7d61726b28b9 
>   src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorSettings.java 5c987fd051728486172c8afd34219e86d56f00d5 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java bd415909ac1abdb670fde57ead0e391de3e5238b 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java cb288bb12d577330888c6a66ddd2baaff4b71e8e 
>   src/main/java/org/apache/aurora/scheduler/offers/HostOffers.java a01c0a87856a63b1734b4ea336bbbf5fa74b8636 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java e90de3eec43a3df9ff019830e927ecff195d1747 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManagerImpl.java 8e806b705d8bee4381ed8afd792d777420405195 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictim.java 69b686639de1eb7d896096e2fadb357fe9ef3141 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java cf6d348d645d78ab77fcd305307df8a94fea114c 
>   src/main/java/org/apache/aurora/scheduler/resources/ResourceManager.java d093753b715ffc9ff37448f0266903d6778af024 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssigner.java 87619b55dae0c49029d4f804a52fdcbf9f16a50a 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImpl.java 54bd1770274dcedecacccba697aee65b8dd48347 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java 3c38f953ed830a2e3627705aa6493e9b37dae155 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImpl.java cff4ab196451fc846827cfc1f48897e645ac3d5d 
>   src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 1219215d610e0dbe1f4e78bb8ea57504de0fa329 
>   src/main/java/org/apache/aurora/scheduler/storage/durability/WriteRecorder.java dea8e690d8325596c70f31212029ddcfdf8e4451 
>   src/main/java/org/apache/aurora/scheduler/updater/UpdateAgentReserver.java b6909a692b7ea590768ecead08b59277b5577866 
>   src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 82e40d509d84c37a19b6a9ef942283d908833840 
>   src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java 64d7a44eaeb627daf7b7bee75ac0be19594a9b25 
>   src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 0de90d70afa1bfa00c93875a325b8a3a7ed7b03d 
>   src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java c27a662a2a03ee15098ff66229f86da831177a52 
>   src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 1e9532bef39591fcc61a2b41a04f2290960d8906 
>   src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java 348386e938ce910c73f1e2e6e2b39f3654a85e7e 
>   src/test/java/org/apache/aurora/scheduler/scheduling/FirstFitOfferSelectorTest.java 5e2fdcb646a3362043f171bfc201b83d76524d78 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImplTest.java 78e12691be0bf20e868ad202aaa306ef29bf5971 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java ecf2987bf9fa124a4de3a93ffb20b4a807d79b3e 
>   src/test/java/org/apache/aurora/scheduler/updater/NullAgentReserverTest.java d8563b86747a7887be9262a621fd761fea832b8c 
>   src/test/java/org/apache/aurora/scheduler/updater/UpdateAgentReserverImplTest.java 7f17be0427e3baf12aaa1d176414f7009baeea94 
> 
> 
> Diff: https://reviews.apache.org/r/64954/diff/2/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Bill Farner
> 
>


Re: Review Request 64954: Refactor scheduling code to split matching and assigning phases

Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64954/#review195060
-----------------------------------------------------------


Ship it!




Master (8e5e08e) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot retry"

- Aurora ReviewBot


On Jan. 9, 2018, 6:32 p.m., Bill Farner wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64954/
> -----------------------------------------------------------
> 
> (Updated Jan. 9, 2018, 6:32 p.m.)
> 
> 
> Review request for Aurora, Daniel Knightly and Jordan Ly.
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> This patch sets the stage for performing the bulk of scheduling work in a separate call path, without holding the write lock.
> 
> Also included is a mechanical refactor pushing the `revocable` flag into `ResourceRequest` (which was ~always needed as a sibling parameter).
> 
> 
> Diffs
> -----
> 
>   commons/src/main/java/org/apache/aurora/common/testing/easymock/EasyMockTest.java c5500b3658437d0a940f953957c880559f5f1b61 
>   src/jmh/java/org/apache/aurora/benchmark/fakes/FakeOfferManager.java f0dacd4e90bc142df8b6489ffb33d7265ec03706 
>   src/main/java/org/apache/aurora/scheduler/TierManager.java c6ad2b1c48673ca2c14ddd308684d81ce536beca 
>   src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 22d5a642687c1ddc91071c27abda7d61726b28b9 
>   src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorSettings.java 5c987fd051728486172c8afd34219e86d56f00d5 
>   src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java bd415909ac1abdb670fde57ead0e391de3e5238b 
>   src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java cb288bb12d577330888c6a66ddd2baaff4b71e8e 
>   src/main/java/org/apache/aurora/scheduler/offers/HostOffers.java a01c0a87856a63b1734b4ea336bbbf5fa74b8636 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java e90de3eec43a3df9ff019830e927ecff195d1747 
>   src/main/java/org/apache/aurora/scheduler/offers/OfferManagerImpl.java 8e806b705d8bee4381ed8afd792d777420405195 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictim.java 69b686639de1eb7d896096e2fadb357fe9ef3141 
>   src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java cf6d348d645d78ab77fcd305307df8a94fea114c 
>   src/main/java/org/apache/aurora/scheduler/resources/ResourceManager.java d093753b715ffc9ff37448f0266903d6778af024 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssigner.java 87619b55dae0c49029d4f804a52fdcbf9f16a50a 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImpl.java 54bd1770274dcedecacccba697aee65b8dd48347 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java 3c38f953ed830a2e3627705aa6493e9b37dae155 
>   src/main/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImpl.java cff4ab196451fc846827cfc1f48897e645ac3d5d 
>   src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 1219215d610e0dbe1f4e78bb8ea57504de0fa329 
>   src/main/java/org/apache/aurora/scheduler/storage/durability/WriteRecorder.java dea8e690d8325596c70f31212029ddcfdf8e4451 
>   src/main/java/org/apache/aurora/scheduler/updater/UpdateAgentReserver.java b6909a692b7ea590768ecead08b59277b5577866 
>   src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 82e40d509d84c37a19b6a9ef942283d908833840 
>   src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java 64d7a44eaeb627daf7b7bee75ac0be19594a9b25 
>   src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 0de90d70afa1bfa00c93875a325b8a3a7ed7b03d 
>   src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java c27a662a2a03ee15098ff66229f86da831177a52 
>   src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 1e9532bef39591fcc61a2b41a04f2290960d8906 
>   src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java 348386e938ce910c73f1e2e6e2b39f3654a85e7e 
>   src/test/java/org/apache/aurora/scheduler/scheduling/FirstFitOfferSelectorTest.java 5e2fdcb646a3362043f171bfc201b83d76524d78 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImplTest.java 78e12691be0bf20e868ad202aaa306ef29bf5971 
>   src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java ecf2987bf9fa124a4de3a93ffb20b4a807d79b3e 
>   src/test/java/org/apache/aurora/scheduler/updater/NullAgentReserverTest.java d8563b86747a7887be9262a621fd761fea832b8c 
>   src/test/java/org/apache/aurora/scheduler/updater/UpdateAgentReserverImplTest.java 7f17be0427e3baf12aaa1d176414f7009baeea94 
> 
> 
> Diff: https://reviews.apache.org/r/64954/diff/2/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Bill Farner
> 
>


Re: Review Request 64954: Refactor scheduling code to split matching and assigning phases

Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64954/
-----------------------------------------------------------

(Updated Jan. 9, 2018, 10:32 a.m.)


Review request for Aurora, Daniel Knightly and Jordan Ly.


Changes
-------

Simplify usage of `ReservationStatus`.  Address review feeback.


Repository: aurora


Description
-------

This patch sets the stage for performing the bulk of scheduling work in a separate call path, without holding the write lock.

Also included is a mechanical refactor pushing the `revocable` flag into `ResourceRequest` (which was ~always needed as a sibling parameter).


Diffs (updated)
-----

  commons/src/main/java/org/apache/aurora/common/testing/easymock/EasyMockTest.java c5500b3658437d0a940f953957c880559f5f1b61 
  src/jmh/java/org/apache/aurora/benchmark/fakes/FakeOfferManager.java f0dacd4e90bc142df8b6489ffb33d7265ec03706 
  src/main/java/org/apache/aurora/scheduler/TierManager.java c6ad2b1c48673ca2c14ddd308684d81ce536beca 
  src/main/java/org/apache/aurora/scheduler/base/TaskTestUtil.java 22d5a642687c1ddc91071c27abda7d61726b28b9 
  src/main/java/org/apache/aurora/scheduler/configuration/executor/ExecutorSettings.java 5c987fd051728486172c8afd34219e86d56f00d5 
  src/main/java/org/apache/aurora/scheduler/filter/SchedulingFilter.java bd415909ac1abdb670fde57ead0e391de3e5238b 
  src/main/java/org/apache/aurora/scheduler/mesos/MesosTaskFactory.java cb288bb12d577330888c6a66ddd2baaff4b71e8e 
  src/main/java/org/apache/aurora/scheduler/offers/HostOffers.java a01c0a87856a63b1734b4ea336bbbf5fa74b8636 
  src/main/java/org/apache/aurora/scheduler/offers/OfferManager.java e90de3eec43a3df9ff019830e927ecff195d1747 
  src/main/java/org/apache/aurora/scheduler/offers/OfferManagerImpl.java 8e806b705d8bee4381ed8afd792d777420405195 
  src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictim.java 69b686639de1eb7d896096e2fadb357fe9ef3141 
  src/main/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilter.java cf6d348d645d78ab77fcd305307df8a94fea114c 
  src/main/java/org/apache/aurora/scheduler/resources/ResourceManager.java d093753b715ffc9ff37448f0266903d6778af024 
  src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssigner.java 87619b55dae0c49029d4f804a52fdcbf9f16a50a 
  src/main/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImpl.java 54bd1770274dcedecacccba697aee65b8dd48347 
  src/main/java/org/apache/aurora/scheduler/scheduling/TaskScheduler.java 3c38f953ed830a2e3627705aa6493e9b37dae155 
  src/main/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImpl.java cff4ab196451fc846827cfc1f48897e645ac3d5d 
  src/main/java/org/apache/aurora/scheduler/storage/TaskStore.java 1219215d610e0dbe1f4e78bb8ea57504de0fa329 
  src/main/java/org/apache/aurora/scheduler/storage/durability/WriteRecorder.java dea8e690d8325596c70f31212029ddcfdf8e4451 
  src/main/java/org/apache/aurora/scheduler/updater/UpdateAgentReserver.java b6909a692b7ea590768ecead08b59277b5577866 
  src/test/java/org/apache/aurora/scheduler/TierManagerTest.java 82e40d509d84c37a19b6a9ef942283d908833840 
  src/test/java/org/apache/aurora/scheduler/events/NotifyingSchedulingFilterTest.java 64d7a44eaeb627daf7b7bee75ac0be19594a9b25 
  src/test/java/org/apache/aurora/scheduler/filter/SchedulingFilterImplTest.java 0de90d70afa1bfa00c93875a325b8a3a7ed7b03d 
  src/test/java/org/apache/aurora/scheduler/mesos/MesosTaskFactoryImplTest.java c27a662a2a03ee15098ff66229f86da831177a52 
  src/test/java/org/apache/aurora/scheduler/offers/OfferManagerImplTest.java 1e9532bef39591fcc61a2b41a04f2290960d8906 
  src/test/java/org/apache/aurora/scheduler/preemptor/PreemptionVictimFilterTest.java 348386e938ce910c73f1e2e6e2b39f3654a85e7e 
  src/test/java/org/apache/aurora/scheduler/scheduling/FirstFitOfferSelectorTest.java 5e2fdcb646a3362043f171bfc201b83d76524d78 
  src/test/java/org/apache/aurora/scheduler/scheduling/TaskAssignerImplTest.java 78e12691be0bf20e868ad202aaa306ef29bf5971 
  src/test/java/org/apache/aurora/scheduler/scheduling/TaskSchedulerImplTest.java ecf2987bf9fa124a4de3a93ffb20b4a807d79b3e 
  src/test/java/org/apache/aurora/scheduler/updater/NullAgentReserverTest.java d8563b86747a7887be9262a621fd761fea832b8c 
  src/test/java/org/apache/aurora/scheduler/updater/UpdateAgentReserverImplTest.java 7f17be0427e3baf12aaa1d176414f7009baeea94 


Diff: https://reviews.apache.org/r/64954/diff/2/

Changes: https://reviews.apache.org/r/64954/diff/1-2/


Testing
-------


Thanks,

Bill Farner