You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by Bill Farner <wf...@apache.org> on 2014/01/08 21:05:33 UTC

Review Request 16740: When rescheduling a task, send it to the THROTTLED state if it has been penalized for flapping.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16740/
-----------------------------------------------------------

Review request for Aurora, Kevin Sweeney, Suman Karumuri, and Maxim Khutornenko.


Bugs: AURORA-23
    https://issues.apache.org/jira/browse/AURORA-23


Repository: aurora


Description
-------

When rescheduling a task, send it to the THROTTLED state if it has been penalized for flapping.

Changes specific to the issue:
- Introduced TaskThrottler[Test], which initiates THROTTLED->PENDING transitions
- RescheduleCalculator methods return a delay rather than a timestamp, for inclusion in transition audit messages
- Removed StorageBackfillTest, which was only testing rewriting the THROTTLED state for backwards compatibility

There's also a fair amount of code cleanup going on in this review:

- A unit test for RescheduleCalculator was pulled out of TaskSchedulerTest test cases (per TODO)
- Introduced Tasks.getLatestEvent, which was independently implemented in several places
- Removed unused TaskRescheduled pubsub event
- Removed REVERSE_CHRON_COMPARATOR from SchedulerzJob, which is === Tasks.LATEST_ACTIVITY.reverse()
- Added a batch changeState helper function in BaseSchedulerCoreImplTest for more concise calling code


Diffs
-----

  src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java 106fbb3e6e09518df3b2d687a0a85e4ecae0e72f 
  src/main/java/org/apache/aurora/scheduler/async/Preemptor.java 0afbef950151e8cd1e865d93b53ffa4ad92f8eeb 
  src/main/java/org/apache/aurora/scheduler/async/RescheduleCalculator.java 0265bf941814ab4b6425ecf4dc9672dfa361e94f 
  src/main/java/org/apache/aurora/scheduler/async/TaskGroup.java 1a008745ac8807edd894269722fa2ae989b70bbd 
  src/main/java/org/apache/aurora/scheduler/async/TaskGroups.java b50c625092812f84b3f0a9e790bbe7011a60c1ce 
  src/main/java/org/apache/aurora/scheduler/async/TaskThrottler.java PRE-CREATION 
  src/main/java/org/apache/aurora/scheduler/base/Tasks.java 569e8c32643acdce35062d283c7411e026b86c8d 
  src/main/java/org/apache/aurora/scheduler/events/PubsubEvent.java 59e18eaefc024b7493b4b2aa3757b831022236f3 
  src/main/java/org/apache/aurora/scheduler/http/SchedulerzJob.java 07a648fad0ea55bdb126b6d8d2ed15754430f556 
  src/main/java/org/apache/aurora/scheduler/http/SchedulerzRole.java cef0ff28bb0c0e08c5efaa1ed326f66bc9ffa5d9 
  src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java b6dd5379481423051b9243ad0dbfae7b057173f4 
  src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java d0f88e5b16b8e4af299cd1d19f806b1a6c53eab4 
  src/main/java/org/apache/aurora/scheduler/storage/StorageBackfill.java 69374ca706ff7ab6267c37021612cefb74df0bc0 
  src/main/thrift/org/apache/aurora/gen/api.thrift 480b8f472bcfbe547a91c41638250350a0110334 
  src/test/java/org/apache/aurora/scheduler/async/RescheduleCalculatorImplTest.java PRE-CREATION 
  src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 9698f283c04197039d22cd38220a1dda448bdcf7 
  src/test/java/org/apache/aurora/scheduler/async/TaskThrottlerTest.java PRE-CREATION 
  src/test/java/org/apache/aurora/scheduler/state/BaseSchedulerCoreImplTest.java 720d0c86d8b112bf92196cbb81ece44476534654 
  src/test/java/org/apache/aurora/scheduler/state/StateManagerImplTest.java b17b983f66ce0ac1ef90b716a28813a505322b17 
  src/test/java/org/apache/aurora/scheduler/state/TaskStateMachineTest.java e89e60afc107b299d572a07140d010040b2c7545 
  src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 724188bfa037caccaf91cf57d82de1777ab5de8e 

Diff: https://reviews.apache.org/r/16740/diff/


Testing
-------

./gradlew build


Thanks,

Bill Farner


Re: Review Request 16740: When rescheduling a task, send it to the THROTTLED state if it has been penalized for flapping.

Posted by Bill Farner <wf...@apache.org>.

> On Jan. 9, 2014, 2:25 a.m., Maxim Khutornenko wrote:
> > Would it make sense to keep the StorageBackfillTest class as a placeholder for future schema migration tests? Will come handy next time we go through a schema migration.

I'm generally pretty firmly against keeping dead code around.  I felt weird deleting a test class as well, but it served no purpose in the current code.


> On Jan. 9, 2014, 2:25 a.m., Maxim Khutornenko wrote:
> > src/main/java/org/apache/aurora/scheduler/async/RescheduleCalculator.java, line 62
> > <https://reviews.apache.org/r/16740/diff/1/?file=419013#file419013line62>
> >
> >     Why dropping Ms suffix here?

That's an oversight.  Thanks, fixed.


> On Jan. 9, 2014, 2:25 a.m., Maxim Khutornenko wrote:
> > src/main/java/org/apache/aurora/scheduler/async/TaskThrottler.java, line 40
> > <https://reviews.apache.org/r/16740/diff/1/?file=419016#file419016line40>
> >
> >     Does it make sense to drop org.apache.aurora.gen? I think SchedulerStatus would resolve in javadoc just fine without it.

Nope, it doesn't because it's not explicitly imported (only fields within are statically imported, and i can't reference those in javadoc).  AFAICT this is the only way to express it.


> On Jan. 9, 2014, 2:25 a.m., Maxim Khutornenko wrote:
> > src/main/java/org/apache/aurora/scheduler/async/TaskThrottler.java, line 66
> > <https://reviews.apache.org/r/16740/diff/1/?file=419016#file419016line66>
> >
> >     Since THROTTLED state is looking rather transient and short-living, the task_store_ stat sampling rate might not be sensitive enough to pick up the correct throttled values. Would it make sense to add a dedicated stat here exposing a real penalty rate?

Sure, now capturing aggregate throttling stats.


> On Jan. 9, 2014, 2:25 a.m., Maxim Khutornenko wrote:
> > src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java, line 275
> > <https://reviews.apache.org/r/16740/diff/1/?file=419022#file419022line275>
> >
> >     Do we need rules to transition from THROTTLED state on scheduler restart?

Aha, that's the beauty of my recent TaskStateChange pubsub patch!  TaskThrottler is notified of a x->THROTTLED transition at startup.


- Bill


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16740/#review31421
-----------------------------------------------------------


On Jan. 8, 2014, 8:05 p.m., Bill Farner wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16740/
> -----------------------------------------------------------
> 
> (Updated Jan. 8, 2014, 8:05 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney, Suman Karumuri, and Maxim Khutornenko.
> 
> 
> Bugs: AURORA-23
>     https://issues.apache.org/jira/browse/AURORA-23
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> When rescheduling a task, send it to the THROTTLED state if it has been penalized for flapping.
> 
> Changes specific to the issue:
> - Introduced TaskThrottler[Test], which initiates THROTTLED->PENDING transitions
> - RescheduleCalculator methods return a delay rather than a timestamp, for inclusion in transition audit messages
> - Removed StorageBackfillTest, which was only testing rewriting the THROTTLED state for backwards compatibility
> 
> There's also a fair amount of code cleanup going on in this review:
> 
> - A unit test for RescheduleCalculator was pulled out of TaskSchedulerTest test cases (per TODO)
> - Introduced Tasks.getLatestEvent, which was independently implemented in several places
> - Removed unused TaskRescheduled pubsub event
> - Removed REVERSE_CHRON_COMPARATOR from SchedulerzJob, which is === Tasks.LATEST_ACTIVITY.reverse()
> - Added a batch changeState helper function in BaseSchedulerCoreImplTest for more concise calling code
> 
> 
> Diffs
> -----
> 
>   src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java 106fbb3e6e09518df3b2d687a0a85e4ecae0e72f 
>   src/main/java/org/apache/aurora/scheduler/async/Preemptor.java 0afbef950151e8cd1e865d93b53ffa4ad92f8eeb 
>   src/main/java/org/apache/aurora/scheduler/async/RescheduleCalculator.java 0265bf941814ab4b6425ecf4dc9672dfa361e94f 
>   src/main/java/org/apache/aurora/scheduler/async/TaskGroup.java 1a008745ac8807edd894269722fa2ae989b70bbd 
>   src/main/java/org/apache/aurora/scheduler/async/TaskGroups.java b50c625092812f84b3f0a9e790bbe7011a60c1ce 
>   src/main/java/org/apache/aurora/scheduler/async/TaskThrottler.java PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/base/Tasks.java 569e8c32643acdce35062d283c7411e026b86c8d 
>   src/main/java/org/apache/aurora/scheduler/events/PubsubEvent.java 59e18eaefc024b7493b4b2aa3757b831022236f3 
>   src/main/java/org/apache/aurora/scheduler/http/SchedulerzJob.java 07a648fad0ea55bdb126b6d8d2ed15754430f556 
>   src/main/java/org/apache/aurora/scheduler/http/SchedulerzRole.java cef0ff28bb0c0e08c5efaa1ed326f66bc9ffa5d9 
>   src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java b6dd5379481423051b9243ad0dbfae7b057173f4 
>   src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java d0f88e5b16b8e4af299cd1d19f806b1a6c53eab4 
>   src/main/java/org/apache/aurora/scheduler/storage/StorageBackfill.java 69374ca706ff7ab6267c37021612cefb74df0bc0 
>   src/main/thrift/org/apache/aurora/gen/api.thrift 480b8f472bcfbe547a91c41638250350a0110334 
>   src/test/java/org/apache/aurora/scheduler/async/RescheduleCalculatorImplTest.java PRE-CREATION 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 9698f283c04197039d22cd38220a1dda448bdcf7 
>   src/test/java/org/apache/aurora/scheduler/async/TaskThrottlerTest.java PRE-CREATION 
>   src/test/java/org/apache/aurora/scheduler/state/BaseSchedulerCoreImplTest.java 720d0c86d8b112bf92196cbb81ece44476534654 
>   src/test/java/org/apache/aurora/scheduler/state/StateManagerImplTest.java b17b983f66ce0ac1ef90b716a28813a505322b17 
>   src/test/java/org/apache/aurora/scheduler/state/TaskStateMachineTest.java e89e60afc107b299d572a07140d010040b2c7545 
>   src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 724188bfa037caccaf91cf57d82de1777ab5de8e 
> 
> Diff: https://reviews.apache.org/r/16740/diff/
> 
> 
> Testing
> -------
> 
> ./gradlew build
> 
> 
> Thanks,
> 
> Bill Farner
> 
>


Re: Review Request 16740: When rescheduling a task, send it to the THROTTLED state if it has been penalized for flapping.

Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16740/#review31421
-----------------------------------------------------------


Would it make sense to keep the StorageBackfillTest class as a placeholder for future schema migration tests? Will come handy next time we go through a schema migration.


src/main/java/org/apache/aurora/scheduler/async/RescheduleCalculator.java
<https://reviews.apache.org/r/16740/#comment59894>

    Why dropping Ms suffix here?



src/main/java/org/apache/aurora/scheduler/async/TaskThrottler.java
<https://reviews.apache.org/r/16740/#comment59895>

    Does it make sense to drop org.apache.aurora.gen? I think SchedulerStatus would resolve in javadoc just fine without it.



src/main/java/org/apache/aurora/scheduler/async/TaskThrottler.java
<https://reviews.apache.org/r/16740/#comment59908>

    Since THROTTLED state is looking rather transient and short-living, the task_store_ stat sampling rate might not be sensitive enough to pick up the correct throttled values. Would it make sense to add a dedicated stat here exposing a real penalty rate?



src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java
<https://reviews.apache.org/r/16740/#comment59909>

    Do we need rules to transition from THROTTLED state on scheduler restart?


- Maxim Khutornenko


On Jan. 8, 2014, 8:05 p.m., Bill Farner wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16740/
> -----------------------------------------------------------
> 
> (Updated Jan. 8, 2014, 8:05 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney, Suman Karumuri, and Maxim Khutornenko.
> 
> 
> Bugs: AURORA-23
>     https://issues.apache.org/jira/browse/AURORA-23
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> When rescheduling a task, send it to the THROTTLED state if it has been penalized for flapping.
> 
> Changes specific to the issue:
> - Introduced TaskThrottler[Test], which initiates THROTTLED->PENDING transitions
> - RescheduleCalculator methods return a delay rather than a timestamp, for inclusion in transition audit messages
> - Removed StorageBackfillTest, which was only testing rewriting the THROTTLED state for backwards compatibility
> 
> There's also a fair amount of code cleanup going on in this review:
> 
> - A unit test for RescheduleCalculator was pulled out of TaskSchedulerTest test cases (per TODO)
> - Introduced Tasks.getLatestEvent, which was independently implemented in several places
> - Removed unused TaskRescheduled pubsub event
> - Removed REVERSE_CHRON_COMPARATOR from SchedulerzJob, which is === Tasks.LATEST_ACTIVITY.reverse()
> - Added a batch changeState helper function in BaseSchedulerCoreImplTest for more concise calling code
> 
> 
> Diffs
> -----
> 
>   src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java 106fbb3e6e09518df3b2d687a0a85e4ecae0e72f 
>   src/main/java/org/apache/aurora/scheduler/async/Preemptor.java 0afbef950151e8cd1e865d93b53ffa4ad92f8eeb 
>   src/main/java/org/apache/aurora/scheduler/async/RescheduleCalculator.java 0265bf941814ab4b6425ecf4dc9672dfa361e94f 
>   src/main/java/org/apache/aurora/scheduler/async/TaskGroup.java 1a008745ac8807edd894269722fa2ae989b70bbd 
>   src/main/java/org/apache/aurora/scheduler/async/TaskGroups.java b50c625092812f84b3f0a9e790bbe7011a60c1ce 
>   src/main/java/org/apache/aurora/scheduler/async/TaskThrottler.java PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/base/Tasks.java 569e8c32643acdce35062d283c7411e026b86c8d 
>   src/main/java/org/apache/aurora/scheduler/events/PubsubEvent.java 59e18eaefc024b7493b4b2aa3757b831022236f3 
>   src/main/java/org/apache/aurora/scheduler/http/SchedulerzJob.java 07a648fad0ea55bdb126b6d8d2ed15754430f556 
>   src/main/java/org/apache/aurora/scheduler/http/SchedulerzRole.java cef0ff28bb0c0e08c5efaa1ed326f66bc9ffa5d9 
>   src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java b6dd5379481423051b9243ad0dbfae7b057173f4 
>   src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java d0f88e5b16b8e4af299cd1d19f806b1a6c53eab4 
>   src/main/java/org/apache/aurora/scheduler/storage/StorageBackfill.java 69374ca706ff7ab6267c37021612cefb74df0bc0 
>   src/main/thrift/org/apache/aurora/gen/api.thrift 480b8f472bcfbe547a91c41638250350a0110334 
>   src/test/java/org/apache/aurora/scheduler/async/RescheduleCalculatorImplTest.java PRE-CREATION 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 9698f283c04197039d22cd38220a1dda448bdcf7 
>   src/test/java/org/apache/aurora/scheduler/async/TaskThrottlerTest.java PRE-CREATION 
>   src/test/java/org/apache/aurora/scheduler/state/BaseSchedulerCoreImplTest.java 720d0c86d8b112bf92196cbb81ece44476534654 
>   src/test/java/org/apache/aurora/scheduler/state/StateManagerImplTest.java b17b983f66ce0ac1ef90b716a28813a505322b17 
>   src/test/java/org/apache/aurora/scheduler/state/TaskStateMachineTest.java e89e60afc107b299d572a07140d010040b2c7545 
>   src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 724188bfa037caccaf91cf57d82de1777ab5de8e 
> 
> Diff: https://reviews.apache.org/r/16740/diff/
> 
> 
> Testing
> -------
> 
> ./gradlew build
> 
> 
> Thanks,
> 
> Bill Farner
> 
>


Re: Review Request 16740: When rescheduling a task, send it to the THROTTLED state if it has been penalized for flapping.

Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16740/#review31484
-----------------------------------------------------------

Ship it!


Ship It!

- Maxim Khutornenko


On Jan. 9, 2014, 10:53 p.m., Bill Farner wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16740/
> -----------------------------------------------------------
> 
> (Updated Jan. 9, 2014, 10:53 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney, Suman Karumuri, and Maxim Khutornenko.
> 
> 
> Bugs: AURORA-23
>     https://issues.apache.org/jira/browse/AURORA-23
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> When rescheduling a task, send it to the THROTTLED state if it has been penalized for flapping.
> 
> Changes specific to the issue:
> - Introduced TaskThrottler[Test], which initiates THROTTLED->PENDING transitions
> - RescheduleCalculator methods return a delay rather than a timestamp, for inclusion in transition audit messages
> - Removed StorageBackfillTest, which was only testing rewriting the THROTTLED state for backwards compatibility
> 
> There's also a fair amount of code cleanup going on in this review:
> 
> - A unit test for RescheduleCalculator was pulled out of TaskSchedulerTest test cases (per TODO)
> - Introduced Tasks.getLatestEvent, which was independently implemented in several places
> - Removed unused TaskRescheduled pubsub event
> - Removed REVERSE_CHRON_COMPARATOR from SchedulerzJob, which is === Tasks.LATEST_ACTIVITY.reverse()
> - Added a batch changeState helper function in BaseSchedulerCoreImplTest for more concise calling code
> 
> 
> Diffs
> -----
> 
>   src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java 106fbb3e6e09518df3b2d687a0a85e4ecae0e72f 
>   src/main/java/org/apache/aurora/scheduler/async/Preemptor.java 0afbef950151e8cd1e865d93b53ffa4ad92f8eeb 
>   src/main/java/org/apache/aurora/scheduler/async/RescheduleCalculator.java 0265bf941814ab4b6425ecf4dc9672dfa361e94f 
>   src/main/java/org/apache/aurora/scheduler/async/TaskGroup.java 1a008745ac8807edd894269722fa2ae989b70bbd 
>   src/main/java/org/apache/aurora/scheduler/async/TaskGroups.java b50c625092812f84b3f0a9e790bbe7011a60c1ce 
>   src/main/java/org/apache/aurora/scheduler/async/TaskThrottler.java PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/base/Tasks.java 569e8c32643acdce35062d283c7411e026b86c8d 
>   src/main/java/org/apache/aurora/scheduler/events/PubsubEvent.java 59e18eaefc024b7493b4b2aa3757b831022236f3 
>   src/main/java/org/apache/aurora/scheduler/http/SchedulerzJob.java 07a648fad0ea55bdb126b6d8d2ed15754430f556 
>   src/main/java/org/apache/aurora/scheduler/http/SchedulerzRole.java cef0ff28bb0c0e08c5efaa1ed326f66bc9ffa5d9 
>   src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java b6dd5379481423051b9243ad0dbfae7b057173f4 
>   src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java d0f88e5b16b8e4af299cd1d19f806b1a6c53eab4 
>   src/main/java/org/apache/aurora/scheduler/storage/StorageBackfill.java 69374ca706ff7ab6267c37021612cefb74df0bc0 
>   src/main/thrift/org/apache/aurora/gen/api.thrift 480b8f472bcfbe547a91c41638250350a0110334 
>   src/test/java/org/apache/aurora/scheduler/async/RescheduleCalculatorImplTest.java PRE-CREATION 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 9698f283c04197039d22cd38220a1dda448bdcf7 
>   src/test/java/org/apache/aurora/scheduler/async/TaskThrottlerTest.java PRE-CREATION 
>   src/test/java/org/apache/aurora/scheduler/state/BaseSchedulerCoreImplTest.java 720d0c86d8b112bf92196cbb81ece44476534654 
>   src/test/java/org/apache/aurora/scheduler/state/StateManagerImplTest.java b17b983f66ce0ac1ef90b716a28813a505322b17 
>   src/test/java/org/apache/aurora/scheduler/state/TaskStateMachineTest.java e89e60afc107b299d572a07140d010040b2c7545 
>   src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 724188bfa037caccaf91cf57d82de1777ab5de8e 
> 
> Diff: https://reviews.apache.org/r/16740/diff/
> 
> 
> Testing
> -------
> 
> ./gradlew build
> 
> 
> Thanks,
> 
> Bill Farner
> 
>


Re: Review Request 16740: When rescheduling a task, send it to the THROTTLED state if it has been penalized for flapping.

Posted by Suman Karumuri <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16740/#review31785
-----------------------------------------------------------

Ship it!


Ship It!

- Suman Karumuri


On Jan. 13, 2014, 11:54 p.m., Bill Farner wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16740/
> -----------------------------------------------------------
> 
> (Updated Jan. 13, 2014, 11:54 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney, Suman Karumuri, and Maxim Khutornenko.
> 
> 
> Bugs: AURORA-23
>     https://issues.apache.org/jira/browse/AURORA-23
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> When rescheduling a task, send it to the THROTTLED state if it has been penalized for flapping.
> 
> Changes specific to the issue:
> - Introduced TaskThrottler[Test], which initiates THROTTLED->PENDING transitions
> - RescheduleCalculator methods return a delay rather than a timestamp, for inclusion in transition audit messages
> - Removed StorageBackfillTest, which was only testing rewriting the THROTTLED state for backwards compatibility
> 
> There's also a fair amount of code cleanup going on in this review:
> 
> - A unit test for RescheduleCalculator was pulled out of TaskSchedulerTest test cases (per TODO)
> - Introduced Tasks.getLatestEvent, which was independently implemented in several places
> - Removed unused TaskRescheduled pubsub event
> - Removed REVERSE_CHRON_COMPARATOR from SchedulerzJob, which is === Tasks.LATEST_ACTIVITY.reverse()
> - Added a batch changeState helper function in BaseSchedulerCoreImplTest for more concise calling code
> 
> 
> Diffs
> -----
> 
>   src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java 6be658d7e90f59c63e6895df3917a7869b4045cc 
>   src/main/java/org/apache/aurora/scheduler/async/Preemptor.java 0afbef950151e8cd1e865d93b53ffa4ad92f8eeb 
>   src/main/java/org/apache/aurora/scheduler/async/RescheduleCalculator.java 0265bf941814ab4b6425ecf4dc9672dfa361e94f 
>   src/main/java/org/apache/aurora/scheduler/async/TaskGroup.java 1a008745ac8807edd894269722fa2ae989b70bbd 
>   src/main/java/org/apache/aurora/scheduler/async/TaskGroups.java b50c625092812f84b3f0a9e790bbe7011a60c1ce 
>   src/main/java/org/apache/aurora/scheduler/async/TaskThrottler.java PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/base/Tasks.java 569e8c32643acdce35062d283c7411e026b86c8d 
>   src/main/java/org/apache/aurora/scheduler/events/PubsubEvent.java 59e18eaefc024b7493b4b2aa3757b831022236f3 
>   src/main/java/org/apache/aurora/scheduler/http/SchedulerzJob.java 07a648fad0ea55bdb126b6d8d2ed15754430f556 
>   src/main/java/org/apache/aurora/scheduler/http/SchedulerzRole.java cef0ff28bb0c0e08c5efaa1ed326f66bc9ffa5d9 
>   src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java b6dd5379481423051b9243ad0dbfae7b057173f4 
>   src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java d0f88e5b16b8e4af299cd1d19f806b1a6c53eab4 
>   src/main/java/org/apache/aurora/scheduler/storage/StorageBackfill.java 69374ca706ff7ab6267c37021612cefb74df0bc0 
>   src/main/thrift/org/apache/aurora/gen/api.thrift 33c70df70713427c186081dd1815dd4e8fe14fa8 
>   src/test/java/org/apache/aurora/scheduler/async/RescheduleCalculatorImplTest.java PRE-CREATION 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 9698f283c04197039d22cd38220a1dda448bdcf7 
>   src/test/java/org/apache/aurora/scheduler/async/TaskThrottlerTest.java PRE-CREATION 
>   src/test/java/org/apache/aurora/scheduler/state/BaseSchedulerCoreImplTest.java 720d0c86d8b112bf92196cbb81ece44476534654 
>   src/test/java/org/apache/aurora/scheduler/state/StateManagerImplTest.java b17b983f66ce0ac1ef90b716a28813a505322b17 
>   src/test/java/org/apache/aurora/scheduler/state/TaskStateMachineTest.java e89e60afc107b299d572a07140d010040b2c7545 
>   src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 724188bfa037caccaf91cf57d82de1777ab5de8e 
> 
> Diff: https://reviews.apache.org/r/16740/diff/
> 
> 
> Testing
> -------
> 
> ./gradlew build
> 
> 
> Thanks,
> 
> Bill Farner
> 
>


Re: Review Request 16740: When rescheduling a task, send it to the THROTTLED state if it has been penalized for flapping.

Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16740/
-----------------------------------------------------------

(Updated Jan. 13, 2014, 11:54 p.m.)


Review request for Aurora, Kevin Sweeney, Suman Karumuri, and Maxim Khutornenko.


Bugs: AURORA-23
    https://issues.apache.org/jira/browse/AURORA-23


Repository: aurora


Description
-------

When rescheduling a task, send it to the THROTTLED state if it has been penalized for flapping.

Changes specific to the issue:
- Introduced TaskThrottler[Test], which initiates THROTTLED->PENDING transitions
- RescheduleCalculator methods return a delay rather than a timestamp, for inclusion in transition audit messages
- Removed StorageBackfillTest, which was only testing rewriting the THROTTLED state for backwards compatibility

There's also a fair amount of code cleanup going on in this review:

- A unit test for RescheduleCalculator was pulled out of TaskSchedulerTest test cases (per TODO)
- Introduced Tasks.getLatestEvent, which was independently implemented in several places
- Removed unused TaskRescheduled pubsub event
- Removed REVERSE_CHRON_COMPARATOR from SchedulerzJob, which is === Tasks.LATEST_ACTIVITY.reverse()
- Added a batch changeState helper function in BaseSchedulerCoreImplTest for more concise calling code


Diffs (updated)
-----

  src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java 6be658d7e90f59c63e6895df3917a7869b4045cc 
  src/main/java/org/apache/aurora/scheduler/async/Preemptor.java 0afbef950151e8cd1e865d93b53ffa4ad92f8eeb 
  src/main/java/org/apache/aurora/scheduler/async/RescheduleCalculator.java 0265bf941814ab4b6425ecf4dc9672dfa361e94f 
  src/main/java/org/apache/aurora/scheduler/async/TaskGroup.java 1a008745ac8807edd894269722fa2ae989b70bbd 
  src/main/java/org/apache/aurora/scheduler/async/TaskGroups.java b50c625092812f84b3f0a9e790bbe7011a60c1ce 
  src/main/java/org/apache/aurora/scheduler/async/TaskThrottler.java PRE-CREATION 
  src/main/java/org/apache/aurora/scheduler/base/Tasks.java 569e8c32643acdce35062d283c7411e026b86c8d 
  src/main/java/org/apache/aurora/scheduler/events/PubsubEvent.java 59e18eaefc024b7493b4b2aa3757b831022236f3 
  src/main/java/org/apache/aurora/scheduler/http/SchedulerzJob.java 07a648fad0ea55bdb126b6d8d2ed15754430f556 
  src/main/java/org/apache/aurora/scheduler/http/SchedulerzRole.java cef0ff28bb0c0e08c5efaa1ed326f66bc9ffa5d9 
  src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java b6dd5379481423051b9243ad0dbfae7b057173f4 
  src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java d0f88e5b16b8e4af299cd1d19f806b1a6c53eab4 
  src/main/java/org/apache/aurora/scheduler/storage/StorageBackfill.java 69374ca706ff7ab6267c37021612cefb74df0bc0 
  src/main/thrift/org/apache/aurora/gen/api.thrift 33c70df70713427c186081dd1815dd4e8fe14fa8 
  src/test/java/org/apache/aurora/scheduler/async/RescheduleCalculatorImplTest.java PRE-CREATION 
  src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 9698f283c04197039d22cd38220a1dda448bdcf7 
  src/test/java/org/apache/aurora/scheduler/async/TaskThrottlerTest.java PRE-CREATION 
  src/test/java/org/apache/aurora/scheduler/state/BaseSchedulerCoreImplTest.java 720d0c86d8b112bf92196cbb81ece44476534654 
  src/test/java/org/apache/aurora/scheduler/state/StateManagerImplTest.java b17b983f66ce0ac1ef90b716a28813a505322b17 
  src/test/java/org/apache/aurora/scheduler/state/TaskStateMachineTest.java e89e60afc107b299d572a07140d010040b2c7545 
  src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 724188bfa037caccaf91cf57d82de1777ab5de8e 

Diff: https://reviews.apache.org/r/16740/diff/


Testing
-------

./gradlew build


Thanks,

Bill Farner


Re: Review Request 16740: When rescheduling a task, send it to the THROTTLED state if it has been penalized for flapping.

Posted by Bill Farner <wf...@apache.org>.

> On Jan. 13, 2014, 11:42 p.m., Kevin Sweeney wrote:
> > src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java, line 237
> > <https://reviews.apache.org/r/16740/diff/1/?file=419011#file419011line237>
> >
> >     Can just call install(...) iirc.

Hah!  Thanks for catching, looks like this contrived block has gone through a few copy-paste iterations :-)  Removed them all.


- Bill


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16740/#review31419
-----------------------------------------------------------


On Jan. 9, 2014, 10:53 p.m., Bill Farner wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16740/
> -----------------------------------------------------------
> 
> (Updated Jan. 9, 2014, 10:53 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney, Suman Karumuri, and Maxim Khutornenko.
> 
> 
> Bugs: AURORA-23
>     https://issues.apache.org/jira/browse/AURORA-23
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> When rescheduling a task, send it to the THROTTLED state if it has been penalized for flapping.
> 
> Changes specific to the issue:
> - Introduced TaskThrottler[Test], which initiates THROTTLED->PENDING transitions
> - RescheduleCalculator methods return a delay rather than a timestamp, for inclusion in transition audit messages
> - Removed StorageBackfillTest, which was only testing rewriting the THROTTLED state for backwards compatibility
> 
> There's also a fair amount of code cleanup going on in this review:
> 
> - A unit test for RescheduleCalculator was pulled out of TaskSchedulerTest test cases (per TODO)
> - Introduced Tasks.getLatestEvent, which was independently implemented in several places
> - Removed unused TaskRescheduled pubsub event
> - Removed REVERSE_CHRON_COMPARATOR from SchedulerzJob, which is === Tasks.LATEST_ACTIVITY.reverse()
> - Added a batch changeState helper function in BaseSchedulerCoreImplTest for more concise calling code
> 
> 
> Diffs
> -----
> 
>   src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java 106fbb3e6e09518df3b2d687a0a85e4ecae0e72f 
>   src/main/java/org/apache/aurora/scheduler/async/Preemptor.java 0afbef950151e8cd1e865d93b53ffa4ad92f8eeb 
>   src/main/java/org/apache/aurora/scheduler/async/RescheduleCalculator.java 0265bf941814ab4b6425ecf4dc9672dfa361e94f 
>   src/main/java/org/apache/aurora/scheduler/async/TaskGroup.java 1a008745ac8807edd894269722fa2ae989b70bbd 
>   src/main/java/org/apache/aurora/scheduler/async/TaskGroups.java b50c625092812f84b3f0a9e790bbe7011a60c1ce 
>   src/main/java/org/apache/aurora/scheduler/async/TaskThrottler.java PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/base/Tasks.java 569e8c32643acdce35062d283c7411e026b86c8d 
>   src/main/java/org/apache/aurora/scheduler/events/PubsubEvent.java 59e18eaefc024b7493b4b2aa3757b831022236f3 
>   src/main/java/org/apache/aurora/scheduler/http/SchedulerzJob.java 07a648fad0ea55bdb126b6d8d2ed15754430f556 
>   src/main/java/org/apache/aurora/scheduler/http/SchedulerzRole.java cef0ff28bb0c0e08c5efaa1ed326f66bc9ffa5d9 
>   src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java b6dd5379481423051b9243ad0dbfae7b057173f4 
>   src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java d0f88e5b16b8e4af299cd1d19f806b1a6c53eab4 
>   src/main/java/org/apache/aurora/scheduler/storage/StorageBackfill.java 69374ca706ff7ab6267c37021612cefb74df0bc0 
>   src/main/thrift/org/apache/aurora/gen/api.thrift 480b8f472bcfbe547a91c41638250350a0110334 
>   src/test/java/org/apache/aurora/scheduler/async/RescheduleCalculatorImplTest.java PRE-CREATION 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 9698f283c04197039d22cd38220a1dda448bdcf7 
>   src/test/java/org/apache/aurora/scheduler/async/TaskThrottlerTest.java PRE-CREATION 
>   src/test/java/org/apache/aurora/scheduler/state/BaseSchedulerCoreImplTest.java 720d0c86d8b112bf92196cbb81ece44476534654 
>   src/test/java/org/apache/aurora/scheduler/state/StateManagerImplTest.java b17b983f66ce0ac1ef90b716a28813a505322b17 
>   src/test/java/org/apache/aurora/scheduler/state/TaskStateMachineTest.java e89e60afc107b299d572a07140d010040b2c7545 
>   src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 724188bfa037caccaf91cf57d82de1777ab5de8e 
> 
> Diff: https://reviews.apache.org/r/16740/diff/
> 
> 
> Testing
> -------
> 
> ./gradlew build
> 
> 
> Thanks,
> 
> Bill Farner
> 
>


Re: Review Request 16740: When rescheduling a task, send it to the THROTTLED state if it has been penalized for flapping.

Posted by Kevin Sweeney <ke...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16740/#review31419
-----------------------------------------------------------

Ship it!


Shipit was lost in the mail...


src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java
<https://reviews.apache.org/r/16740/#comment59893>

    Can just call install(...) iirc.


- Kevin Sweeney


On Jan. 9, 2014, 2:53 p.m., Bill Farner wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16740/
> -----------------------------------------------------------
> 
> (Updated Jan. 9, 2014, 2:53 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney, Suman Karumuri, and Maxim Khutornenko.
> 
> 
> Bugs: AURORA-23
>     https://issues.apache.org/jira/browse/AURORA-23
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> When rescheduling a task, send it to the THROTTLED state if it has been penalized for flapping.
> 
> Changes specific to the issue:
> - Introduced TaskThrottler[Test], which initiates THROTTLED->PENDING transitions
> - RescheduleCalculator methods return a delay rather than a timestamp, for inclusion in transition audit messages
> - Removed StorageBackfillTest, which was only testing rewriting the THROTTLED state for backwards compatibility
> 
> There's also a fair amount of code cleanup going on in this review:
> 
> - A unit test for RescheduleCalculator was pulled out of TaskSchedulerTest test cases (per TODO)
> - Introduced Tasks.getLatestEvent, which was independently implemented in several places
> - Removed unused TaskRescheduled pubsub event
> - Removed REVERSE_CHRON_COMPARATOR from SchedulerzJob, which is === Tasks.LATEST_ACTIVITY.reverse()
> - Added a batch changeState helper function in BaseSchedulerCoreImplTest for more concise calling code
> 
> 
> Diffs
> -----
> 
>   src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java 106fbb3e6e09518df3b2d687a0a85e4ecae0e72f 
>   src/main/java/org/apache/aurora/scheduler/async/Preemptor.java 0afbef950151e8cd1e865d93b53ffa4ad92f8eeb 
>   src/main/java/org/apache/aurora/scheduler/async/RescheduleCalculator.java 0265bf941814ab4b6425ecf4dc9672dfa361e94f 
>   src/main/java/org/apache/aurora/scheduler/async/TaskGroup.java 1a008745ac8807edd894269722fa2ae989b70bbd 
>   src/main/java/org/apache/aurora/scheduler/async/TaskGroups.java b50c625092812f84b3f0a9e790bbe7011a60c1ce 
>   src/main/java/org/apache/aurora/scheduler/async/TaskThrottler.java PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/base/Tasks.java 569e8c32643acdce35062d283c7411e026b86c8d 
>   src/main/java/org/apache/aurora/scheduler/events/PubsubEvent.java 59e18eaefc024b7493b4b2aa3757b831022236f3 
>   src/main/java/org/apache/aurora/scheduler/http/SchedulerzJob.java 07a648fad0ea55bdb126b6d8d2ed15754430f556 
>   src/main/java/org/apache/aurora/scheduler/http/SchedulerzRole.java cef0ff28bb0c0e08c5efaa1ed326f66bc9ffa5d9 
>   src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java b6dd5379481423051b9243ad0dbfae7b057173f4 
>   src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java d0f88e5b16b8e4af299cd1d19f806b1a6c53eab4 
>   src/main/java/org/apache/aurora/scheduler/storage/StorageBackfill.java 69374ca706ff7ab6267c37021612cefb74df0bc0 
>   src/main/thrift/org/apache/aurora/gen/api.thrift 480b8f472bcfbe547a91c41638250350a0110334 
>   src/test/java/org/apache/aurora/scheduler/async/RescheduleCalculatorImplTest.java PRE-CREATION 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 9698f283c04197039d22cd38220a1dda448bdcf7 
>   src/test/java/org/apache/aurora/scheduler/async/TaskThrottlerTest.java PRE-CREATION 
>   src/test/java/org/apache/aurora/scheduler/state/BaseSchedulerCoreImplTest.java 720d0c86d8b112bf92196cbb81ece44476534654 
>   src/test/java/org/apache/aurora/scheduler/state/StateManagerImplTest.java b17b983f66ce0ac1ef90b716a28813a505322b17 
>   src/test/java/org/apache/aurora/scheduler/state/TaskStateMachineTest.java e89e60afc107b299d572a07140d010040b2c7545 
>   src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 724188bfa037caccaf91cf57d82de1777ab5de8e 
> 
> Diff: https://reviews.apache.org/r/16740/diff/
> 
> 
> Testing
> -------
> 
> ./gradlew build
> 
> 
> Thanks,
> 
> Bill Farner
> 
>


Re: Review Request 16740: When rescheduling a task, send it to the THROTTLED state if it has been penalized for flapping.

Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16740/#review31667
-----------------------------------------------------------


Post-weekend ping.  Waiting on Kevin and Suman here.

- Bill Farner


On Jan. 9, 2014, 10:53 p.m., Bill Farner wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/16740/
> -----------------------------------------------------------
> 
> (Updated Jan. 9, 2014, 10:53 p.m.)
> 
> 
> Review request for Aurora, Kevin Sweeney, Suman Karumuri, and Maxim Khutornenko.
> 
> 
> Bugs: AURORA-23
>     https://issues.apache.org/jira/browse/AURORA-23
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> When rescheduling a task, send it to the THROTTLED state if it has been penalized for flapping.
> 
> Changes specific to the issue:
> - Introduced TaskThrottler[Test], which initiates THROTTLED->PENDING transitions
> - RescheduleCalculator methods return a delay rather than a timestamp, for inclusion in transition audit messages
> - Removed StorageBackfillTest, which was only testing rewriting the THROTTLED state for backwards compatibility
> 
> There's also a fair amount of code cleanup going on in this review:
> 
> - A unit test for RescheduleCalculator was pulled out of TaskSchedulerTest test cases (per TODO)
> - Introduced Tasks.getLatestEvent, which was independently implemented in several places
> - Removed unused TaskRescheduled pubsub event
> - Removed REVERSE_CHRON_COMPARATOR from SchedulerzJob, which is === Tasks.LATEST_ACTIVITY.reverse()
> - Added a batch changeState helper function in BaseSchedulerCoreImplTest for more concise calling code
> 
> 
> Diffs
> -----
> 
>   src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java 106fbb3e6e09518df3b2d687a0a85e4ecae0e72f 
>   src/main/java/org/apache/aurora/scheduler/async/Preemptor.java 0afbef950151e8cd1e865d93b53ffa4ad92f8eeb 
>   src/main/java/org/apache/aurora/scheduler/async/RescheduleCalculator.java 0265bf941814ab4b6425ecf4dc9672dfa361e94f 
>   src/main/java/org/apache/aurora/scheduler/async/TaskGroup.java 1a008745ac8807edd894269722fa2ae989b70bbd 
>   src/main/java/org/apache/aurora/scheduler/async/TaskGroups.java b50c625092812f84b3f0a9e790bbe7011a60c1ce 
>   src/main/java/org/apache/aurora/scheduler/async/TaskThrottler.java PRE-CREATION 
>   src/main/java/org/apache/aurora/scheduler/base/Tasks.java 569e8c32643acdce35062d283c7411e026b86c8d 
>   src/main/java/org/apache/aurora/scheduler/events/PubsubEvent.java 59e18eaefc024b7493b4b2aa3757b831022236f3 
>   src/main/java/org/apache/aurora/scheduler/http/SchedulerzJob.java 07a648fad0ea55bdb126b6d8d2ed15754430f556 
>   src/main/java/org/apache/aurora/scheduler/http/SchedulerzRole.java cef0ff28bb0c0e08c5efaa1ed326f66bc9ffa5d9 
>   src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java b6dd5379481423051b9243ad0dbfae7b057173f4 
>   src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java d0f88e5b16b8e4af299cd1d19f806b1a6c53eab4 
>   src/main/java/org/apache/aurora/scheduler/storage/StorageBackfill.java 69374ca706ff7ab6267c37021612cefb74df0bc0 
>   src/main/thrift/org/apache/aurora/gen/api.thrift 480b8f472bcfbe547a91c41638250350a0110334 
>   src/test/java/org/apache/aurora/scheduler/async/RescheduleCalculatorImplTest.java PRE-CREATION 
>   src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 9698f283c04197039d22cd38220a1dda448bdcf7 
>   src/test/java/org/apache/aurora/scheduler/async/TaskThrottlerTest.java PRE-CREATION 
>   src/test/java/org/apache/aurora/scheduler/state/BaseSchedulerCoreImplTest.java 720d0c86d8b112bf92196cbb81ece44476534654 
>   src/test/java/org/apache/aurora/scheduler/state/StateManagerImplTest.java b17b983f66ce0ac1ef90b716a28813a505322b17 
>   src/test/java/org/apache/aurora/scheduler/state/TaskStateMachineTest.java e89e60afc107b299d572a07140d010040b2c7545 
>   src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 724188bfa037caccaf91cf57d82de1777ab5de8e 
> 
> Diff: https://reviews.apache.org/r/16740/diff/
> 
> 
> Testing
> -------
> 
> ./gradlew build
> 
> 
> Thanks,
> 
> Bill Farner
> 
>


Re: Review Request 16740: When rescheduling a task, send it to the THROTTLED state if it has been penalized for flapping.

Posted by Bill Farner <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16740/
-----------------------------------------------------------

(Updated Jan. 9, 2014, 10:53 p.m.)


Review request for Aurora, Kevin Sweeney, Suman Karumuri, and Maxim Khutornenko.


Bugs: AURORA-23
    https://issues.apache.org/jira/browse/AURORA-23


Repository: aurora


Description
-------

When rescheduling a task, send it to the THROTTLED state if it has been penalized for flapping.

Changes specific to the issue:
- Introduced TaskThrottler[Test], which initiates THROTTLED->PENDING transitions
- RescheduleCalculator methods return a delay rather than a timestamp, for inclusion in transition audit messages
- Removed StorageBackfillTest, which was only testing rewriting the THROTTLED state for backwards compatibility

There's also a fair amount of code cleanup going on in this review:

- A unit test for RescheduleCalculator was pulled out of TaskSchedulerTest test cases (per TODO)
- Introduced Tasks.getLatestEvent, which was independently implemented in several places
- Removed unused TaskRescheduled pubsub event
- Removed REVERSE_CHRON_COMPARATOR from SchedulerzJob, which is === Tasks.LATEST_ACTIVITY.reverse()
- Added a batch changeState helper function in BaseSchedulerCoreImplTest for more concise calling code


Diffs (updated)
-----

  src/main/java/org/apache/aurora/scheduler/async/AsyncModule.java 106fbb3e6e09518df3b2d687a0a85e4ecae0e72f 
  src/main/java/org/apache/aurora/scheduler/async/Preemptor.java 0afbef950151e8cd1e865d93b53ffa4ad92f8eeb 
  src/main/java/org/apache/aurora/scheduler/async/RescheduleCalculator.java 0265bf941814ab4b6425ecf4dc9672dfa361e94f 
  src/main/java/org/apache/aurora/scheduler/async/TaskGroup.java 1a008745ac8807edd894269722fa2ae989b70bbd 
  src/main/java/org/apache/aurora/scheduler/async/TaskGroups.java b50c625092812f84b3f0a9e790bbe7011a60c1ce 
  src/main/java/org/apache/aurora/scheduler/async/TaskThrottler.java PRE-CREATION 
  src/main/java/org/apache/aurora/scheduler/base/Tasks.java 569e8c32643acdce35062d283c7411e026b86c8d 
  src/main/java/org/apache/aurora/scheduler/events/PubsubEvent.java 59e18eaefc024b7493b4b2aa3757b831022236f3 
  src/main/java/org/apache/aurora/scheduler/http/SchedulerzJob.java 07a648fad0ea55bdb126b6d8d2ed15754430f556 
  src/main/java/org/apache/aurora/scheduler/http/SchedulerzRole.java cef0ff28bb0c0e08c5efaa1ed326f66bc9ffa5d9 
  src/main/java/org/apache/aurora/scheduler/state/StateManagerImpl.java b6dd5379481423051b9243ad0dbfae7b057173f4 
  src/main/java/org/apache/aurora/scheduler/state/TaskStateMachine.java d0f88e5b16b8e4af299cd1d19f806b1a6c53eab4 
  src/main/java/org/apache/aurora/scheduler/storage/StorageBackfill.java 69374ca706ff7ab6267c37021612cefb74df0bc0 
  src/main/thrift/org/apache/aurora/gen/api.thrift 480b8f472bcfbe547a91c41638250350a0110334 
  src/test/java/org/apache/aurora/scheduler/async/RescheduleCalculatorImplTest.java PRE-CREATION 
  src/test/java/org/apache/aurora/scheduler/async/TaskSchedulerTest.java 9698f283c04197039d22cd38220a1dda448bdcf7 
  src/test/java/org/apache/aurora/scheduler/async/TaskThrottlerTest.java PRE-CREATION 
  src/test/java/org/apache/aurora/scheduler/state/BaseSchedulerCoreImplTest.java 720d0c86d8b112bf92196cbb81ece44476534654 
  src/test/java/org/apache/aurora/scheduler/state/StateManagerImplTest.java b17b983f66ce0ac1ef90b716a28813a505322b17 
  src/test/java/org/apache/aurora/scheduler/state/TaskStateMachineTest.java e89e60afc107b299d572a07140d010040b2c7545 
  src/test/java/org/apache/aurora/scheduler/storage/StorageBackfillTest.java 724188bfa037caccaf91cf57d82de1777ab5de8e 

Diff: https://reviews.apache.org/r/16740/diff/


Testing
-------

./gradlew build


Thanks,

Bill Farner