You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@aurora.apache.org by Jordan Ly <jo...@gmail.com> on 2017/06/01 23:48:02 UTC

Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/
-----------------------------------------------------------

Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.


Bugs: AURORA-1931
    https://issues.apache.org/jira/browse/AURORA-1931


Repository: aurora


Description
-------

We have some services that require more than the current 10 seconds given to 
gracefully shutdown (they need to close resources, finish requests, etc).

We would like to be able to configure the amount of time we wait between each
stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
analysis.


Diffs
-----

  src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
  src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
  src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
  src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
  src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
  src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 


Diff: https://reviews.apache.org/r/59733/diff/1/


Testing
-------

Ran unit and integration tests.

Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.


Thanks,

Jordan Ly

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Jordan Ly <jo...@gmail.com>.


> On June 1, 2017, 11:53 p.m., David McLaughlin wrote:
> > src/main/python/apache/aurora/executor/http_lifecycle.py
> > Lines 37 (patched)
> > <https://reviews.apache.org/r/59733/diff/1/?file=1739103#file1739103line39>
> >
> >     Will this break for tasks that were deployed before the client was updated? Should we also inject a default here?

Good point! I definitely need to add a default to avoid this case.


- Jordan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review176705
-----------------------------------------------------------


On June 1, 2017, 11:48 p.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 1, 2017, 11:48 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/1/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by David McLaughlin <da...@dmclaughlin.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review176705
-----------------------------------------------------------




src/main/python/apache/aurora/executor/http_lifecycle.py
Lines 37 (patched)
<https://reviews.apache.org/r/59733/#comment250135>

    Will this break for tasks that were deployed before the client was updated? Should we also inject a default here?


- David McLaughlin


On June 1, 2017, 11:48 p.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 1, 2017, 11:48 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/1/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Santhosh Kumar Shanmugham <sa...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review176707
-----------------------------------------------------------



Can you add a new end-to-end test that exercises this feature?


src/main/python/apache/aurora/executor/http_lifecycle.py
Lines 37 (patched)
<https://reviews.apache.org/r/59733/#comment250138>

    +1


- Santhosh Kumar Shanmugham


On June 1, 2017, 4:48 p.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 1, 2017, 4:48 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/1/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Zameer Manji <zm...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review177092
-----------------------------------------------------------


Ship it!




Ship It!

- Zameer Manji


On June 6, 2017, 1:02 p.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 6, 2017, 1:02 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   RELEASE-NOTES.md c73643d669363a56e530c2d8c79d6ea06cc9415b 
>   docs/reference/task-lifecycle.md cf1b6794dca92756cee4a54f00f33fd3b1a481ea 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py c6c08983fb2d204afedd419798135d667169bce4 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py 38deae69dce7d88dab87f70e0128e0b86091fdef 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_executor.py e628ccd21f0588d0c76001b8f336ffa844c3ec35 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/3/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by David McLaughlin <da...@dmclaughlin.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review177090
-----------------------------------------------------------


Ship it!




Ship It!

- David McLaughlin


On June 6, 2017, 8:02 p.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 6, 2017, 8:02 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   RELEASE-NOTES.md c73643d669363a56e530c2d8c79d6ea06cc9415b 
>   docs/reference/task-lifecycle.md cf1b6794dca92756cee4a54f00f33fd3b1a481ea 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py c6c08983fb2d204afedd419798135d667169bce4 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py 38deae69dce7d88dab87f70e0128e0b86091fdef 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_executor.py e628ccd21f0588d0c76001b8f336ffa844c3ec35 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/3/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Aurora ReviewBot <wf...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review177139
-----------------------------------------------------------


Ship it!




Master (73234d0) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot retry"

- Aurora ReviewBot


On June 7, 2017, 1:06 a.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 7, 2017, 1:06 a.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   RELEASE-NOTES.md c73643d669363a56e530c2d8c79d6ea06cc9415b 
>   docs/reference/configuration.md 0040de17a85f5820c069474716678c66afce373b 
>   docs/reference/task-lifecycle.md cf1b6794dca92756cee4a54f00f33fd3b1a481ea 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py c6c08983fb2d204afedd419798135d667169bce4 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py 38deae69dce7d88dab87f70e0128e0b86091fdef 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_executor.py e628ccd21f0588d0c76001b8f336ffa844c3ec35 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/4/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Santhosh Kumar Shanmugham <sa...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review177764
-----------------------------------------------------------


Ship it!




LGTM. I will ship it after minor grammar fix.


docs/reference/configuration.md
Line 531 (original), 531 (patched)
<https://reviews.apache.org/r/59733/#comment251475>

    s/exceeds either of these either of these scheduler options/exceeds either of these scheduler options/


- Santhosh Kumar Shanmugham


On June 11, 2017, 11:37 p.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 11, 2017, 11:37 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   RELEASE-NOTES.md d14c4ad2a1bbdfdc845f1ec6dcd931c16b94569a 
>   docs/reference/configuration.md 0040de17a85f5820c069474716678c66afce373b 
>   docs/reference/task-lifecycle.md cf1b6794dca92756cee4a54f00f33fd3b1a481ea 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py c6c08983fb2d204afedd419798135d667169bce4 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py 38deae69dce7d88dab87f70e0128e0b86091fdef 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_executor.py e628ccd21f0588d0c76001b8f336ffa844c3ec35 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/5/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Aurora ReviewBot <wf...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review177770
-----------------------------------------------------------


Ship it!




Master (40d9d4d) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot retry"

- Aurora ReviewBot


On June 13, 2017, 5:58 p.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 13, 2017, 5:58 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   RELEASE-NOTES.md d14c4ad2a1bbdfdc845f1ec6dcd931c16b94569a 
>   docs/reference/configuration.md 0040de17a85f5820c069474716678c66afce373b 
>   docs/reference/task-lifecycle.md cf1b6794dca92756cee4a54f00f33fd3b1a481ea 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py c6c08983fb2d204afedd419798135d667169bce4 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py 38deae69dce7d88dab87f70e0128e0b86091fdef 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_executor.py e628ccd21f0588d0c76001b8f336ffa844c3ec35 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/6/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Jordan Ly <jo...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/
-----------------------------------------------------------

(Updated June 13, 2017, 5:58 p.m.)


Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.


Bugs: AURORA-1931
    https://issues.apache.org/jira/browse/AURORA-1931


Repository: aurora


Description
-------

We have some services that require more than the current 10 seconds given to 
gracefully shutdown (they need to close resources, finish requests, etc).

We would like to be able to configure the amount of time we wait between each
stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
analysis.


Diffs (updated)
-----

  RELEASE-NOTES.md d14c4ad2a1bbdfdc845f1ec6dcd931c16b94569a 
  docs/reference/configuration.md 0040de17a85f5820c069474716678c66afce373b 
  docs/reference/task-lifecycle.md cf1b6794dca92756cee4a54f00f33fd3b1a481ea 
  src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
  src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py c6c08983fb2d204afedd419798135d667169bce4 
  src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
  src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
  src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py 38deae69dce7d88dab87f70e0128e0b86091fdef 
  src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
  src/test/python/apache/aurora/executor/test_thermos_executor.py e628ccd21f0588d0c76001b8f336ffa844c3ec35 
  src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 


Diff: https://reviews.apache.org/r/59733/diff/6/

Changes: https://reviews.apache.org/r/59733/diff/5-6/


Testing
-------

Ran unit and integration tests.

Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.


Thanks,

Jordan Ly

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Aurora ReviewBot <wf...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review177645
-----------------------------------------------------------


Ship it!




Master (06518c8) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot retry"

- Aurora ReviewBot


On June 12, 2017, 6:37 a.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 12, 2017, 6:37 a.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   RELEASE-NOTES.md d14c4ad2a1bbdfdc845f1ec6dcd931c16b94569a 
>   docs/reference/configuration.md 0040de17a85f5820c069474716678c66afce373b 
>   docs/reference/task-lifecycle.md cf1b6794dca92756cee4a54f00f33fd3b1a481ea 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py c6c08983fb2d204afedd419798135d667169bce4 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py 38deae69dce7d88dab87f70e0128e0b86091fdef 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_executor.py e628ccd21f0588d0c76001b8f336ffa844c3ec35 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/5/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by David McLaughlin <da...@dmclaughlin.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review177659
-----------------------------------------------------------


Ship it!




Ship It!

- David McLaughlin


On June 12, 2017, 6:37 a.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 12, 2017, 6:37 a.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   RELEASE-NOTES.md d14c4ad2a1bbdfdc845f1ec6dcd931c16b94569a 
>   docs/reference/configuration.md 0040de17a85f5820c069474716678c66afce373b 
>   docs/reference/task-lifecycle.md cf1b6794dca92756cee4a54f00f33fd3b1a481ea 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py c6c08983fb2d204afedd419798135d667169bce4 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py 38deae69dce7d88dab87f70e0128e0b86091fdef 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_executor.py e628ccd21f0588d0c76001b8f336ffa844c3ec35 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/5/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Jordan Ly <jo...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review177639
-----------------------------------------------------------



@ReviewBot retry

- Jordan Ly


On June 12, 2017, 6:37 a.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 12, 2017, 6:37 a.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   RELEASE-NOTES.md d14c4ad2a1bbdfdc845f1ec6dcd931c16b94569a 
>   docs/reference/configuration.md 0040de17a85f5820c069474716678c66afce373b 
>   docs/reference/task-lifecycle.md cf1b6794dca92756cee4a54f00f33fd3b1a481ea 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py c6c08983fb2d204afedd419798135d667169bce4 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py 38deae69dce7d88dab87f70e0128e0b86091fdef 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_executor.py e628ccd21f0588d0c76001b8f336ffa844c3ec35 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/5/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Jordan Ly <jo...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/
-----------------------------------------------------------

(Updated June 12, 2017, 6:37 a.m.)


Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.


Changes
-------

Refactored `wait_escalation_time` into two fields for the separate endpoints, `graceful_shutdown_wait_secs` and `shutdown_wait_secs`.

Refactored the way the terminate cycle works. Previously it would wait the full escalation time before checking if the process was killed. Now, since the wait times are configurable and can be longer, we poll every 1 second up to the wait time in case the process was killed early.

Rewrote tests and documentation.

Tested on Vagrant devcluster, ran unit + integration tests.


Bugs: AURORA-1931
    https://issues.apache.org/jira/browse/AURORA-1931


Repository: aurora


Description
-------

We have some services that require more than the current 10 seconds given to 
gracefully shutdown (they need to close resources, finish requests, etc).

We would like to be able to configure the amount of time we wait between each
stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
analysis.


Diffs (updated)
-----

  RELEASE-NOTES.md d14c4ad2a1bbdfdc845f1ec6dcd931c16b94569a 
  docs/reference/configuration.md 0040de17a85f5820c069474716678c66afce373b 
  docs/reference/task-lifecycle.md cf1b6794dca92756cee4a54f00f33fd3b1a481ea 
  src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
  src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py c6c08983fb2d204afedd419798135d667169bce4 
  src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
  src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
  src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py 38deae69dce7d88dab87f70e0128e0b86091fdef 
  src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
  src/test/python/apache/aurora/executor/test_thermos_executor.py e628ccd21f0588d0c76001b8f336ffa844c3ec35 
  src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 


Diff: https://reviews.apache.org/r/59733/diff/5/

Changes: https://reviews.apache.org/r/59733/diff/4-5/


Testing
-------

Ran unit and integration tests.

Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.


Thanks,

Jordan Ly

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Jordan Ly <jo...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review177506
-----------------------------------------------------------



After some testing, I believe it would be better to expose two config fields instead of one, allowing the user to set timeouts for each of the shutdown endpoints independently.
It will be something like `graceful_shutdown_timeout_secs` and `shutdown_timeout_secs`. This will give greater control over the termination cycle and make it easier
to understand the configurations.

I will update this ticket in a few days with the changes. Please do not merge yet :)

- Jordan Ly


On June 7, 2017, 1:06 a.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 7, 2017, 1:06 a.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   RELEASE-NOTES.md c73643d669363a56e530c2d8c79d6ea06cc9415b 
>   docs/reference/configuration.md 0040de17a85f5820c069474716678c66afce373b 
>   docs/reference/task-lifecycle.md cf1b6794dca92756cee4a54f00f33fd3b1a481ea 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py c6c08983fb2d204afedd419798135d667169bce4 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py 38deae69dce7d88dab87f70e0128e0b86091fdef 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_executor.py e628ccd21f0588d0c76001b8f336ffa844c3ec35 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/4/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Jordan Ly <jo...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review177207
-----------------------------------------------------------



Please wait before merging, doing some additional testing

- Jordan Ly


On June 7, 2017, 1:06 a.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 7, 2017, 1:06 a.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   RELEASE-NOTES.md c73643d669363a56e530c2d8c79d6ea06cc9415b 
>   docs/reference/configuration.md 0040de17a85f5820c069474716678c66afce373b 
>   docs/reference/task-lifecycle.md cf1b6794dca92756cee4a54f00f33fd3b1a481ea 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py c6c08983fb2d204afedd419798135d667169bce4 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py 38deae69dce7d88dab87f70e0128e0b86091fdef 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_executor.py e628ccd21f0588d0c76001b8f336ffa844c3ec35 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/4/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Jordan Ly <jo...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/
-----------------------------------------------------------

(Updated June 7, 2017, 1:06 a.m.)


Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.


Changes
-------

Added additional documentation


Bugs: AURORA-1931
    https://issues.apache.org/jira/browse/AURORA-1931


Repository: aurora


Description
-------

We have some services that require more than the current 10 seconds given to 
gracefully shutdown (they need to close resources, finish requests, etc).

We would like to be able to configure the amount of time we wait between each
stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
analysis.


Diffs (updated)
-----

  RELEASE-NOTES.md c73643d669363a56e530c2d8c79d6ea06cc9415b 
  docs/reference/configuration.md 0040de17a85f5820c069474716678c66afce373b 
  docs/reference/task-lifecycle.md cf1b6794dca92756cee4a54f00f33fd3b1a481ea 
  src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
  src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py c6c08983fb2d204afedd419798135d667169bce4 
  src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
  src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
  src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py 38deae69dce7d88dab87f70e0128e0b86091fdef 
  src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
  src/test/python/apache/aurora/executor/test_thermos_executor.py e628ccd21f0588d0c76001b8f336ffa844c3ec35 
  src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 


Diff: https://reviews.apache.org/r/59733/diff/4/

Changes: https://reviews.apache.org/r/59733/diff/3-4/


Testing
-------

Ran unit and integration tests.

Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.


Thanks,

Jordan Ly

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Stephan Erb <se...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review177071
-----------------------------------------------------------


Ship it!




LGTM. A few minor documentation remarks below.


docs/reference/task-lifecycle.md
Lines 87 (patched)
<https://reviews.apache.org/r/59733/#comment250633>

    Please mention this field in https://github.com/apache/aurora/blob/master/docs/reference/configuration.md#httplifecycleconfig-objects as well. This is also a natural place where you could pick up the correspondance to the transient task state timeout.



src/main/python/apache/aurora/executor/aurora_executor.py
Line 217 (original), 218 (patched)
<https://reviews.apache.org/r/59733/#comment250637>

    Note to self: Took me a moment to realize that this changes the timeout of the deadline, but _not_ the timeout of the stop() function. This remains at 60 seconds. This also implies the timeout between SIGKILL and SIGTERM remains at 60 seconds.



src/main/python/apache/aurora/executor/bin/thermos_executor_main.py
Lines 196-197 (patched)
<https://reviews.apache.org/r/59733/#comment250638>

    Please mention the HttpLifecycleConfig here explicitly. A casual reader could think of SIGKILL and SIGTERM when hearing forceful and graceful termination.


- Stephan Erb


On June 6, 2017, 10:02 p.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 6, 2017, 10:02 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   RELEASE-NOTES.md c73643d669363a56e530c2d8c79d6ea06cc9415b 
>   docs/reference/task-lifecycle.md cf1b6794dca92756cee4a54f00f33fd3b1a481ea 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py c6c08983fb2d204afedd419798135d667169bce4 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py 38deae69dce7d88dab87f70e0128e0b86091fdef 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_executor.py e628ccd21f0588d0c76001b8f336ffa844c3ec35 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/3/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Aurora ReviewBot <wf...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review177089
-----------------------------------------------------------


Ship it!




Master (2cbaeec) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot retry"

- Aurora ReviewBot


On June 6, 2017, 8:02 p.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 6, 2017, 8:02 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   RELEASE-NOTES.md c73643d669363a56e530c2d8c79d6ea06cc9415b 
>   docs/reference/task-lifecycle.md cf1b6794dca92756cee4a54f00f33fd3b1a481ea 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py c6c08983fb2d204afedd419798135d667169bce4 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py 38deae69dce7d88dab87f70e0128e0b86091fdef 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_executor.py e628ccd21f0588d0c76001b8f336ffa844c3ec35 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/3/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Jordan Ly <jo...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/
-----------------------------------------------------------

(Updated June 6, 2017, 8:02 p.m.)


Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.


Changes
-------

Fixed build style


Bugs: AURORA-1931
    https://issues.apache.org/jira/browse/AURORA-1931


Repository: aurora


Description
-------

We have some services that require more than the current 10 seconds given to 
gracefully shutdown (they need to close resources, finish requests, etc).

We would like to be able to configure the amount of time we wait between each
stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
analysis.


Diffs (updated)
-----

  RELEASE-NOTES.md c73643d669363a56e530c2d8c79d6ea06cc9415b 
  docs/reference/task-lifecycle.md cf1b6794dca92756cee4a54f00f33fd3b1a481ea 
  src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
  src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py c6c08983fb2d204afedd419798135d667169bce4 
  src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
  src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
  src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py 38deae69dce7d88dab87f70e0128e0b86091fdef 
  src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
  src/test/python/apache/aurora/executor/test_thermos_executor.py e628ccd21f0588d0c76001b8f336ffa844c3ec35 
  src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 


Diff: https://reviews.apache.org/r/59733/diff/3/

Changes: https://reviews.apache.org/r/59733/diff/2-3/


Testing
-------

Ran unit and integration tests.

Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.


Thanks,

Jordan Ly

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Aurora ReviewBot <wf...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review177065
-----------------------------------------------------------



Master (2cbaeec) is red with this patch.
  ./build-support/jenkins/build.sh

  Running setup.py bdist_wheel for twitter.common.log: finished with status 'done'
  Stored in directory: /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pip/wheels/80/e3/3a/297d8950fcbd822ab5a6a62175818fab38b668cc5a86dbd0b0
  Running setup.py bdist_wheel for pycparser: started
  Running setup.py bdist_wheel for pycparser: finished with status 'done'
  Stored in directory: /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pip/wheels/a8/0b/41/dc95621f9d3a0da7bc191b8a71f0e8182ffd3cc5f33ac55005
  Running setup.py bdist_wheel for twitter.common.options: started
  Running setup.py bdist_wheel for twitter.common.options: finished with status 'done'
  Stored in directory: /home/jenkins/jenkins-slave/workspace/AuroraBot/.home/.cache/pip/wheels/17/41/80/c4811d8c1c7ca7007e520c3399872fc340f45c3a26a6a23e6a
Successfully built pantsbuild.pants twitter.common.collections setproctitle ansicolors pathspec scandir twitter.common.dirutil psutil pystache docutils Markdown Pygments twitter.common.confluence coverage lmdb pywatchman twitter.common.lang twitter.common.log pycparser twitter.common.options
Installing collected packages: twitter.common.lang, twitter.common.collections, setproctitle, setuptools, six, ansicolors, pyparsing, packaging, pathspec, scandir, twitter.common.dirutil, psutil, requests, pystache, pex, docutils, Markdown, Pygments, twitter.common.options, twitter.common.log, twitter.common.confluence, monotonic, fasteners, coverage, pycparser, cffi, lmdb, pywatchman, futures, pantsbuild.pants
  Found existing installation: setuptools 21.2.1
    Uninstalling setuptools-21.2.1:
      Successfully uninstalled setuptools-21.2.1
Successfully installed Markdown-2.1.1 Pygments-1.4 ansicolors-1.0.2 cffi-1.7.0 coverage-3.7.1 docutils-0.12 fasteners-0.14.1 futures-3.0.5 lmdb-0.89 monotonic-1.3 packaging-16.7 pantsbuild.pants-1.3.0.dev3 pathspec-0.3.4 pex-1.1.16 psutil-4.3.0 pycparser-2.17 pyparsing-2.2.0 pystache-0.5.3 pywatchman-1.3.0 requests-2.5.3 scandir-1.2 setproctitle-1.1.10 setuptools-30.0.0 six-1.10.0 twitter.common.collections-0.3.9 twitter.common.confluence-0.3.9 twitter.common.dirutil-0.3.9 twitter.common.lang-0.3.9 twitter.common.log-0.3.9 twitter.common.options-0.3.9
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

18:17:52 00:00 [main]
               (To run a reporting server: ./pants server)
18:17:52 00:00   [setup]
18:17:52 00:00     [parse][32m
               Executing tasks in goals: compile[0m
18:17:52 00:00   [compile]
18:17:52 00:00     [compile-prep-command]
18:17:52 00:00       [prep_command]
18:17:56 00:04     [compile]
18:17:56 00:04     [python-eval]
18:17:56 00:04     [pythonstyle]
18:17:56 00:04       [cache]                                          [32m
                   No cached artifacts for 42 targets.[0m[32m
                   Invalidated 42 targets.[0m
T301:ERROR   src/test/python/apache/aurora/executor/test_thermos_executor.py:401 Expected 1 blank lines, found 0
     |      def __eq__(self, other):


FAILURE: 1 Python Style issues found. For import order related issues, please try `./pants fmt.isort <targets>`


18:18:20 00:28   [complete][31m
               FAILURE[0m


I will refresh this build result if you post a review containing "@ReviewBot retry"

- Aurora ReviewBot


On June 6, 2017, 5:47 p.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 6, 2017, 5:47 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   RELEASE-NOTES.md c73643d669363a56e530c2d8c79d6ea06cc9415b 
>   docs/reference/task-lifecycle.md cf1b6794dca92756cee4a54f00f33fd3b1a481ea 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/bin/thermos_executor_main.py c6c08983fb2d204afedd419798135d667169bce4 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py 38deae69dce7d88dab87f70e0128e0b86091fdef 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_executor.py e628ccd21f0588d0c76001b8f336ffa844c3ec35 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/2/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Jordan Ly <jo...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/
-----------------------------------------------------------

(Updated June 6, 2017, 5:47 p.m.)


Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.


Changes
-------

Added default for `escalation_wait_secs` in `http_lifecycle.py` if the field is not present in the config.
Default is 5 seconds (the current value)

Added the ability to specify maximum stop timeout for the executor through the flag `stop_timeout_in_secs`.
Default is 120 seconds (the current value).

Added test to ensure the new executor flag is used when creating the executor, and that the stop_timeout is
used when killing a task.

Added/updated documentation.


Bugs: AURORA-1931
    https://issues.apache.org/jira/browse/AURORA-1931


Repository: aurora


Description
-------

We have some services that require more than the current 10 seconds given to 
gracefully shutdown (they need to close resources, finish requests, etc).

We would like to be able to configure the amount of time we wait between each
stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
analysis.


Diffs (updated)
-----

  RELEASE-NOTES.md c73643d669363a56e530c2d8c79d6ea06cc9415b 
  docs/reference/task-lifecycle.md cf1b6794dca92756cee4a54f00f33fd3b1a481ea 
  src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
  src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
  src/main/python/apache/aurora/executor/bin/thermos_executor_main.py c6c08983fb2d204afedd419798135d667169bce4 
  src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
  src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
  src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py 38deae69dce7d88dab87f70e0128e0b86091fdef 
  src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
  src/test/python/apache/aurora/executor/test_thermos_executor.py e628ccd21f0588d0c76001b8f336ffa844c3ec35 
  src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 


Diff: https://reviews.apache.org/r/59733/diff/2/

Changes: https://reviews.apache.org/r/59733/diff/1-2/


Testing
-------

Ran unit and integration tests.

Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.


Thanks,

Jordan Ly

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Zameer Manji <zm...@apache.org>.


> On June 2, 2017, 10:28 a.m., Zameer Manji wrote:
> > I don't have comment access on the doc, so I will leave my questions here:
> > 
> > 1. Should operators (via executor flags) be able to cap this value? That is should `STOP_TIMEOUT` be an operator flag?
> > 2. Should the client cap this value and display an error message to the user?
> > 3. AFAIK, a task can remain in `KILLING` forever. There is no timeout in the scheduler, as it just retries kills. If a user puts a large value here, I'm not sure tasks will actually termiante. Please add an e2e test here to confirm/deny.
> 
> David McLaughlin wrote:
>     For (3), this is exactly what STOP_TIMEOUT in the executor is for. 
>     
>     The issue for STOP_TIMEOUT is it is Thermos-specific and we support multiple executors.
> 
> David McLaughlin wrote:
>     Sorry to be clear: for (1), the issue is we support multiple executors. So we'd need some generic way of passing parameters from the scheduler to the executor. Or to be honest, I don't think the Scheduler should really get to make this decision. What we need is some way of overriding all the magic strings and numbers littered throughout Thermos. But I think this is a separate ticket. All we're really doing here is bumping up a constant timeout from 2 minutes to 5 minutes.
> 
> Jordan Ly wrote:
>     On (3), David is correct: STOP_TIMEOUT is essentially a hard limit on how long the KILLING process can go on for. An e2e test might be disruptive since you would have to wait the full 5 minutes for the timeout to be hit --- unless there is another way I am missing :)
>     
>     Could you elaborate on what you mean in (2)?

For (1) we do have a generic way of passing parameters from the scheduler to the executor. It is called `thermos_executor_flags` for legacy reasons but it can pass arbitrary arguments to the executor.

(3) Thanks for clearing up what `STOP_TIMEOUT` does. 

I have no objection to adding a 5m e2e test. I think capturing this edge case behaviour would be worth while.

I was just thinking here that since users can specify a value here, it would be nice to control what the cap is. In the linked design it mentions a 10s long shutdown, but I can think of valid cases where this value should be 2m or 6m.

It would be nice to add a thermos flag `--maximum_wait_escalation_time` that one could set in the scheduler `thermos_executor_flags`

I don't feel too strong about this flag as I could just add it later. However, I do feel that an e2e test of some kind here is required to ensure we never regress.


- Zameer


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review176802
-----------------------------------------------------------


On June 1, 2017, 4:48 p.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 1, 2017, 4:48 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/1/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by David McLaughlin <da...@dmclaughlin.com>.


> On June 2, 2017, 5:28 p.m., Zameer Manji wrote:
> > I don't have comment access on the doc, so I will leave my questions here:
> > 
> > 1. Should operators (via executor flags) be able to cap this value? That is should `STOP_TIMEOUT` be an operator flag?
> > 2. Should the client cap this value and display an error message to the user?
> > 3. AFAIK, a task can remain in `KILLING` forever. There is no timeout in the scheduler, as it just retries kills. If a user puts a large value here, I'm not sure tasks will actually termiante. Please add an e2e test here to confirm/deny.
> 
> David McLaughlin wrote:
>     For (3), this is exactly what STOP_TIMEOUT in the executor is for. 
>     
>     The issue for STOP_TIMEOUT is it is Thermos-specific and we support multiple executors.
> 
> David McLaughlin wrote:
>     Sorry to be clear: for (1), the issue is we support multiple executors. So we'd need some generic way of passing parameters from the scheduler to the executor. Or to be honest, I don't think the Scheduler should really get to make this decision. What we need is some way of overriding all the magic strings and numbers littered throughout Thermos. But I think this is a separate ticket. All we're really doing here is bumping up a constant timeout from 2 minutes to 5 minutes.
> 
> Jordan Ly wrote:
>     On (3), David is correct: STOP_TIMEOUT is essentially a hard limit on how long the KILLING process can go on for. An e2e test might be disruptive since you would have to wait the full 5 minutes for the timeout to be hit --- unless there is another way I am missing :)
>     
>     Could you elaborate on what you mean in (2)?
> 
> Zameer Manji wrote:
>     For (1) we do have a generic way of passing parameters from the scheduler to the executor. It is called `thermos_executor_flags` for legacy reasons but it can pass arbitrary arguments to the executor.
>     
>     (3) Thanks for clearing up what `STOP_TIMEOUT` does. 
>     
>     I have no objection to adding a 5m e2e test. I think capturing this edge case behaviour would be worth while.
>     
>     I was just thinking here that since users can specify a value here, it would be nice to control what the cap is. In the linked design it mentions a 10s long shutdown, but I can think of valid cases where this value should be 2m or 6m.
>     
>     It would be nice to add a thermos flag `--maximum_wait_escalation_time` that one could set in the scheduler `thermos_executor_flags`
>     
>     I don't feel too strong about this flag as I could just add it later. However, I do feel that an e2e test of some kind here is required to ensure we never regress.

If the community is fine with just using thermos_executor_flags and leaving custom executors to deal with this themselves, then I'm +1 that approach too.

Pretty strong -1 to a 5 minute e2e test though.


- David


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review176802
-----------------------------------------------------------


On June 1, 2017, 11:48 p.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 1, 2017, 11:48 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/1/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by David McLaughlin <da...@dmclaughlin.com>.


> On June 2, 2017, 5:28 p.m., Zameer Manji wrote:
> > I don't have comment access on the doc, so I will leave my questions here:
> > 
> > 1. Should operators (via executor flags) be able to cap this value? That is should `STOP_TIMEOUT` be an operator flag?
> > 2. Should the client cap this value and display an error message to the user?
> > 3. AFAIK, a task can remain in `KILLING` forever. There is no timeout in the scheduler, as it just retries kills. If a user puts a large value here, I'm not sure tasks will actually termiante. Please add an e2e test here to confirm/deny.
> 
> David McLaughlin wrote:
>     For (3), this is exactly what STOP_TIMEOUT in the executor is for. 
>     
>     The issue for STOP_TIMEOUT is it is Thermos-specific and we support multiple executors.

Sorry to be clear: for (1), the issue is we support multiple executors. So we'd need some generic way of passing parameters from the scheduler to the executor. Or to be honest, I don't think the Scheduler should really get to make this decision. What we need is some way of overriding all the magic strings and numbers littered throughout Thermos. But I think this is a separate ticket. All we're really doing here is bumping up a constant timeout from 2 minutes to 5 minutes.


- David


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review176802
-----------------------------------------------------------


On June 1, 2017, 11:48 p.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 1, 2017, 11:48 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/1/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Jordan Ly <jo...@gmail.com>.


> On June 2, 2017, 5:28 p.m., Zameer Manji wrote:
> > I don't have comment access on the doc, so I will leave my questions here:
> > 
> > 1. Should operators (via executor flags) be able to cap this value? That is should `STOP_TIMEOUT` be an operator flag?
> > 2. Should the client cap this value and display an error message to the user?
> > 3. AFAIK, a task can remain in `KILLING` forever. There is no timeout in the scheduler, as it just retries kills. If a user puts a large value here, I'm not sure tasks will actually termiante. Please add an e2e test here to confirm/deny.
> 
> David McLaughlin wrote:
>     For (3), this is exactly what STOP_TIMEOUT in the executor is for. 
>     
>     The issue for STOP_TIMEOUT is it is Thermos-specific and we support multiple executors.
> 
> David McLaughlin wrote:
>     Sorry to be clear: for (1), the issue is we support multiple executors. So we'd need some generic way of passing parameters from the scheduler to the executor. Or to be honest, I don't think the Scheduler should really get to make this decision. What we need is some way of overriding all the magic strings and numbers littered throughout Thermos. But I think this is a separate ticket. All we're really doing here is bumping up a constant timeout from 2 minutes to 5 minutes.

On (3), David is correct: STOP_TIMEOUT is essentially a hard limit on how long the KILLING process can go on for. An e2e test might be disruptive since you would have to wait the full 5 minutes for the timeout to be hit --- unless there is another way I am missing :)

Could you elaborate on what you mean in (2)?


- Jordan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review176802
-----------------------------------------------------------


On June 1, 2017, 11:48 p.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 1, 2017, 11:48 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/1/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by David McLaughlin <da...@dmclaughlin.com>.


> On June 2, 2017, 5:28 p.m., Zameer Manji wrote:
> > I don't have comment access on the doc, so I will leave my questions here:
> > 
> > 1. Should operators (via executor flags) be able to cap this value? That is should `STOP_TIMEOUT` be an operator flag?
> > 2. Should the client cap this value and display an error message to the user?
> > 3. AFAIK, a task can remain in `KILLING` forever. There is no timeout in the scheduler, as it just retries kills. If a user puts a large value here, I'm not sure tasks will actually termiante. Please add an e2e test here to confirm/deny.

For (3), this is exactly what STOP_TIMEOUT in the executor is for. 

The issue for STOP_TIMEOUT is it is Thermos-specific and we support multiple executors.


- David


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review176802
-----------------------------------------------------------


On June 1, 2017, 11:48 p.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 1, 2017, 11:48 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/1/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Zameer Manji <zm...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review176802
-----------------------------------------------------------



I don't have comment access on the doc, so I will leave my questions here:

1. Should operators (via executor flags) be able to cap this value? That is should `STOP_TIMEOUT` be an operator flag?
2. Should the client cap this value and display an error message to the user?
3. AFAIK, a task can remain in `KILLING` forever. There is no timeout in the scheduler, as it just retries kills. If a user puts a large value here, I'm not sure tasks will actually termiante. Please add an e2e test here to confirm/deny.

- Zameer Manji


On June 1, 2017, 4:48 p.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 1, 2017, 4:48 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/1/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Reza Motamedi <re...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review176717
-----------------------------------------------------------




src/main/python/apache/aurora/executor/http_lifecycle.py
Lines 37 (patched)
<https://reviews.apache.org/r/59733/#comment250153>

    +1


- Reza Motamedi


On June 1, 2017, 11:48 p.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 1, 2017, 11:48 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/1/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Santhosh Kumar Shanmugham <sa...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review176945
-----------------------------------------------------------


Ship it!




Ship It!

- Santhosh Kumar Shanmugham


On June 1, 2017, 4:48 p.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 1, 2017, 4:48 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/1/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Aurora ReviewBot <wf...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review176708
-----------------------------------------------------------


Ship it!




Master (e76862a) is green with this patch.
  ./build-support/jenkins/build.sh

I will refresh this build result if you post a review containing "@ReviewBot retry"

- Aurora ReviewBot


On June 1, 2017, 4:48 p.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 1, 2017, 4:48 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/1/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Stephan Erb <se...@apache.org>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review177054
-----------------------------------------------------------



I will try to have a look at this until tomorrow. Sorry for the delay.

- Stephan Erb


On June 2, 2017, 1:48 a.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 2, 2017, 1:48 a.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/1/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>

Re: Review Request 59733: Adding Configurable Wait Period for Graceful Shutdowns

Posted by Reza Motamedi <re...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59733/#review177058
-----------------------------------------------------------


Ship it!




Ship It!

- Reza Motamedi


On June 1, 2017, 11:48 p.m., Jordan Ly wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59733/
> -----------------------------------------------------------
> 
> (Updated June 1, 2017, 11:48 p.m.)
> 
> 
> Review request for Aurora, David McLaughlin, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
> 
> 
> Bugs: AURORA-1931
>     https://issues.apache.org/jira/browse/AURORA-1931
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> We have some services that require more than the current 10 seconds given to 
> gracefully shutdown (they need to close resources, finish requests, etc).
> 
> We would like to be able to configure the amount of time we wait between each
> stage of the graceful shutdown sequence. See this [proposal](https://docs.google.com/document/d/1Sl-KWNyt1j0nIndinqfJsH3pkUY5IYXfGWyLHU2wacs/edit?usp=sharing) for a more in-depth
> analysis.
> 
> 
> Diffs
> -----
> 
>   src/main/python/apache/aurora/config/schema/base.py b2692a648645a195a24491e4978fb833c6c20be8 
>   src/main/python/apache/aurora/executor/aurora_executor.py 81461cb49ac223f3bdfa59e8c59e150a07771dea 
>   src/main/python/apache/aurora/executor/http_lifecycle.py 9280bf29da9bda1691adbf3a4c34c4f3d4900517 
>   src/test/python/apache/aurora/client/cli/test_inspect.py 4a23c5984c2d093e2f53e93aec71418f84b65928 
>   src/test/python/apache/aurora/executor/test_http_lifecycle.py a967e3410a4d2dc2e1721f505a4d76da9209d177 
>   src/test/python/apache/aurora/executor/test_thermos_task_runner.py 1b92667bceabc8ea1540122477a51cb58ea2ae36 
> 
> 
> Diff: https://reviews.apache.org/r/59733/diff/1/
> 
> 
> Testing
> -------
> 
> Ran unit and integration tests.
> 
> Created and killed jobs with varying wait_escalation_secs values on the Vagrant devcluster.
> 
> 
> Thanks,
> 
> Jordan Ly
> 
>