You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by David McLaughlin <da...@dmclaughlin.com> on 2017/02/14 01:36:14 UTC
Review Request 56629: Expose task pruning endpoint in aurora_admin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/
-----------------------------------------------------------
Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
Repository: aurora
Description
-------
Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
Diffs
-----
api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
src/main/java/org/apache/aurora/scheduler/thrift/aop/AnnotatedAuroraAdmin.java d63b11d36748de796e96685622a3290bb94e7106
src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
Diff: https://reviews.apache.org/r/56629/diff/
Testing
-------
Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
Also:
./pants test src/test/python/apache/aurora/admin::
Thanks,
David McLaughlin
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165421
-----------------------------------------------------------
Master (40d91fe) is red with this patch.
./build-support/jenkins/build.sh
:commons:generateThriftResources
:commons:processResources
:commons:classes
:commons:jar
:compileJava/home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/storage/log/WriteAheadStorage.java:74: Note: Wrote forwarder org.apache.aurora.scheduler.storage.log.WriteAheadStorageForwarder
@Forward({
^
Note: Writing file:/home/jenkins/jenkins-slave/workspace/AuroraBot/dist/classes/main/org/apache/aurora/common/args/apt/cmdline.arg.info.txt.2
Note: Writing file:/home/jenkins/jenkins-slave/workspace/AuroraBot/dist/classes/main/META-INF/compiler/resource-mappings/org.apache.aurora.common.args.apt.CmdLineProcessor
:generateBuildProperties
:processResources
:classes
:jar
:startScripts
:distTar
:distZip
:assemble
:compileJmhJavaNote: /home/jenkins/jenkins-slave/workspace/AuroraBot/src/jmh/java/org/apache/aurora/benchmark/fakes/FakeSchedulerDriver.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
:processJmhResources UP-TO-DATE
:jmhClasses
:checkstyleJmh
:jsHint
:checkstyleMain[ant:checkstyle] [ERROR] /home/jenkins/jenkins-slave/workspace/AuroraBot/src/main/java/org/apache/aurora/scheduler/thrift/aop/AnnotatedAuroraAdmin.java:27:8: Unused import - org.apache.aurora.gen.TaskQuery. [UnusedImports]
FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':checkstyleMain'.
> Checkstyle rule violations were found. See the report at: file:///home/jenkins/jenkins-slave/workspace/AuroraBot/dist/reports/checkstyle/main.html
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
BUILD FAILED
Total time: 1 mins 22.991 secs
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Feb. 14, 2017, 1:36 a.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 1:36 a.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/java/org/apache/aurora/scheduler/thrift/aop/AnnotatedAuroraAdmin.java d63b11d36748de796e96685622a3290bb94e7106
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by David McLaughlin <da...@dmclaughlin.com>.
> On Feb. 14, 2017, 3:48 a.m., Mehrdad Nurolahzade wrote:
> > src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java, line 1135
> > <https://reviews.apache.org/r/56629/diff/2/?file=1632701#file1632701line1135>
> >
> > We are skipping task delete through `StateManager` which publishes `PubsubEvent.TasksDeleted` to subscribers like `TaskGroups` and `JobUpdateController`.
> >
> > In the case of `JobUpdateController` the dependency does not seem to matter. Did you verify ignoring the `TaskGroups` dependency is safe?
Looks like I need to go through StateManager for TaskVars. The other subscribers only deal with PENDING tasks.
It's a shame because the pubsub path is really ineffecient.
- David
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165434
-----------------------------------------------------------
On Feb. 14, 2017, 4:12 a.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 4:12 a.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
> src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by Mehrdad Nurolahzade <me...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165434
-----------------------------------------------------------
src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java (line 1135)
<https://reviews.apache.org/r/56629/#comment237283>
We are skipping task delete through `StateManager` which publishes `PubsubEvent.TasksDeleted` to subscribers like `TaskGroups` and `JobUpdateController`.
In the case of `JobUpdateController` the dependency does not seem to matter. Did you verify ignoring the `TaskGroups` dependency is safe?
- Mehrdad Nurolahzade
On Feb. 13, 2017, 6:01 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 13, 2017, 6:01 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165428
-----------------------------------------------------------
Master (40d91fe) is red with this patch.
./build-support/jenkins/build.sh
src/test/python/apache/thermos/common/test_task_planner.py::test_task_process_cannot_depend_upon_daemon [32mPASSED[0m
src/test/python/apache/thermos/common/test_task_planner.py::test_task_non_ephemeral_process_cannot_depend_on_ephemeral_process [32mPASSED[0m
src/test/python/apache/thermos/common/test_task_planner.py::test_task_failed_predecessor_does_not_make_process_runnable [32mPASSED[0m
src/test/python/apache/thermos/common/test_task_planner.py::test_task_daemon_duration [32mPASSED[0m
src/test/python/apache/thermos/common/test_task_planner.py::test_task_waits [32mPASSED[0m
src/test/python/apache/thermos/common/test_task_planner.py::test_task_fails [32mPASSED[0m
src/test/python/apache/thermos/common/test_task_planner.py::test_task_lost [32mPASSED[0m
src/test/python/apache/thermos/common/test_task_planner.py::test_task_filters [32mPASSED[0m
src/test/python/apache/thermos/common/test_task_planner.py::test_task_max_runs [32mPASSED[0m
src/test/python/apache/thermos/common/test_pathspec.py::test_legacy_task_roots [32mPASSED[0m
src/test/python/apache/thermos/common/test_pathspec.py::test_legacy_log_dirs [32mPASSED[0m
src/test/python/apache/thermos/common/test_pathspec.py::test_exception_on_none_keys [32mPASSED[0m
src/test/python/apache/thermos/common/test_planner.py::test_planner_empty [32mPASSED[0m
src/test/python/apache/thermos/common/test_planner.py::test_planner_unordered [32mPASSED[0m
src/test/python/apache/thermos/common/test_planner.py::test_planner_ordered [32mPASSED[0m
src/test/python/apache/thermos/common/test_planner.py::test_planner_mixed [32mPASSED[0m
src/test/python/apache/thermos/common/test_planner.py::test_planner_unsatisfiables [32mPASSED[0m
==================== FAILURES ====================
_________________ test_coverage __________________
[1m def test_coverage():[0m
[1m """Make sure a new thrift RPC doesn't get added without minimal test coverage."""[0m
[1m for name, klass in inspect.getmembers(AuroraAdmin) + inspect.getmembers(AuroraSchedulerManager):[0m
[1m if name.endswith('_args'):[0m
[1m rpc_name = name[:-len('_args')][0m
[1m> assert hasattr(TestSchedulerProxyAdminInjection, 'test_%s' % rpc_name), ([0m
[1m 'No test defined for RPC %s' % rpc_name)[0m
[1m[31mE AssertionError: No test defined for RPC pruneTasks[0m
[1m[31mE assert hasattr(TestSchedulerProxyAdminInjection, ('test_%s' % 'pruneTasks'))[0m
src/test/python/apache/aurora/client/api/test_scheduler_client.py:65: AssertionError
generated xml file: /home/jenkins/jenkins-slave/workspace/AuroraBot/dist/test-results/aaf4d108c31293299a0839bdc404a91802f80937.xml
[1m[31m 1 failed, 777 passed, 6 skipped, 1 warnings in 326.98 seconds [0m
FAILURE
02:23:08 06:04 [complete][31m
FAILURE[0m
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Feb. 14, 2017, 2:01 a.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 2:01 a.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by Santhosh Kumar Shanmugham <sa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165447
-----------------------------------------------------------
src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java (lines 1130 - 1134)
<https://reviews.apache.org/r/56629/#comment237292>
Limit needs to be applied at the Store, both `MemTaskStore` and `DBTaskStore` do not appear to use the `limit` and `offset` filters. We can fix this in another RB.
- Santhosh Kumar Shanmugham
On Feb. 13, 2017, 9:06 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 13, 2017, 9:06 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
> src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by David McLaughlin <da...@dmclaughlin.com>.
> On Feb. 14, 2017, 7:34 p.m., Zameer Manji wrote:
> > Linking this to a ticket would be nice for posterity.
> >
> > This seems like a useful admin tool. Thanks for adding it. I would probably use this in cases where some large jobs go flapping and populate the DB with thousands of terminal tasks.
Filed AURORA-1893.
> On Feb. 14, 2017, 7:34 p.m., Zameer Manji wrote:
> > src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java, line 1130
> > <https://reviews.apache.org/r/56629/diff/7/?file=1632961#file1632961line1130>
> >
> > Could you refer to AURORA-1892?
Done.
- David
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165562
-----------------------------------------------------------
On Feb. 14, 2017, 9:13 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 9:13 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Bugs: AURORA-1893
> https://issues.apache.org/jira/browse/AURORA-1893
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md 3e9880249aa40215a08424b58f85a634337cc67e
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
> src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by Zameer Manji <zm...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165562
-----------------------------------------------------------
Ship it!
Linking this to a ticket would be nice for posterity.
This seems like a useful admin tool. Thanks for adding it. I would probably use this in cases where some large jobs go flapping and populate the DB with thousands of terminal tasks.
src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java (line 1130)
<https://reviews.apache.org/r/56629/#comment237421>
Could you refer to AURORA-1892?
- Zameer Manji
On Feb. 13, 2017, 9:41 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 13, 2017, 9:41 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md 3e9880249aa40215a08424b58f85a634337cc67e
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
> src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by Reza Motamedi <re...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165565
-----------------------------------------------------------
Ship it!
Ship It!
- Reza Motamedi
On Feb. 14, 2017, 5:41 a.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 5:41 a.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md 3e9880249aa40215a08424b58f85a634337cc67e
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
> src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165460
-----------------------------------------------------------
Ship it!
Master (40d91fe) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Feb. 14, 2017, 5:41 a.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 5:41 a.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md 3e9880249aa40215a08424b58f85a634337cc67e
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
> src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165577
-----------------------------------------------------------
Master (40d91fe) is red with this patch.
./build-support/jenkins/build.sh
Test coverage missing for org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl
Test coverage missing for org/apache/aurora/scheduler/storage/log/EntrySerializer$EntrySerializerImpl$1
Test coverage missing for org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl$8
Test coverage missing for org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl$7
Test coverage missing for org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl$4
Test coverage missing for org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl$3
Test coverage missing for org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl$6
Test coverage missing for org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl$5
Test coverage missing for org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl$2
Test coverage missing for org/apache/aurora/scheduler/storage/log/SnapshotStoreImpl$1
Test coverage missing for org/apache/aurora/scheduler/storage/log/LogStorage$Settings
Test coverage missing for org/apache/aurora/scheduler/storage/log/LogStorage$ScheduledExecutorSchedulingService
Test coverage missing for org/apache/aurora/scheduler/storage/log/LogStorageModule
Test coverage missing for org/apache/aurora/scheduler/storage/backup/TemporaryStorage$TemporaryStorageFactory$1
Test coverage missing for org/apache/aurora/scheduler/storage/backup/BackupModule
Test coverage missing for org/apache/aurora/scheduler/storage/backup/Recovery$RecoveryImpl
Test coverage missing for org/apache/aurora/scheduler/storage/backup/TemporaryStorage$TemporaryStorageFactory
Test coverage missing for org/apache/aurora/scheduler/storage/backup/Recovery$RecoveryImpl$PendingRecovery
Test coverage missing for org/apache/aurora/scheduler/TaskVars
Test coverage missing for org/apache/aurora/scheduler/SchedulerLifecycle$DefaultDelayedActions
Test coverage missing for org/apache/aurora/scheduler/TierManager$TierManagerImpl$TierConfig
Test coverage missing for org/apache/aurora/scheduler/TaskVars$Counter
Test coverage missing for org/apache/aurora/scheduler/TaskVars$1
Test coverage missing for org/apache/aurora/scheduler/SchedulerModule$TaskEventBatchWorker
Test coverage missing for org/apache/aurora/scheduler/HostOffer$1
Test coverage missing for org/apache/aurora/scheduler/SchedulerModule
Test coverage missing for org/apache/aurora/scheduler/TaskIdGenerator$TaskIdGeneratorImpl
Test coverage missing for org/apache/aurora/scheduler/SchedulerModule$1
Test coverage missing for org/apache/aurora/scheduler/TaskStatusHandlerImpl
Test coverage missing for org/apache/aurora/scheduler/TaskStatusHandlerImpl$1
Test coverage missing for org/apache/aurora/scheduler/TierModule
Test coverage missing for org/apache/aurora/scheduler/storage/db/typehandlers/VolumeModeTypeHandler
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
==============================================================================
BUILD FAILED
Total time: 6 mins 30.931 secs
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Feb. 14, 2017, 9:13 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 9:13 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Bugs: AURORA-1893
> https://issues.apache.org/jira/browse/AURORA-1893
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md 3e9880249aa40215a08424b58f85a634337cc67e
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
> src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by David McLaughlin <da...@dmclaughlin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/
-----------------------------------------------------------
(Updated Feb. 14, 2017, 9:13 p.m.)
Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
Changes
-------
Add ticket to comment.
Bugs: AURORA-1893
https://issues.apache.org/jira/browse/AURORA-1893
Repository: aurora
Description
-------
Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
Diffs (updated)
-----
RELEASE-NOTES.md 3e9880249aa40215a08424b58f85a634337cc67e
api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
Diff: https://reviews.apache.org/r/56629/diff/
Testing
-------
Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
Also:
./pants test src/test/python/apache/aurora/admin::
Thanks,
David McLaughlin
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by David McLaughlin <da...@dmclaughlin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/
-----------------------------------------------------------
(Updated Feb. 14, 2017, 9:11 p.m.)
Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
Changes
-------
Filed JIRA ticket to track motivation.
Bugs: AURORA-1893
https://issues.apache.org/jira/browse/AURORA-1893
Repository: aurora
Description
-------
Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
Diffs
-----
RELEASE-NOTES.md 3e9880249aa40215a08424b58f85a634337cc67e
api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
Diff: https://reviews.apache.org/r/56629/diff/
Testing
-------
Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
Also:
./pants test src/test/python/apache/aurora/admin::
Thanks,
David McLaughlin
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by David McLaughlin <da...@dmclaughlin.com>.
> On Feb. 14, 2017, 5:44 a.m., Santhosh Kumar Shanmugham wrote:
> > LGTM modulo my pervious comment,
> >
> > "Since this command has the potential to hurt the Scheduler, should we make sure that atleast one of the filters is provided? Or force a minimum default limit to avoid operators from accidentally bringing down the scheduler?"
Not a huge fan of client-side limits like that. We should address how expensive task deletion is on the server side.
- David
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165456
-----------------------------------------------------------
On Feb. 14, 2017, 5:41 a.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 5:41 a.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md 3e9880249aa40215a08424b58f85a634337cc67e
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
> src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by Santhosh Kumar Shanmugham <sa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165456
-----------------------------------------------------------
Ship it!
LGTM modulo my pervious comment,
"Since this command has the potential to hurt the Scheduler, should we make sure that atleast one of the filters is provided? Or force a minimum default limit to avoid operators from accidentally bringing down the scheduler?"
- Santhosh Kumar Shanmugham
On Feb. 13, 2017, 9:41 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 13, 2017, 9:41 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> RELEASE-NOTES.md 3e9880249aa40215a08424b58f85a634337cc67e
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
> src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by David McLaughlin <da...@dmclaughlin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/
-----------------------------------------------------------
(Updated Feb. 14, 2017, 5:41 a.m.)
Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
Changes
-------
Add entry to RELEASE-NOTES.
Repository: aurora
Description
-------
Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
Diffs (updated)
-----
RELEASE-NOTES.md 3e9880249aa40215a08424b58f85a634337cc67e
api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
Diff: https://reviews.apache.org/r/56629/diff/
Testing
-------
Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
Also:
./pants test src/test/python/apache/aurora/admin::
Thanks,
David McLaughlin
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165450
-----------------------------------------------------------
Ship it!
Master (40d91fe) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Feb. 14, 2017, 5:06 a.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 5:06 a.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
> src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by Santhosh Kumar Shanmugham <sa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165448
-----------------------------------------------------------
Add entry in RELEASE-NOTES.
- Santhosh Kumar Shanmugham
On Feb. 13, 2017, 9:06 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 13, 2017, 9:06 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
> src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by David McLaughlin <da...@dmclaughlin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/
-----------------------------------------------------------
(Updated Feb. 14, 2017, 5:06 a.m.)
Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
Changes
-------
Support limit option of query.
Repository: aurora
Description
-------
Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
Diffs (updated)
-----
api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
Diff: https://reviews.apache.org/r/56629/diff/
Testing
-------
Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
Also:
./pants test src/test/python/apache/aurora/admin::
Thanks,
David McLaughlin
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by Santhosh Kumar Shanmugham <sa...@gmail.com>.
> On Feb. 13, 2017, 8:48 p.m., Santhosh Kumar Shanmugham wrote:
> > src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java, lines 1128-1129
> > <https://reviews.apache.org/r/56629/diff/5/?file=1632842#file1632842line1128>
> >
> > Depending on th query this can be a full-scan on the store, which can be inefficient. As mentioned in https://reviews.apache.org/r/56575/ this approach can cause GC pressure. We should use the `TaskStore.pruneHistory` implementation which might be more efficient.
>
> David McLaughlin wrote:
> pruneHistory would search for candidates that are eligible for pruning according to the pruning settings. This is a way to circumvent the grace periods and other limits, and prune all terminal tasks that match the provided query. Since this would be called explicitly and can be supplied with a limit, the operator is able to manage the GC pressure themselves.
Since this command has the potential to hurt the Scheduler, should we make sure that atleast one of the filters is provided? Or force a minimum default `limit` to avoid operators from accidentally bringing down the scheduler?
- Santhosh Kumar
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165440
-----------------------------------------------------------
On Feb. 13, 2017, 8:28 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 13, 2017, 8:28 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
> src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by David McLaughlin <da...@dmclaughlin.com>.
> On Feb. 14, 2017, 4:48 a.m., Santhosh Kumar Shanmugham wrote:
> > src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java, lines 1128-1129
> > <https://reviews.apache.org/r/56629/diff/5/?file=1632842#file1632842line1128>
> >
> > Depending on th query this can be a full-scan on the store, which can be inefficient. As mentioned in https://reviews.apache.org/r/56575/ this approach can cause GC pressure. We should use the `TaskStore.pruneHistory` implementation which might be more efficient.
pruneHistory would search for candidates that are eligible for pruning according to the pruning settings. This is a way to circumvent the grace periods and other limits, and prune all terminal tasks that match the provided query. Since this would be called explicitly and can be supplied with a limit, the operator is able to manage the GC pressure themselves.
- David
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165440
-----------------------------------------------------------
On Feb. 14, 2017, 4:28 a.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 4:28 a.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
> src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by Santhosh Kumar Shanmugham <sa...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165440
-----------------------------------------------------------
src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java (lines 1128 - 1129)
<https://reviews.apache.org/r/56629/#comment237287>
Depending on th query this can be a full-scan on the store, which can be inefficient. As mentioned in https://reviews.apache.org/r/56575/ this approach can cause GC pressure. We should use the `TaskStore.pruneHistory` implementation which might be more efficient.
- Santhosh Kumar Shanmugham
On Feb. 13, 2017, 8:28 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 13, 2017, 8:28 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
> src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165439
-----------------------------------------------------------
Ship it!
Master (40d91fe) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Feb. 14, 2017, 4:28 a.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 4:28 a.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
> src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by David McLaughlin <da...@dmclaughlin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/
-----------------------------------------------------------
(Updated Feb. 14, 2017, 4:28 a.m.)
Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
Repository: aurora
Description
-------
Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
Diffs (updated)
-----
api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
Diff: https://reviews.apache.org/r/56629/diff/
Testing
-------
Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
Also:
./pants test src/test/python/apache/aurora/admin::
Thanks,
David McLaughlin
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by David McLaughlin <da...@dmclaughlin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/
-----------------------------------------------------------
(Updated Feb. 14, 2017, 4:27 a.m.)
Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
Changes
-------
Use StateManager to delete tasks.
Repository: aurora
Description
-------
Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
Diffs (updated)
-----
api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
Diff: https://reviews.apache.org/r/56629/diff/
Testing
-------
Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
Also:
./pants test src/test/python/apache/aurora/admin::
Thanks,
David McLaughlin
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by David McLaughlin <da...@dmclaughlin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/
-----------------------------------------------------------
(Updated Feb. 14, 2017, 4:12 a.m.)
Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
Changes
-------
Fix Python tests.
Repository: aurora
Description
-------
Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
Diffs (updated)
-----
api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
Diff: https://reviews.apache.org/r/56629/diff/
Testing
-------
Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
Also:
./pants test src/test/python/apache/aurora/admin::
Thanks,
David McLaughlin
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by David McLaughlin <da...@dmclaughlin.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/
-----------------------------------------------------------
(Updated Feb. 14, 2017, 2:01 a.m.)
Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
Changes
-------
Checkstyle and pmd fixes.
Repository: aurora
Description
-------
Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
Diffs (updated)
-----
api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
Diff: https://reviews.apache.org/r/56629/diff/
Testing
-------
Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
Also:
./pants test src/test/python/apache/aurora/admin::
Thanks,
David McLaughlin
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by Santhosh Kumar Shanmugham <sa...@gmail.com>.
> On Feb. 13, 2017, 5:54 p.m., Reza Motamedi wrote:
> > LGTM!
> >
> > But, should we test this in the test cluster? At least to check the effect of _limit_?
>
> David McLaughlin wrote:
> I did test this in Vagrant. All the options work correctly.
>
> David McLaughlin wrote:
> Weird, verifying the StateManager change in Vagrant and now limit does not appear to work.
Weird - `TaskQuery.getLimit()` is only used in `ReadOnlySchedulerImpl.getTasks`. (I wonder how your previous test passed?)
Even this implementation is of limit in `getTasks` is inefficient, since we fetch all the tasks from the store and then apply the limit. Limit needs to be applied as part of `TaskStore.queryFilter`.
- Santhosh Kumar
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165425
-----------------------------------------------------------
On Feb. 13, 2017, 8:28 p.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 13, 2017, 8:28 p.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
> src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by David McLaughlin <da...@dmclaughlin.com>.
> On Feb. 14, 2017, 1:54 a.m., Reza Motamedi wrote:
> > LGTM!
> >
> > But, should we test this in the test cluster? At least to check the effect of _limit_?
I did test this in Vagrant. All the options work correctly.
- David
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165425
-----------------------------------------------------------
On Feb. 14, 2017, 2:01 a.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 2:01 a.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by David McLaughlin <da...@dmclaughlin.com>.
> On Feb. 14, 2017, 1:54 a.m., Reza Motamedi wrote:
> > LGTM!
> >
> > But, should we test this in the test cluster? At least to check the effect of _limit_?
>
> David McLaughlin wrote:
> I did test this in Vagrant. All the options work correctly.
Weird, verifying the StateManager change in Vagrant and now limit does not appear to work.
- David
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165425
-----------------------------------------------------------
On Feb. 14, 2017, 4:28 a.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 4:28 a.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
> src/test/python/apache/aurora/client/api/test_scheduler_client.py fab97986dcecf7761e161f4074d5108a8c362e6e
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>
Re: Review Request 56629: Expose task pruning endpoint in aurora_admin
Posted by Reza Motamedi <re...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56629/#review165425
-----------------------------------------------------------
LGTM!
But, should we test this in the test cluster? At least to check the effect of _limit_?
- Reza Motamedi
On Feb. 14, 2017, 1:36 a.m., David McLaughlin wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56629/
> -----------------------------------------------------------
>
> (Updated Feb. 14, 2017, 1:36 a.m.)
>
>
> Review request for Aurora, Santhosh Kumar Shanmugham, Stephan Erb, and Zameer Manji.
>
>
> Repository: aurora
>
>
> Description
> -------
>
> Expose task pruning endpoint in aurora_admin. Useful for scale testing in order to 'clean up' after a test run, but also useful in production if you have a bad actor inflating the size of your task index.
>
>
> Diffs
> -----
>
> api/src/main/thrift/org/apache/aurora/gen/api.thrift 6205c2ed1e2d5898a322de5bc2bcba6b2c0cd4b1
> src/main/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterface.java a211483caf59755b1f9b286b58f183e00db6eee6
> src/main/java/org/apache/aurora/scheduler/thrift/aop/AnnotatedAuroraAdmin.java d63b11d36748de796e96685622a3290bb94e7106
> src/main/python/apache/aurora/admin/admin.py 070c348d2ca5db1edecf832efd9aa5481bddaa4b
> src/main/python/apache/aurora/client/api/__init__.py 1250ccd16f906eec08c159df3300d1b94b306d8e
> src/test/java/org/apache/aurora/scheduler/thrift/SchedulerThriftInterfaceTest.java 0cdd9829417bb3ef0215278a9458a3dd78a49a20
> src/test/python/apache/aurora/admin/test_admin.py 66abade378d3974dbf031a7bbed9cd6fc4f22d5f
>
> Diff: https://reviews.apache.org/r/56629/diff/
>
>
> Testing
> -------
>
> Ran various combinations of task pruning via aurora_admin in Vagrant. Works as expected.
>
> Also:
>
> ./pants test src/test/python/apache/aurora/admin::
>
>
> Thanks,
>
> David McLaughlin
>
>