You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by Maxim Khutornenko <ma...@apache.org> on 2014/07/21 23:09:15 UTC
Re: Review Request 21440: Implementing parallel updater
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21440/
-----------------------------------------------------------
(Updated July 21, 2014, 9:09 p.m.)
Review request for Aurora, Mark Chu-Carroll and Brian Wickman.
Changes
-------
Rebased. Brian, is this fully addressing your latest comments?
Bugs: AURORA-350
https://issues.apache.org/jira/browse/AURORA-350
Repository: aurora
Description
-------
The updater now spawns upto batch_size threads to process one instance per thread.
All mutating calls are multiplexed by the SchedulerMux to do batch kill/add/restart calls. This is the first step towards a fully multiplexed SchedulerProxy and is intended to mitigate LDAP/scheduler load.
Diffs (updated)
-----
src/main/python/apache/aurora/client/api/BUILD c205a7d5f281c2b9e52d778c3d84647f25dafc15
src/main/python/apache/aurora/client/api/error_handling_thread.py PRE-CREATION
src/main/python/apache/aurora/client/api/instance_watcher.py e09aa9a6c32c17f13c9b8ff3a589919587bd839b
src/main/python/apache/aurora/client/api/job_monitor.py d176995fca68d42fcc2d3989483eaf520d0d737f
src/main/python/apache/aurora/client/api/scheduler_client.py 10a956c50de893c8380ef1f763ed7d6686b5c00c
src/main/python/apache/aurora/client/api/scheduler_mux.py PRE-CREATION
src/main/python/apache/aurora/client/api/sla.py 8fc9af329a726087c0f93fae975298e042c25d8b
src/main/python/apache/aurora/client/api/task_util.py PRE-CREATION
src/main/python/apache/aurora/client/api/updater.py c592651770f9426fd42b7f92fe2185e1e3065c43
src/main/python/apache/aurora/client/api/updater_util.py 6b689c11b86c99ef5538c0b7bf4dbb81e0df2d13
src/main/python/apache/aurora/client/base.py 663a2471ccb53efc80e9918a26c324475ee50938
src/test/python/apache/aurora/client/api/BUILD 804195b719cad7652b3ccb15f03c7c80fc2a13f3
src/test/python/apache/aurora/client/api/test_instance_watcher.py 723a5b6f429f85202ee4ad3a004f019cec632a98
src/test/python/apache/aurora/client/api/test_job_monitor.py 3aa96075e085ac7c2607a8c303ee5daf6e3e43b0
src/test/python/apache/aurora/client/api/test_scheduler_mux.py PRE-CREATION
src/test/python/apache/aurora/client/api/test_task_util.py PRE-CREATION
src/test/python/apache/aurora/client/api/test_updater.py 48f82c8d9606b7e3bae77eaabbbca8edc92ab045
src/test/python/apache/aurora/client/cli/test_create.py cda95ceca4803cb39ad1c2b2bebddc5fba61d145
src/test/python/apache/aurora/client/cli/test_kill.py e9e167991cd06e556dbbc839eb9a808eb7d88fe9
src/test/python/apache/aurora/client/cli/test_restart.py 407eb41eb6e78721ef29baefdee5699e605193f9
src/test/python/apache/aurora/client/cli/test_update.py 44a180e88893fea1d7acf1862d5aaa95d98598f5
src/test/python/apache/aurora/client/cli/util.py 65e679fbfad421ce3373e07d01e8619b5f10af5b
src/test/python/apache/aurora/client/commands/test_create.py 4cd1343b19e9d13abe20d3b88dc7460ec512d57b
src/test/python/apache/aurora/client/commands/test_kill.py 94bbe1bfff1c8d333837aa6bb14700714f46288b
src/test/python/apache/aurora/client/commands/test_restart.py 0c6d5a069c43d96098c126aa340a734cb37c667a
src/test/python/apache/aurora/client/commands/test_update.py ea8e092267fd65ccd24ca1be3c04f50ba89f5236
src/test/python/apache/aurora/client/commands/util.py e24f130234a431969d6f52233e9b7fa9297bbaa4
src/test/python/apache/aurora/client/fake_scheduler_proxy.py 2a4773c81efb390385f675854e9631500b263a45
src/test/sh/org/apache/aurora/e2e/http/http_example.aurora 664c3e3c27a58d64d73ac8c9066bf4dc721415ad
src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora dcf61033743317a51dd2f77e8d254c4781cb6a93
src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 9c5652829ac306dda5f7e95e164c85713e18988f
src/test/sh/org/apache/aurora/e2e/test_end_to_end_v2.sh 2af256d65850bd861111279dff4b5c53f234cf7a
Diff: https://reviews.apache.org/r/21440/diff/
Testing
-------
./pants src/test/python:all
src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
Thanks,
Maxim Khutornenko
Re: Review Request 21440: Implementing parallel updater
Posted by Maxim Khutornenko <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21440/
-----------------------------------------------------------
(Updated July 23, 2014, 5:05 p.m.)
Review request for Aurora, Mark Chu-Carroll and Brian Wickman.
Changes
-------
Rebasing.
Bugs: AURORA-350
https://issues.apache.org/jira/browse/AURORA-350
Repository: aurora
Description
-------
The updater now spawns upto batch_size threads to process one instance per thread.
All mutating calls are multiplexed by the SchedulerMux to do batch kill/add/restart calls. This is the first step towards a fully multiplexed SchedulerProxy and is intended to mitigate LDAP/scheduler load.
Diffs (updated)
-----
src/main/python/apache/aurora/client/api/BUILD c205a7d5f281c2b9e52d778c3d84647f25dafc15
src/main/python/apache/aurora/client/api/error_handling_thread.py PRE-CREATION
src/main/python/apache/aurora/client/api/instance_watcher.py d2ad6fde52316cd43429952ce862b3c023a9697f
src/main/python/apache/aurora/client/api/job_monitor.py 6c26cb95372cc654b415d2480e366e7673446384
src/main/python/apache/aurora/client/api/scheduler_client.py e9111354e6fe61d33dd70acea6345b369516ffd2
src/main/python/apache/aurora/client/api/scheduler_mux.py PRE-CREATION
src/main/python/apache/aurora/client/api/sla.py 50befea298e1a0cd0cb2178ae22d86d6522a75bd
src/main/python/apache/aurora/client/api/task_util.py PRE-CREATION
src/main/python/apache/aurora/client/api/updater.py c592651770f9426fd42b7f92fe2185e1e3065c43
src/main/python/apache/aurora/client/api/updater_util.py 6b689c11b86c99ef5538c0b7bf4dbb81e0df2d13
src/main/python/apache/aurora/client/base.py 663a2471ccb53efc80e9918a26c324475ee50938
src/test/python/apache/aurora/client/api/BUILD 804195b719cad7652b3ccb15f03c7c80fc2a13f3
src/test/python/apache/aurora/client/api/test_instance_watcher.py 93b43e8e2f9ffa3477b3f311a94476579a530d8f
src/test/python/apache/aurora/client/api/test_job_monitor.py 3cc876f4fc02420e418d3cf524761c3736884fed
src/test/python/apache/aurora/client/api/test_scheduler_mux.py PRE-CREATION
src/test/python/apache/aurora/client/api/test_task_util.py PRE-CREATION
src/test/python/apache/aurora/client/api/test_updater.py 48f82c8d9606b7e3bae77eaabbbca8edc92ab045
src/test/python/apache/aurora/client/cli/test_create.py af548ae54ac270679ba7b34d998ea95c05f5b640
src/test/python/apache/aurora/client/cli/test_kill.py 48badec1996db0d9fe537234253258e44dda6ebe
src/test/python/apache/aurora/client/cli/test_restart.py e85951a1a7e86887a524529748e9e3cdd1e02749
src/test/python/apache/aurora/client/cli/test_update.py c6440e4469d4eea895feb91cb7dcde343388e2cc
src/test/python/apache/aurora/client/cli/util.py fb849f0f34e3fc1c9685953fcb094e5f4af874c0
src/test/python/apache/aurora/client/commands/test_create.py b8df1f2daab95ac83461d706ec5c27daddfcd220
src/test/python/apache/aurora/client/commands/test_kill.py 642ee64a903dbd040e3e94a3600448ed5da7f5a3
src/test/python/apache/aurora/client/commands/test_restart.py b95e7ffe77432f3c6492bcfad1ea71f5a41ddb53
src/test/python/apache/aurora/client/commands/test_update.py 114ac01d0579b494c914d3af3fc581b9c1d632e5
src/test/python/apache/aurora/client/commands/util.py e24f130234a431969d6f52233e9b7fa9297bbaa4
src/test/python/apache/aurora/client/fake_scheduler_proxy.py 2a4773c81efb390385f675854e9631500b263a45
src/test/sh/org/apache/aurora/e2e/http/http_example.aurora 664c3e3c27a58d64d73ac8c9066bf4dc721415ad
src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora dcf61033743317a51dd2f77e8d254c4781cb6a93
src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 9c5652829ac306dda5f7e95e164c85713e18988f
src/test/sh/org/apache/aurora/e2e/test_end_to_end_v2.sh 2af256d65850bd861111279dff4b5c53f234cf7a
Diff: https://reviews.apache.org/r/21440/diff/
Testing
-------
./pants src/test/python:all
src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
Thanks,
Maxim Khutornenko
Re: Review Request 21440: Implementing parallel updater
Posted by Brian Wickman <wi...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21440/#review48459
-----------------------------------------------------------
Ship it!
Ship It!
- Brian Wickman
On July 21, 2014, 9:09 p.m., Maxim Khutornenko wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21440/
> -----------------------------------------------------------
>
> (Updated July 21, 2014, 9:09 p.m.)
>
>
> Review request for Aurora, Mark Chu-Carroll and Brian Wickman.
>
>
> Bugs: AURORA-350
> https://issues.apache.org/jira/browse/AURORA-350
>
>
> Repository: aurora
>
>
> Description
> -------
>
> The updater now spawns upto batch_size threads to process one instance per thread.
>
> All mutating calls are multiplexed by the SchedulerMux to do batch kill/add/restart calls. This is the first step towards a fully multiplexed SchedulerProxy and is intended to mitigate LDAP/scheduler load.
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/client/api/BUILD c205a7d5f281c2b9e52d778c3d84647f25dafc15
> src/main/python/apache/aurora/client/api/error_handling_thread.py PRE-CREATION
> src/main/python/apache/aurora/client/api/instance_watcher.py e09aa9a6c32c17f13c9b8ff3a589919587bd839b
> src/main/python/apache/aurora/client/api/job_monitor.py d176995fca68d42fcc2d3989483eaf520d0d737f
> src/main/python/apache/aurora/client/api/scheduler_client.py 10a956c50de893c8380ef1f763ed7d6686b5c00c
> src/main/python/apache/aurora/client/api/scheduler_mux.py PRE-CREATION
> src/main/python/apache/aurora/client/api/sla.py 8fc9af329a726087c0f93fae975298e042c25d8b
> src/main/python/apache/aurora/client/api/task_util.py PRE-CREATION
> src/main/python/apache/aurora/client/api/updater.py c592651770f9426fd42b7f92fe2185e1e3065c43
> src/main/python/apache/aurora/client/api/updater_util.py 6b689c11b86c99ef5538c0b7bf4dbb81e0df2d13
> src/main/python/apache/aurora/client/base.py 663a2471ccb53efc80e9918a26c324475ee50938
> src/test/python/apache/aurora/client/api/BUILD 804195b719cad7652b3ccb15f03c7c80fc2a13f3
> src/test/python/apache/aurora/client/api/test_instance_watcher.py 723a5b6f429f85202ee4ad3a004f019cec632a98
> src/test/python/apache/aurora/client/api/test_job_monitor.py 3aa96075e085ac7c2607a8c303ee5daf6e3e43b0
> src/test/python/apache/aurora/client/api/test_scheduler_mux.py PRE-CREATION
> src/test/python/apache/aurora/client/api/test_task_util.py PRE-CREATION
> src/test/python/apache/aurora/client/api/test_updater.py 48f82c8d9606b7e3bae77eaabbbca8edc92ab045
> src/test/python/apache/aurora/client/cli/test_create.py cda95ceca4803cb39ad1c2b2bebddc5fba61d145
> src/test/python/apache/aurora/client/cli/test_kill.py e9e167991cd06e556dbbc839eb9a808eb7d88fe9
> src/test/python/apache/aurora/client/cli/test_restart.py 407eb41eb6e78721ef29baefdee5699e605193f9
> src/test/python/apache/aurora/client/cli/test_update.py 44a180e88893fea1d7acf1862d5aaa95d98598f5
> src/test/python/apache/aurora/client/cli/util.py 65e679fbfad421ce3373e07d01e8619b5f10af5b
> src/test/python/apache/aurora/client/commands/test_create.py 4cd1343b19e9d13abe20d3b88dc7460ec512d57b
> src/test/python/apache/aurora/client/commands/test_kill.py 94bbe1bfff1c8d333837aa6bb14700714f46288b
> src/test/python/apache/aurora/client/commands/test_restart.py 0c6d5a069c43d96098c126aa340a734cb37c667a
> src/test/python/apache/aurora/client/commands/test_update.py ea8e092267fd65ccd24ca1be3c04f50ba89f5236
> src/test/python/apache/aurora/client/commands/util.py e24f130234a431969d6f52233e9b7fa9297bbaa4
> src/test/python/apache/aurora/client/fake_scheduler_proxy.py 2a4773c81efb390385f675854e9631500b263a45
> src/test/sh/org/apache/aurora/e2e/http/http_example.aurora 664c3e3c27a58d64d73ac8c9066bf4dc721415ad
> src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora dcf61033743317a51dd2f77e8d254c4781cb6a93
> src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 9c5652829ac306dda5f7e95e164c85713e18988f
> src/test/sh/org/apache/aurora/e2e/test_end_to_end_v2.sh 2af256d65850bd861111279dff4b5c53f234cf7a
>
> Diff: https://reviews.apache.org/r/21440/diff/
>
>
> Testing
> -------
>
> ./pants src/test/python:all
> src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
>
>
> Thanks,
>
> Maxim Khutornenko
>
>