Re: Review Request 21440: Implementing parallel updater

[Brian Wickman <wi...@apache.org>]

> On May 27, 2014, 6:38 p.m., Brian Wickman wrote:
> > src/test/python/apache/aurora/client/api/test_instance_watcher.py, line 19
> > <https://reviews.apache.org/r/21440/diff/4/?file=588632#file588632line19>
> >
> >     I don't think we can rely upon patching or subclassing _Event (an internal implementation detail) for testing.  it'd be better to just create duck-typed ClockEvent and NoWaitEvent with .wait and .is_set methods.  these probably belong in twitter.common.testing but can be created a la carte for now.
> >     
> >     mba=science=; python3
> >     Python 3.3.3 (default, Apr 15 2014, 11:17:36) 
> >     [GCC 4.2.1 Compatible Apple Clang 4.0 ((tags/Apple/clang-421.0.60))] on darwin
> >     Type "help", "copyright", "credits" or "license" for more information.
> >     >>> from threading import _Event
> >     Traceback (most recent call last):
> >       File "<stdin>", line 1, in <module>
> >     ImportError: cannot import name _Event
> >     >>> 
> >
> 
> Maxim Khutornenko wrote:
>     Done for the fakes. I was not able to make patch work with threading.Event though no matter what I tried (side_effect, return_value and etc.). Dropped a TODO to investigate a possible solution.

can you file a JIRA and add it to the TODO?


- Brian


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/21440/#review44016
-----------------------------------------------------------


On May 30, 2014, 10:08 p.m., Maxim Khutornenko wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/21440/
> -----------------------------------------------------------
> 
> (Updated May 30, 2014, 10:08 p.m.)
> 
> 
> Review request for Aurora, Mark Chu-Carroll and Brian Wickman.
> 
> 
> Bugs: AURORA-350
>     https://issues.apache.org/jira/browse/AURORA-350
> 
> 
> Repository: aurora
> 
> 
> Description
> -------
> 
> The updater now spawns upto batch_size threads to process one instance per thread. 
> 
> All mutating calls are multiplexed by the SchedulerMux to do batch kill/add/restart calls. This is the first step towards a fully multiplexed SchedulerProxy and is intended to mitigate LDAP/scheduler load.
> 
> 
> Diffs
> -----
> 
>   src/main/python/apache/aurora/client/api/instance_watcher.py e09aa9a6c32c17f13c9b8ff3a589919587bd839b 
>   src/main/python/apache/aurora/client/api/job_monitor.py d176995fca68d42fcc2d3989483eaf520d0d737f 
>   src/main/python/apache/aurora/client/api/scheduler_client.py 7be974eb91089f776656ce65b64ee6d8c5b46394 
>   src/main/python/apache/aurora/client/api/updater.py ea7285a75020a47142e1761c7ed455cdc838e37c 
>   src/main/python/apache/aurora/client/api/updater_util.py 04105de8fb2ce1cab049eb06fd313a43bdcd28db 
>   src/test/python/apache/aurora/client/api/test_instance_watcher.py b2d0c804ae2b2095d8d2a99ea42f4da06041cec8 
>   src/test/python/apache/aurora/client/api/test_job_monitor.py 665db74475f4828af2050e98e20bbb3b1b29cf0c 
>   src/test/python/apache/aurora/client/api/test_updater.py ba783da7c0d93bb0bfd03809f62ddcad3f98cd0a 
>   src/test/python/apache/aurora/client/cli/test_create.py b186b52416a2fae8de28fd1d21e7eec07fea8e55 
>   src/test/python/apache/aurora/client/cli/test_kill.py 666ec3aa0745191aa1395e47728343cd0eda7115 
>   src/test/python/apache/aurora/client/cli/test_restart.py 50acc09491ac21935af78499ad66726df5a8f2ff 
>   src/test/python/apache/aurora/client/cli/test_update.py a2abc5eb0f11f9bc563f4504c93fcf5b7520d141 
>   src/test/python/apache/aurora/client/cli/util.py dac4928111200136a9987c9622087e8cdca7f2d2 
>   src/test/python/apache/aurora/client/commands/test_create.py 75f068250b31b656c9c87a6aa66872fbb777b0c0 
>   src/test/python/apache/aurora/client/commands/test_kill.py 3e2ac1fcea301f0ae986b61d9851d10e86996a20 
>   src/test/python/apache/aurora/client/commands/test_restart.py 6e0159f134388a251cb44cd700102d05467a9062 
>   src/test/python/apache/aurora/client/commands/test_update.py c5afbd33d1b2f82e9603c93b967fbc942c0952d7 
>   src/test/python/apache/aurora/client/commands/util.py 84784171816797f3a4fa4c5238d19b626e68ff44 
>   src/test/python/apache/aurora/client/fake_scheduler_proxy.py 2a4773c81efb390385f675854e9631500b263a45 
>   src/test/sh/org/apache/aurora/e2e/http/http_example.aurora 80871273fc4d47558253e6b09c92724e8693bc11 
>   src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora fc723cf232ddbc10458fc394e37358c8523118c2 
>   src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh 9c5652829ac306dda5f7e95e164c85713e18988f 
>   src/test/sh/org/apache/aurora/e2e/test_end_to_end_v2.sh 2af256d65850bd861111279dff4b5c53f234cf7a 
> 
> Diff: https://reviews.apache.org/r/21440/diff/
> 
> 
> Testing
> -------
> 
> ./pants src/test/python:all
> src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
> 
> 
> Thanks,
> 
> Maxim Khutornenko
> 
>