You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@aurora.apache.org by Brian Wickman <wi...@apache.org> on 2015/02/07 00:31:42 UTC
Review Request 30749: Port GC executor to PathDetector interface
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/
-----------------------------------------------------------
Review request for Aurora, Joshua Cohen and Zameer Manji.
Bugs: AURORA-1025
https://issues.apache.org/jira/browse/AURORA-1025
Repository: aurora
Description
-------
This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
Diffs
-----
src/main/python/apache/aurora/executor/bin/BUILD 4d4f3ab9442ceda912ac4ce07eb15e291a4ed08c
src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
Diff: https://reviews.apache.org/r/30749/diff/
Testing
-------
./pants test.pytest --no-fast src/main/python::
Thanks,
Brian Wickman
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review71538
-----------------------------------------------------------
Master (11a65d2) is red with this patch.
./build-support/jenkins/build.sh
Skipping installation of /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter/common/__init__.py (namespace package)
Installing /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter.common.util-0.3.0-py2.7-nspkg.pth
Running setup.py install for twitter.common.log
Skipping installation of /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter/__init__.py (namespace package)
Skipping installation of /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter/common/__init__.py (namespace package)
Installing /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter.common.log-0.3.0-py2.7-nspkg.pth
Running setup.py install for twitter.common.process
Skipping installation of /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter/__init__.py (namespace package)
Skipping installation of /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter/common/__init__.py (namespace package)
Installing /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter.common.process-0.3.0-py2.7-nspkg.pth
Running setup.py install for gitdb
building 'gitdb._perf' extension
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Igitdb -I/usr/include/python2.7 -c gitdb/_fun.c -o build/temp.linux-x86_64-2.7/gitdb/_fun.o
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Igitdb -I/usr/include/python2.7 -c gitdb/_delta_apply.c -o build/temp.linux-x86_64-2.7/gitdb/_delta_apply.o
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/gitdb/_fun.o build/temp.linux-x86_64-2.7/gitdb/_delta_apply.o -o build/lib.linux-x86_64-2.7/gitdb/_perf.so
Running setup.py install for twitter.common.app
Skipping installation of /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter/__init__.py (namespace package)
Skipping installation of /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter/common/__init__.py (namespace package)
Installing /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter.common.app-0.3.0-py2.7-nspkg.pth
Running setup.py install for GitPython
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/local/lib/python2.7/site-packages/setuptools/dist.py:292: UserWarning: The version specified ('0.3.2 RC1') is an invalid version, this may not work as expected with newer versions of setuptools, pip, and PyPI. Please see PEP 440 for more details.
"details." % self.metadata.version
Running setup.py install for pep8
Installing pep8 script to /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/bin
Running setup.py install for pyflakes
Installing pyflakes script to /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/bin
Running setup.py install for twitter.checkstyle
Skipping installation of /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter/__init__.py (namespace package)
Installing /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter.checkstyle-0.1.0-py2.7-nspkg.pth
Installing twitterstyle script to /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/bin
Successfully installed GitPython-0.3.2rc1 gitdb-0.6.4 pep8-1.4.5 pyflakes-0.7.2 smmap-0.9.0 twitter.checkstyle-0.1.0 twitter.common.app-0.3.0 twitter.common.collections-0.3.0 twitter.common.contextutil-0.3.0 twitter.common.dirutil-0.3.0 twitter.common.lang-0.3.0 twitter.common.log-0.3.0 twitter.common.options-0.3.0 twitter.common.process-0.3.0 twitter.common.string-0.3.0 twitter.common.util-0.3.0
F401:ERROR src/main/python/apache/thermos/monitoring/garbage.py:018 'abstractmethod' imported but unused
|from abc import abstractmethod
F401:ERROR src/main/python/apache/thermos/monitoring/garbage.py:022 'Interface' imported but unused
|from twitter.common.lang import Interface
F821:ERROR src/main/python/apache/thermos/monitoring/garbage.py:038 undefined name 'task_id'
| return os.path.getmtime(self._detector.get_checkpoint(task_id))
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Feb. 7, 2015, 12:46 a.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 7, 2015, 12:46 a.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/executor/bin/BUILD 1fa1e9e839bf28093b4e9ded403a761cf4bf5f44
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review71544
-----------------------------------------------------------
Ship it!
Master (11a65d2) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Feb. 7, 2015, 2:27 a.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 7, 2015, 2:27 a.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> pants.ini 99648e46ca755c3c8e22d90c7b682107d8dee333
> src/main/python/apache/aurora/executor/bin/BUILD 1fa1e9e839bf28093b4e9ded403a761cf4bf5f44
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review71676
-----------------------------------------------------------
Ship it!
Master (dcae1e8) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Feb. 9, 2015, 6:56 p.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 9, 2015, 6:56 p.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/executor/bin/BUILD 1fa1e9e839bf28093b4e9ded403a761cf4bf5f44
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Brian Wickman <wi...@apache.org>.
> On Feb. 20, 2015, 11:11 p.m., Brian Wickman wrote:
> > @Reviewbot retry
>
> Bill Farner wrote:
> Is that test known to be flaky? If so, can you file a ticket?
https://reviews.apache.org/r/30647/ should deflake it.
- Brian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review73337
-----------------------------------------------------------
On Feb. 20, 2015, 7:06 p.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 20, 2015, 7:06 p.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/executor/bin/BUILD 6530f4914736754f92ba192c1a345e4b7e4a5398
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 0752d50015b2ff936f079c4a9f2777172dc00a93
> src/main/python/apache/aurora/executor/gc_executor.py dbec82ffe4e155059f3e6b1aa0e67ec4c52a9611
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/bin/BUILD PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_gc_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/test_gc_executor.py 27dee7fa10a4141ec7e9f4440bde2dd257db1cc6
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Bill Farner <wf...@apache.org>.
> On Feb. 20, 2015, 11:11 p.m., Brian Wickman wrote:
> > @Reviewbot retry
>
> Bill Farner wrote:
> Is that test known to be flaky? If so, can you file a ticket?
>
> Brian Wickman wrote:
> https://reviews.apache.org/r/30647/ should deflake it.
Gotcha.
- Bill
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review73337
-----------------------------------------------------------
On Feb. 20, 2015, 7:06 p.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 20, 2015, 7:06 p.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/executor/bin/BUILD 6530f4914736754f92ba192c1a345e4b7e4a5398
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 0752d50015b2ff936f079c4a9f2777172dc00a93
> src/main/python/apache/aurora/executor/gc_executor.py dbec82ffe4e155059f3e6b1aa0e67ec4c52a9611
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/bin/BUILD PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_gc_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/test_gc_executor.py 27dee7fa10a4141ec7e9f4440bde2dd257db1cc6
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Bill Farner <wf...@apache.org>.
> On Feb. 20, 2015, 11:11 p.m., Brian Wickman wrote:
> > @Reviewbot retry
Is that test known to be flaky? If so, can you file a ticket?
- Bill
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review73337
-----------------------------------------------------------
On Feb. 20, 2015, 7:06 p.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 20, 2015, 7:06 p.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/executor/bin/BUILD 6530f4914736754f92ba192c1a345e4b7e4a5398
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 0752d50015b2ff936f079c4a9f2777172dc00a93
> src/main/python/apache/aurora/executor/gc_executor.py dbec82ffe4e155059f3e6b1aa0e67ec4c52a9611
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/bin/BUILD PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_gc_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/test_gc_executor.py 27dee7fa10a4141ec7e9f4440bde2dd257db1cc6
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Brian Wickman <wi...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review73337
-----------------------------------------------------------
@Reviewbot retry
- Brian Wickman
On Feb. 20, 2015, 7:06 p.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 20, 2015, 7:06 p.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/executor/bin/BUILD 6530f4914736754f92ba192c1a345e4b7e4a5398
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 0752d50015b2ff936f079c4a9f2777172dc00a93
> src/main/python/apache/aurora/executor/gc_executor.py dbec82ffe4e155059f3e6b1aa0e67ec4c52a9611
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/bin/BUILD PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_gc_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/test_gc_executor.py 27dee7fa10a4141ec7e9f4440bde2dd257db1cc6
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Joshua Cohen <jc...@twopensource.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review73350
-----------------------------------------------------------
Ship it!
Pending green build, etc.~
- Joshua Cohen
On Feb. 20, 2015, 7:06 p.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 20, 2015, 7:06 p.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/executor/bin/BUILD 6530f4914736754f92ba192c1a345e4b7e4a5398
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 0752d50015b2ff936f079c4a9f2777172dc00a93
> src/main/python/apache/aurora/executor/gc_executor.py dbec82ffe4e155059f3e6b1aa0e67ec4c52a9611
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/bin/BUILD PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_gc_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/test_gc_executor.py 27dee7fa10a4141ec7e9f4440bde2dd257db1cc6
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review73303
-----------------------------------------------------------
Master (e0e3f2e) is red with this patch.
./build-support/jenkins/build.sh
src.test.python.apache.aurora.client.cli.sla ..... SUCCESS
src.test.python.apache.aurora.client.cli.supdate ..... SUCCESS
src.test.python.apache.aurora.client.cli.task ..... SUCCESS
src.test.python.apache.aurora.client.cli.update ..... SUCCESS
src.test.python.apache.aurora.client.cli.version ..... SUCCESS
src.test.python.apache.aurora.client.config ..... SUCCESS
src.test.python.apache.aurora.client.factory ..... SUCCESS
src.test.python.apache.aurora.client.hooks.hooked_api ..... SUCCESS
src.test.python.apache.aurora.client.hooks.non_hooked_api ..... SUCCESS
src.test.python.apache.aurora.common.test_aurora_job_key ..... SUCCESS
src.test.python.apache.aurora.common.test_cluster ..... SUCCESS
src.test.python.apache.aurora.common.test_cluster_option ..... SUCCESS
src.test.python.apache.aurora.common.test_clusters ..... SUCCESS
src.test.python.apache.aurora.common.test_http_signaler ..... SUCCESS
src.test.python.apache.aurora.common.test_pex_version ..... SUCCESS
src.test.python.apache.aurora.common.test_shellify ..... SUCCESS
src.test.python.apache.aurora.common.test_transport ..... SUCCESS
src.test.python.apache.aurora.config.test_base ..... SUCCESS
src.test.python.apache.aurora.config.test_constraint_parsing ..... SUCCESS
src.test.python.apache.aurora.config.test_loader ..... SUCCESS
src.test.python.apache.aurora.config.test_thrift ..... SUCCESS
src.test.python.apache.aurora.executor.common.announcer ..... SUCCESS
src.test.python.apache.aurora.executor.common.directory_sandbox ..... SUCCESS
src.test.python.apache.aurora.executor.common.executor_detector ..... SUCCESS
src.test.python.apache.aurora.executor.common.executor_timeout ..... SUCCESS
src.test.python.apache.aurora.executor.common.health_checker ..... FAILURE
src.test.python.apache.aurora.executor.common.path_detector ..... SUCCESS
src.test.python.apache.aurora.executor.common.task_info ..... SUCCESS
src.test.python.apache.aurora.executor.executor_base ..... SUCCESS
src.test.python.apache.aurora.executor.executor_vars ..... SUCCESS
src.test.python.apache.aurora.executor.status_manager ..... SUCCESS
src.test.python.apache.aurora.executor.thermos_task_runner ..... SUCCESS
src.test.python.apache.thermos.common.test_pathspec ..... SUCCESS
src.test.python.apache.thermos.core.test_runner_integration ..... SUCCESS
src.test.python.apache.thermos.monitoring.test_disk ..... SUCCESS
FAILURE
[31m
FAILURE[0m
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Feb. 20, 2015, 7:06 p.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 20, 2015, 7:06 p.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/executor/bin/BUILD 6530f4914736754f92ba192c1a345e4b7e4a5398
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 0752d50015b2ff936f079c4a9f2777172dc00a93
> src/main/python/apache/aurora/executor/gc_executor.py dbec82ffe4e155059f3e6b1aa0e67ec4c52a9611
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/bin/BUILD PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_gc_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/test_gc_executor.py 27dee7fa10a4141ec7e9f4440bde2dd257db1cc6
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review73383
-----------------------------------------------------------
Ship it!
Master (e5de618) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Feb. 20, 2015, 7:06 p.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 20, 2015, 7:06 p.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/executor/bin/BUILD 6530f4914736754f92ba192c1a345e4b7e4a5398
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 0752d50015b2ff936f079c4a9f2777172dc00a93
> src/main/python/apache/aurora/executor/gc_executor.py dbec82ffe4e155059f3e6b1aa0e67ec4c52a9611
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/bin/BUILD PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_gc_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/test_gc_executor.py 27dee7fa10a4141ec7e9f4440bde2dd257db1cc6
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Brian Wickman <wi...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/
-----------------------------------------------------------
(Updated Feb. 20, 2015, 7:06 p.m.)
Review request for Aurora, Joshua Cohen and Zameer Manji.
Changes
-------
Address feedback.
Bugs: AURORA-1025
https://issues.apache.org/jira/browse/AURORA-1025
Repository: aurora
Description
-------
This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
Diffs (updated)
-----
src/main/python/apache/aurora/executor/bin/BUILD 6530f4914736754f92ba192c1a345e4b7e4a5398
src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 0752d50015b2ff936f079c4a9f2777172dc00a93
src/main/python/apache/aurora/executor/gc_executor.py dbec82ffe4e155059f3e6b1aa0e67ec4c52a9611
src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
src/test/python/apache/aurora/executor/bin/BUILD PRE-CREATION
src/test/python/apache/aurora/executor/bin/test_gc_executor_entry_point.py PRE-CREATION
src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py PRE-CREATION
src/test/python/apache/aurora/executor/test_gc_executor.py 27dee7fa10a4141ec7e9f4440bde2dd257db1cc6
Diff: https://reviews.apache.org/r/30749/diff/
Testing
-------
./pants test.pytest --no-fast src/main/python::
Thanks,
Brian Wickman
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Zameer Manji <zm...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review71741
-----------------------------------------------------------
Ship it!
Ship It!
- Zameer Manji
On Feb. 9, 2015, 3:16 p.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 9, 2015, 3:16 p.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/executor/bin/BUILD 1fa1e9e839bf28093b4e9ded403a761cf4bf5f44
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 0752d50015b2ff936f079c4a9f2777172dc00a93
> src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/bin/BUILD PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_gc_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Brian Wickman <wi...@apache.org>.
> On Feb. 10, 2015, 7:19 p.m., Joe Smith wrote:
> > src/main/python/apache/aurora/executor/gc_executor.py, line 73
> > <https://reviews.apache.org/r/30749/diff/9/?file=859047#file859047line73>
> >
> > Some documentation would go a long way here. I assume this is the way to go from `task_id` <-> `path_of_the_checkpoint_stream_on_disk` ?
Added some commentary.
> On Feb. 10, 2015, 7:19 p.m., Joe Smith wrote:
> > src/main/python/apache/aurora/executor/gc_executor.py, line 129
> > <https://reviews.apache.org/r/30749/diff/9/?file=859047#file859047line129>
> >
> > s/PathDetector/path_detector, I think
whoops, thanks.
> On Feb. 10, 2015, 7:19 p.m., Joe Smith wrote:
> > src/main/python/apache/aurora/executor/gc_executor.py, line 135
> > <https://reviews.apache.org/r/30749/diff/9/?file=859047#file859047line135>
> >
> > ```
> > :param task: an instance of a task to retrieve checkpoint path
> > :type task: RootedTask instance
> > ```
Does it make sense to pydoc a private method? i'll add it but i also don't want to necessarily set a precedent.
> On Feb. 10, 2015, 7:19 p.m., Joe Smith wrote:
> > src/main/python/apache/thermos/monitoring/garbage.py, line 31
> > <https://reviews.apache.org/r/30749/diff/9/?file=859049#file859049line31>
> >
> > Mind adding a docstring here and for the `__init__` ?
done
> On Feb. 10, 2015, 7:19 p.m., Joe Smith wrote:
> > src/main/python/apache/thermos/monitoring/garbage.py, line 173
> > <https://reviews.apache.org/r/30749/diff/9/?file=859049#file859049line173>
> >
> > Everytime I see a namedtuple I cringe a bit (since I really only see them as a poor replacement for spec'd mocks) but if you don't mind adding docs here that'd be ~alright.
This code is on the chopping block, so I wouldn't lament the general practices here too much.
- Brian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review71831
-----------------------------------------------------------
On Feb. 9, 2015, 11:16 p.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 9, 2015, 11:16 p.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/executor/bin/BUILD 1fa1e9e839bf28093b4e9ded403a761cf4bf5f44
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 0752d50015b2ff936f079c4a9f2777172dc00a93
> src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/bin/BUILD PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_gc_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Joe Smith <ya...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review71831
-----------------------------------------------------------
src/main/python/apache/aurora/executor/bin/BUILD
<https://reviews.apache.org/r/30749/#comment117696>
+100
src/main/python/apache/aurora/executor/gc_executor.py
<https://reviews.apache.org/r/30749/#comment117690>
Some documentation would go a long way here. I assume this is the way to go from `task_id` <-> `path_of_the_checkpoint_stream_on_disk` ?
src/main/python/apache/aurora/executor/gc_executor.py
<https://reviews.apache.org/r/30749/#comment117691>
s/PathDetector/path_detector, I think
src/main/python/apache/aurora/executor/gc_executor.py
<https://reviews.apache.org/r/30749/#comment117693>
```
:param task: an instance of a task to retrieve checkpoint path
:type task: RootedTask instance
```
src/main/python/apache/thermos/monitoring/garbage.py
<https://reviews.apache.org/r/30749/#comment117697>
Mind adding a docstring here and for the `__init__` ?
src/main/python/apache/thermos/monitoring/garbage.py
<https://reviews.apache.org/r/30749/#comment117698>
Thanks for this cleanup!
src/main/python/apache/thermos/monitoring/garbage.py
<https://reviews.apache.org/r/30749/#comment117700>
Everytime I see a namedtuple I cringe a bit (since I really only see them as a poor replacement for spec'd mocks) but if you don't mind adding docs here that'd be ~alright.
- Joe Smith
On Feb. 9, 2015, 3:16 p.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 9, 2015, 3:16 p.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/executor/bin/BUILD 1fa1e9e839bf28093b4e9ded403a761cf4bf5f44
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 0752d50015b2ff936f079c4a9f2777172dc00a93
> src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/bin/BUILD PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_gc_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review71720
-----------------------------------------------------------
Ship it!
Master (68aa285) is green with this patch.
./build-support/jenkins/build.sh
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Feb. 9, 2015, 11:16 p.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 9, 2015, 11:16 p.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/executor/bin/BUILD 1fa1e9e839bf28093b4e9ded403a761cf4bf5f44
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 0752d50015b2ff936f079c4a9f2777172dc00a93
> src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/bin/BUILD PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_gc_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Brian Wickman <wi...@apache.org>.
> On Feb. 10, 2015, 7:12 p.m., Joshua Cohen wrote:
> > src/main/python/apache/thermos/monitoring/garbage.py, line 120
> > <https://reviews.apache.org/r/30749/diff/9/?file=859049#file859049line120>
> >
> > Are we no longer worried about the scenario where the checkpoint root is $HOME or /?
The checkpoint root could be set to this but only maliciously.
> On Feb. 10, 2015, 7:12 p.m., Joshua Cohen wrote:
> > src/test/python/apache/aurora/executor/test_gc_executor.py, line 180
> > <https://reviews.apache.org/r/30749/diff/9/?file=859053#file859053line180>
> >
> > s/'fake_root'/FAKE_ROOT
fixed
> On Feb. 10, 2015, 7:12 p.m., Joshua Cohen wrote:
> > src/test/python/apache/aurora/executor/test_gc_executor.py, lines 210-211
> > <https://reviews.apache.org/r/30749/diff/9/?file=859053#file859053line210>
> >
> > Am I missing something or is this a duplicate of the make_task defined above?
totally a dupe, deleted.
> On Feb. 10, 2015, 7:12 p.m., Joshua Cohen wrote:
> > src/test/python/apache/aurora/executor/test_gc_executor.py, line 254
> > <https://reviews.apache.org/r/30749/diff/9/?file=859053#file859053line254>
> >
> > nit: maybe give task_id a default value in make_task instead of having to always pass in the same placeholder value?
Nah, because in all of these assertions we actually need to know the task id so it wouldn't make sense to omit from make_task().
- Brian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review71833
-----------------------------------------------------------
On Feb. 20, 2015, 7:06 p.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 20, 2015, 7:06 p.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/executor/bin/BUILD 6530f4914736754f92ba192c1a345e4b7e4a5398
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 0752d50015b2ff936f079c4a9f2777172dc00a93
> src/main/python/apache/aurora/executor/gc_executor.py dbec82ffe4e155059f3e6b1aa0e67ec4c52a9611
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/bin/BUILD PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_gc_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/test_gc_executor.py 27dee7fa10a4141ec7e9f4440bde2dd257db1cc6
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Joshua Cohen <jc...@twopensource.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review71833
-----------------------------------------------------------
src/main/python/apache/thermos/monitoring/garbage.py
<https://reviews.apache.org/r/30749/#comment117694>
Are we no longer worried about the scenario where the checkpoint root is $HOME or /?
src/test/python/apache/aurora/executor/test_gc_executor.py
<https://reviews.apache.org/r/30749/#comment117699>
s/'fake_root'/FAKE_ROOT
src/test/python/apache/aurora/executor/test_gc_executor.py
<https://reviews.apache.org/r/30749/#comment117701>
Am I missing something or is this a duplicate of the make_task defined above?
src/test/python/apache/aurora/executor/test_gc_executor.py
<https://reviews.apache.org/r/30749/#comment117702>
nit: maybe give task_id a default value in make_task instead of having to always pass in the same placeholder value?
- Joshua Cohen
On Feb. 9, 2015, 11:16 p.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 9, 2015, 11:16 p.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/executor/bin/BUILD 1fa1e9e839bf28093b4e9ded403a761cf4bf5f44
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 0752d50015b2ff936f079c4a9f2777172dc00a93
> src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/bin/BUILD PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_gc_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Brian Wickman <wi...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/
-----------------------------------------------------------
(Updated Feb. 9, 2015, 11:16 p.m.)
Review request for Aurora, Joshua Cohen and Zameer Manji.
Changes
-------
Checkstyle
Bugs: AURORA-1025
https://issues.apache.org/jira/browse/AURORA-1025
Repository: aurora
Description
-------
This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
Diffs (updated)
-----
src/main/python/apache/aurora/executor/bin/BUILD 1fa1e9e839bf28093b4e9ded403a761cf4bf5f44
src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 0752d50015b2ff936f079c4a9f2777172dc00a93
src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
src/test/python/apache/aurora/executor/bin/BUILD PRE-CREATION
src/test/python/apache/aurora/executor/bin/test_gc_executor_entry_point.py PRE-CREATION
src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py PRE-CREATION
src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
Diff: https://reviews.apache.org/r/30749/diff/
Testing
-------
./pants test.pytest --no-fast src/main/python::
Thanks,
Brian Wickman
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review71705
-----------------------------------------------------------
Master (dcae1e8) is red with this patch.
./build-support/jenkins/build.sh
Running setup.py install for twitter.common.collections
Skipping installation of /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter/__init__.py (namespace package)
Skipping installation of /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter/common/__init__.py (namespace package)
Installing /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter.common.collections-0.3.0-py2.7-nspkg.pth
Running setup.py install for twitter.common.util
Skipping installation of /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter/__init__.py (namespace package)
Skipping installation of /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter/common/__init__.py (namespace package)
Installing /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter.common.util-0.3.0-py2.7-nspkg.pth
Running setup.py install for twitter.common.log
Skipping installation of /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter/__init__.py (namespace package)
Skipping installation of /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter/common/__init__.py (namespace package)
Installing /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter.common.log-0.3.0-py2.7-nspkg.pth
Running setup.py install for twitter.common.process
Skipping installation of /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter/__init__.py (namespace package)
Skipping installation of /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter/common/__init__.py (namespace package)
Installing /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter.common.process-0.3.0-py2.7-nspkg.pth
Running setup.py install for gitdb
building 'gitdb._perf' extension
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Igitdb -I/usr/include/python2.7 -c gitdb/_fun.c -o build/temp.linux-x86_64-2.7/gitdb/_fun.o
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Igitdb -I/usr/include/python2.7 -c gitdb/_delta_apply.c -o build/temp.linux-x86_64-2.7/gitdb/_delta_apply.o
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/gitdb/_fun.o build/temp.linux-x86_64-2.7/gitdb/_delta_apply.o -o build/lib.linux-x86_64-2.7/gitdb/_perf.so
Running setup.py install for twitter.common.app
Skipping installation of /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter/__init__.py (namespace package)
Skipping installation of /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter/common/__init__.py (namespace package)
Installing /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter.common.app-0.3.0-py2.7-nspkg.pth
Running setup.py install for GitPython
/home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/local/lib/python2.7/site-packages/setuptools/dist.py:292: UserWarning: The version specified ('0.3.2 RC1') is an invalid version, this may not work as expected with newer versions of setuptools, pip, and PyPI. Please see PEP 440 for more details.
"details." % self.metadata.version
Running setup.py install for pep8
Installing pep8 script to /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/bin
Running setup.py install for pyflakes
Installing pyflakes script to /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/bin
Running setup.py install for twitter.checkstyle
Skipping installation of /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter/__init__.py (namespace package)
Installing /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/lib/python2.7/site-packages/twitter.checkstyle-0.1.0-py2.7-nspkg.pth
Installing twitterstyle script to /home/jenkins/jenkins-slave/workspace/AuroraBot/build-support/python/checkstyle.venv/bin
Successfully installed GitPython-0.3.2rc1 gitdb-0.6.4 pep8-1.4.5 pyflakes-0.7.2 smmap-0.9.0 twitter.checkstyle-0.1.0 twitter.common.app-0.3.0 twitter.common.collections-0.3.0 twitter.common.contextutil-0.3.0 twitter.common.dirutil-0.3.0 twitter.common.lang-0.3.0 twitter.common.log-0.3.0 twitter.common.options-0.3.0 twitter.common.process-0.3.0 twitter.common.string-0.3.0 twitter.common.util-0.3.0
E303:ERROR src/main/python/apache/aurora/executor/bin/thermos_executor_main.py:046 too many blank lines (3)
|CWD = os.environ.get('MESOS_SANDBOX', '.')
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Feb. 9, 2015, 8:24 p.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 9, 2015, 8:24 p.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/executor/bin/BUILD 1fa1e9e839bf28093b4e9ded403a761cf4bf5f44
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 0752d50015b2ff936f079c4a9f2777172dc00a93
> src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/bin/BUILD PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_gc_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py PRE-CREATION
> src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Brian Wickman <wi...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/
-----------------------------------------------------------
(Updated Feb. 9, 2015, 8:24 p.m.)
Review request for Aurora, Joshua Cohen and Zameer Manji.
Changes
-------
Argh, can't catch a break. Fixes build typo.
Bugs: AURORA-1025
https://issues.apache.org/jira/browse/AURORA-1025
Repository: aurora
Description
-------
This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
Diffs (updated)
-----
src/main/python/apache/aurora/executor/bin/BUILD 1fa1e9e839bf28093b4e9ded403a761cf4bf5f44
src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 0752d50015b2ff936f079c4a9f2777172dc00a93
src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
src/test/python/apache/aurora/executor/bin/BUILD PRE-CREATION
src/test/python/apache/aurora/executor/bin/test_gc_executor_entry_point.py PRE-CREATION
src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py PRE-CREATION
src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
Diff: https://reviews.apache.org/r/30749/diff/
Testing
-------
./pants test.pytest --no-fast src/main/python::
Thanks,
Brian Wickman
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Brian Wickman <wi...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/
-----------------------------------------------------------
(Updated Feb. 9, 2015, 8:22 p.m.)
Review request for Aurora, Joshua Cohen and Zameer Manji.
Changes
-------
Adds entry point tests for gc/thermos executors, since I got some of the import wiring wrong:
vagrant@192:/var/lib/mesos/slaves/20150209-182237-119646400-5050-15827-S0/frameworks/20150209-182237-119646400-5050-15827-0000/executors/aurora.gc/runs/latest$ cat stderr
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0209 19:16:58.301468 22286 fetcher.cpp:76] Fetching URI '/home/vagrant/aurora/dist/gc_executor.pex'
I0209 19:16:58.301730 22286 fetcher.cpp:179] Copying resource from '/home/vagrant/aurora/dist/gc_executor.pex' to '/var/lib/mesos/slaves/20150209-182237-119646400-5050-15827-S0/frameworks/20150209-182237-119646400-5050-15827-0000/executors/aurora.gc/runs/6f1f2749-66a9-41e7-bb68-5e31c08cac6d'
Traceback (most recent call last):
File "/var/lib/mesos/slaves/20150209-182237-119646400-5050-15827-S0/frameworks/20150209-182237-119646400-5050-15827-0000/executors/aurora.gc/runs/6f1f2749-66a9-41e7-bb68-5e31c08cac6d/gc_executor.pex/.bootstrap/_pex/pex.py", line 272, in execute
self.execute_entry(entry_point, args)
File "/var/lib/mesos/slaves/20150209-182237-119646400-5050-15827-S0/frameworks/20150209-182237-119646400-5050-15827-0000/executors/aurora.gc/runs/6f1f2749-66a9-41e7-bb68-5e31c08cac6d/gc_executor.pex/.bootstrap/_pex/pex.py", line 320, in execute_entry
runner(entry_point)
File "/var/lib/mesos/slaves/20150209-182237-119646400-5050-15827-S0/frameworks/20150209-182237-119646400-5050-15827-0000/executors/aurora.gc/runs/6f1f2749-66a9-41e7-bb68-5e31c08cac6d/gc_executor.pex/.bootstrap/_pex/pex.py", line 342, in execute_pkg_resources
runner = entry.load(require=False) # trust that the environment is sane
File "/var/lib/mesos/slaves/20150209-182237-119646400-5050-15827-S0/frameworks/20150209-182237-119646400-5050-15827-0000/executors/aurora.gc/runs/6f1f2749-66a9-41e7-bb68-5e31c08cac6d/gc_executor.pex/.bootstrap/pkg_resources.py", line 2048, in load
entry = __import__(self.module_name, globals(),globals(), ['__name__'])
File "/var/lib/mesos/slaves/20150209-182237-119646400-5050-15827-S0/frameworks/20150209-182237-119646400-5050-15827-0000/executors/aurora.gc/runs/6f1f2749-66a9-41e7-bb68-5e31c08cac6d/gc_executor.pex/apache/aurora/executor/bin/gc_executor_main.py", line 27, in <module>
ImportError: No module named executor_detector
vagrant@192:/var/lib/mesos/slaves/20150209-182237-119646400-5050-15827-S0/frameworks/20150209-182237-119646400-5050-15827-0000/executors/aurora.gc/runs/latest$
Bugs: AURORA-1025
https://issues.apache.org/jira/browse/AURORA-1025
Repository: aurora
Description
-------
This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
Diffs (updated)
-----
src/main/python/apache/aurora/executor/bin/BUILD 1fa1e9e839bf28093b4e9ded403a761cf4bf5f44
src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
src/main/python/apache/aurora/executor/bin/thermos_executor_main.py 0752d50015b2ff936f079c4a9f2777172dc00a93
src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
src/test/python/apache/aurora/executor/bin/BUILD PRE-CREATION
src/test/python/apache/aurora/executor/bin/test_gc_executor_entry_point.py PRE-CREATION
src/test/python/apache/aurora/executor/bin/test_thermos_executor_entry_point.py PRE-CREATION
src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
Diff: https://reviews.apache.org/r/30749/diff/
Testing
-------
./pants test.pytest --no-fast src/main/python::
Thanks,
Brian Wickman
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Zameer Manji <zm...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review71678
-----------------------------------------------------------
Ship it!
There is a lot of churn because of interface changes but this change LGTM.
- Zameer Manji
On Feb. 9, 2015, 10:56 a.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 9, 2015, 10:56 a.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/executor/bin/BUILD 1fa1e9e839bf28093b4e9ded403a761cf4bf5f44
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Brian Wickman <wi...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/
-----------------------------------------------------------
(Updated Feb. 9, 2015, 6:56 p.m.)
Review request for Aurora, Joshua Cohen and Zameer Manji.
Changes
-------
Post master merge.
Bugs: AURORA-1025
https://issues.apache.org/jira/browse/AURORA-1025
Repository: aurora
Description
-------
This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
Diffs (updated)
-----
src/main/python/apache/aurora/executor/bin/BUILD 1fa1e9e839bf28093b4e9ded403a761cf4bf5f44
src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
Diff: https://reviews.apache.org/r/30749/diff/
Testing
-------
./pants test.pytest --no-fast src/main/python::
Thanks,
Brian Wickman
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Aurora ReviewBot <wf...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review71671
-----------------------------------------------------------
This patch does not apply cleanly on master (dcae1e8), do you need to rebase?
I will refresh this build result if you post a review containing "@ReviewBot retry"
- Aurora ReviewBot
On Feb. 9, 2015, 6:45 p.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 9, 2015, 6:45 p.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> src/main/python/apache/aurora/executor/bin/BUILD 1fa1e9e839bf28093b4e9ded403a761cf4bf5f44
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
> src/main/python/apache/aurora/executor/thermos_task_runner.py 9ff8c5379aad7ac011115e44b1f5a2b74f759f26
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Brian Wickman <wi...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/
-----------------------------------------------------------
(Updated Feb. 9, 2015, 6:45 p.m.)
Review request for Aurora, Joshua Cohen and Zameer Manji.
Changes
-------
Revert unintended pants.ini change. Pull in a slice of r/30768 to pass e2e.
Bugs: AURORA-1025
https://issues.apache.org/jira/browse/AURORA-1025
Repository: aurora
Description
-------
This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
Diffs (updated)
-----
src/main/python/apache/aurora/executor/bin/BUILD 1fa1e9e839bf28093b4e9ded403a761cf4bf5f44
src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
src/main/python/apache/aurora/executor/thermos_task_runner.py 9ff8c5379aad7ac011115e44b1f5a2b74f759f26
src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
Diff: https://reviews.apache.org/r/30749/diff/
Testing
-------
./pants test.pytest --no-fast src/main/python::
Thanks,
Brian Wickman
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Brian Wickman <wi...@apache.org>.
> On Feb. 8, 2015, 2:20 a.m., Zameer Manji wrote:
> > src/main/python/apache/aurora/executor/gc_executor.py, line 73
> > <https://reviews.apache.org/r/30749/diff/4/?file=853457#file853457line73>
> >
> > What's the benefit of using a namedtuple here?
As opposed to? This means we don't have to change every function signature in this file. (I suppose on the other hand we get less "type safety" because functions called w/o the correct number of parameters would at least TypeError.)
> On Feb. 8, 2015, 2:20 a.m., Zameer Manji wrote:
> > src/main/python/apache/aurora/executor/gc_executor.py, line 411
> > <https://reviews.apache.org/r/30749/diff/4/?file=853457#file853457line411>
> >
> > Shouldn't this log messsage be inside the collector implementation?
The gc executor is on its deathbed -- I propose not changing anything not completely necessary for its deprecation.
- Brian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review71567
-----------------------------------------------------------
On Feb. 7, 2015, 2:27 a.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 7, 2015, 2:27 a.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> pants.ini 99648e46ca755c3c8e22d90c7b682107d8dee333
> src/main/python/apache/aurora/executor/bin/BUILD 1fa1e9e839bf28093b4e9ded403a761cf4bf5f44
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Zameer Manji <zm...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/#review71567
-----------------------------------------------------------
pants.ini
<https://reviews.apache.org/r/30749/#comment117327>
This not does not appear to be a change related to your work here.
src/main/python/apache/aurora/executor/gc_executor.py
<https://reviews.apache.org/r/30749/#comment117328>
What's the benefit of using a namedtuple here?
src/main/python/apache/aurora/executor/gc_executor.py
<https://reviews.apache.org/r/30749/#comment117329>
Shouldn't this log messsage be inside the collector implementation?
- Zameer Manji
On Feb. 6, 2015, 6:27 p.m., Brian Wickman wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30749/
> -----------------------------------------------------------
>
> (Updated Feb. 6, 2015, 6:27 p.m.)
>
>
> Review request for Aurora, Joshua Cohen and Zameer Manji.
>
>
> Bugs: AURORA-1025
> https://issues.apache.org/jira/browse/AURORA-1025
>
>
> Repository: aurora
>
>
> Description
> -------
>
> This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
>
>
> Diffs
> -----
>
> pants.ini 99648e46ca755c3c8e22d90c7b682107d8dee333
> src/main/python/apache/aurora/executor/bin/BUILD 1fa1e9e839bf28093b4e9ded403a761cf4bf5f44
> src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
> src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
> src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
> src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
> src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
>
> Diff: https://reviews.apache.org/r/30749/diff/
>
>
> Testing
> -------
>
> ./pants test.pytest --no-fast src/main/python::
>
>
> Thanks,
>
> Brian Wickman
>
>
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Brian Wickman <wi...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/
-----------------------------------------------------------
(Updated Feb. 7, 2015, 2:27 a.m.)
Review request for Aurora, Joshua Cohen and Zameer Manji.
Changes
-------
Fix checkstyle + legit flakes bug!
Bugs: AURORA-1025
https://issues.apache.org/jira/browse/AURORA-1025
Repository: aurora
Description
-------
This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
Diffs (updated)
-----
pants.ini 99648e46ca755c3c8e22d90c7b682107d8dee333
src/main/python/apache/aurora/executor/bin/BUILD 1fa1e9e839bf28093b4e9ded403a761cf4bf5f44
src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
Diff: https://reviews.apache.org/r/30749/diff/
Testing
-------
./pants test.pytest --no-fast src/main/python::
Thanks,
Brian Wickman
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Brian Wickman <wi...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/
-----------------------------------------------------------
(Updated Feb. 7, 2015, 12:46 a.m.)
Review request for Aurora, Joshua Cohen and Zameer Manji.
Changes
-------
Master merge.
Bugs: AURORA-1025
https://issues.apache.org/jira/browse/AURORA-1025
Repository: aurora
Description
-------
This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
Diffs (updated)
-----
src/main/python/apache/aurora/executor/bin/BUILD 1fa1e9e839bf28093b4e9ded403a761cf4bf5f44
src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
Diff: https://reviews.apache.org/r/30749/diff/
Testing
-------
./pants test.pytest --no-fast src/main/python::
Thanks,
Brian Wickman
Re: Review Request 30749: Port GC executor to PathDetector interface
Posted by Brian Wickman <wi...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30749/
-----------------------------------------------------------
(Updated Feb. 6, 2015, 11:41 p.m.)
Review request for Aurora, Joshua Cohen and Zameer Manji.
Changes
-------
Update updates[] to only map task_ids.
Bugs: AURORA-1025
https://issues.apache.org/jira/browse/AURORA-1025
Repository: aurora
Description
-------
This makes the GC executor detect checkpoint roots via the PathDetector interface. This paves the way to detecting checkpoint roots both from /var/run/thermos and from /var/lib/mesos/**
Diffs (updated)
-----
examples/jobs/cron_hello_world.aurora a99b3ab7b1c82e4e8d72ac7dece6e44d1bba498f
src/main/python/apache/aurora/executor/bin/BUILD 4d4f3ab9442ceda912ac4ce07eb15e291a4ed08c
src/main/python/apache/aurora/executor/bin/gc_executor_main.py b903bcb3630a8a8d50a2008bfae532b2eb947356
src/main/python/apache/aurora/executor/gc_executor.py 43b415bc6c5177be24ec036cc32ae7cbd87fc70f
src/main/python/apache/thermos/bin/thermos.py 161bbdbc4de95c82e2b596e77b0eac7b920eae66
src/main/python/apache/thermos/monitoring/garbage.py 69bf8e4c2e2e5f85f6b822fbe45f828d61814d7f
src/test/python/apache/aurora/executor/test_gc_executor.py b1bbc89a822302d8ea12324eb767631326639ebb
Diff: https://reviews.apache.org/r/30749/diff/
Testing
-------
./pants test.pytest --no-fast src/main/python::
Thanks,
Brian Wickman