You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@aurora.apache.org by "Stephan Erb (JIRA)" <ji...@apache.org> on 2015/02/26 17:35:04 UTC
[jira] [Commented] (AURORA-1155) Insufficient automated testing of
GC executor
[ https://issues.apache.org/jira/browse/AURORA-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14338659#comment-14338659 ]
Stephan Erb commented on AURORA-1155:
-------------------------------------
Even with the latest git snapshot (including the related GC changes from [~wickman] https://reviews.apache.org/r/30749/) the GC executor is still broken:
{code}
WARNING: Logging before InitGoogleLogging() is written to STDERR
I0226 14:36:47.105795 25505 fetcher.cpp:76] Fetching URI '/opt/thermos/bin/gc_executor.pex'
I0226 14:36:47.106696 25505 fetcher.cpp:179] Copying resource from '/opt/thermos/bin/gc_executor.pex' to '/var/lib/mesos/slaves/20150218-102226-1985259712-5050-12423-S6/frameworks/20140919-174559-16842879-5050-27194-0000/executors/aurora.gc/runs/e2a9f631-0c14-4e78-8f56-127ef2104315'
twitter.common.app debug: Initializing: twitter.common.log (Logging subsystem.)
Writing log files to disk in executor_logs
I0226 14:36:47.898592 25518 exec.cpp:132] Version: 0.21.1
I0226 14:36:47.908756 25531 exec.cpp:206] Executor registered on slave 20150218-102226-1985259712-5050-12423-S6
Traceback (most recent call last):
File "/.pex/install/twitter.common.exceptions-0.3.3-py2-none-any.whl.9bce1b9fc9e64880a8d0b5233b6f394bc9496bdb/twitter.common.exceptions-0.3.3-py2-none-any.whl/twitter/common/exceptions/__init__.py", line 126, in _excepting_run
self.__real_run(*args, **kw)
File "/var/lib/mesos/slaves/20150218-102226-1985259712-5050-12423-S6/frameworks/20140919-174559-16842879-5050-27194-0000/executors/aurora.gc/runs/e2a9f631-0c14-4e78-8f56-127ef2104315/gc_executor.pex/apache/aurora/executor/gc_executor.py", line 497, in run
File "/var/lib/mesos/slaves/20150218-102226-1985259712-5050-12423-S6/frameworks/20140919-174559-16842879-5050-27194-0000/executors/aurora.gc/runs/e2a9f631-0c14-4e78-8f56-127ef2104315/gc_executor.pex/apache/aurora/executor/gc_executor.py", line 474, in _run_gc
File "/var/lib/mesos/slaves/20150218-102226-1985259712-5050-12423-S6/frameworks/20140919-174559-16842879-5050-27194-0000/executors/aurora.gc/runs/e2a9f631-0c14-4e78-8f56-127ef2104315/gc_executor.pex/apache/aurora/executor/gc_executor.py", line 435, in garbage_collect
File "/var/lib/mesos/slaves/20150218-102226-1985259712-5050-12423-S6/frameworks/20140919-174559-16842879-5050-27194-0000/executors/aurora.gc/runs/e2a9f631-0c14-4e78-8f56-127ef2104315/gc_executor.pex/apache/aurora/executor/gc_executor.py", line 408, in _gc
File "/var/lib/mesos/slaves/20150218-102226-1985259712-5050-12423-S6/frameworks/20140919-174559-16842879-5050-27194-0000/executors/aurora.gc/runs/e2a9f631-0c14-4e78-8f56-127ef2104315/gc_executor.pex/apache/thermos/monitoring/garbage.py", line 88, in erase_logs
AttributeError: 'TaskGarbageCollector' object has no attribute '_root'
{code}
> Insufficient automated testing of GC executor
> ---------------------------------------------
>
> Key: AURORA-1155
> URL: https://issues.apache.org/jira/browse/AURORA-1155
> Project: Aurora
> Issue Type: Bug
> Reporter: Stephan Erb
> Priority: Critical
>
> Last week, we deployed git revision e0e3f2e (https://github.com/apache/incubator-aurora/tree/e0e3f2e) onto our test cluster.
> For this revision, all GC executor runs are failing:
> {code}
> WARNING: Logging before InitGoogleLogging() is written to STDERR
> I0224 15:00:23.165495 26402 fetcher.cpp:76] Fetching URI '/opt/thermos/bin/gc_executor.pex'
> I0224 15:00:23.165679 26402 fetcher.cpp:179] Copying resource from '/opt/thermos/bin/gc_executor.pex' to '/var/lib/mesos/slaves/20150218-102226-1985259712-5050-12423-S6/frameworks/20140919-174559-16842879-5050-27194-0000/executors/aurora.gc/runs/3f1813a3-76f7-4716-bd75-261d6187a2fc'
> Traceback (most recent call last):
> File "/var/lib/mesos/slaves/20150218-102226-1985259712-5050-12423-S6/frameworks/20140919-174559-16842879-5050-27194-0000/executors/aurora.gc/runs/3f1813a3-76f7-4716-bd75-261d6187a2fc/gc_executor.pex/.bootstrap/_pex/pex.py", line 272, in execute
> self.execute_entry(entry_point, args)
> File "/var/lib/mesos/slaves/20150218-102226-1985259712-5050-12423-S6/frameworks/20140919-174559-16842879-5050-27194-0000/executors/aurora.gc/runs/3f1813a3-76f7-4716-bd75-261d6187a2fc/gc_executor.pex/.bootstrap/_pex/pex.py", line 320, in execute_entry
> runner(entry_point)
> File "/var/lib/mesos/slaves/20150218-102226-1985259712-5050-12423-S6/frameworks/20140919-174559-16842879-5050-27194-0000/executors/aurora.gc/runs/3f1813a3-76f7-4716-bd75-261d6187a2fc/gc_executor.pex/.bootstrap/_pex/pex.py", line 342, in execute_pkg_resources
> runner = entry.load(require=False) # trust that the environment is sane
> File "/var/lib/mesos/slaves/20150218-102226-1985259712-5050-12423-S6/frameworks/20140919-174559-16842879-5050-27194-0000/executors/aurora.gc/runs/3f1813a3-76f7-4716-bd75-261d6187a2fc/gc_executor.pex/.bootstrap/pkg_resources.py", line 2048, in load
> entry = __import__(self.module_name, globals(),globals(), ['__name__'])
> File "/var/lib/mesos/slaves/20150218-102226-1985259712-5050-12423-S6/frameworks/20140919-174559-16842879-5050-27194-0000/executors/aurora.gc/runs/3f1813a3-76f7-4716-bd75-261d6187a2fc/gc_executor.pex/apache/aurora/executor/bin/gc_executor_main.py", line 27, in <module>
> ImportError: No module named executor_detector
> {code}
> For me this is an indication that the test infrastructure is missing an integration or end to end tests which thoroughly tests the GC executor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)