You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "t oo (Jira)" <ji...@apache.org> on 2019/08/31 06:00:00 UTC
[jira] [Issue Comment Deleted] (AIRFLOW-5109) reap_process_group()
can still result in uncaught ProcessLookupErrors / OSErrors
[ https://issues.apache.org/jira/browse/AIRFLOW-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
t oo updated AIRFLOW-5109:
--------------------------
Comment: was deleted
(was: [~kaxilnaik] [~hblanks] resolution is fixed but no PR linked?)
> reap_process_group() can still result in uncaught ProcessLookupErrors / OSErrors
> --------------------------------------------------------------------------------
>
> Key: AIRFLOW-5109
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5109
> Project: Apache Airflow
> Issue Type: Bug
> Components: utils
> Affects Versions: 1.10.5
> Environment: Found in production using current python3.6-slim-stretch Docker image and airflow 1.10.3, also present on current master branch.
> Reporter: Hunter Blanks
> Priority: Major
> Fix For: 1.10.5
>
>
> {{airflow.utils.helpers.reap_process_group()}} can result in uncaught ProcessLookupError's due to race conditions in calls to Python's {{os}} module.
> {noformat}
> ProcessLookupError: [Errno 3] No such process
> File "airflow", line 32, in <module>
> args.func(args)
> File "airflow/utils/cli.py", line 74, in wrapper
> return f(*args, **kwargs)
> File "airflow/bin/cli.py", line 523, in run
> _run(args, dag, ti)
> File "airflow/bin/cli.py", line 437, in _run
> run_job.run()
> File "airflow/jobs.py", line 209, in run
> self._execute()
> File "airflow/jobs.py", line 2568, in _execute
> self.heartbeat()
> File "airflow/jobs.py", line 192, in heartbeat
> self.heartbeat_callback(session=session)
> File "airflow/utils/db.py", line 69, in wrapper
> return func(*args, **kwargs)
> File "airflow/jobs.py", line 2632, in heartbeat_callback
> self.task_runner.terminate()
> File "airflow/task/task_runner/standard_task_runner.py", line 41, in terminate
> reap_process_group(self.process.pid, self.log)
> File "airflow/utils/helpers.py", line 290, in reap_process_group
> os.killpg(os.getpgid(pid), signal.SIGKILL)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)