[jira] [Commented] (AIRFLOW-5109) reap_process_group() can still result in uncaught ProcessLookupErrors / OSErrors

["t oo (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16920037#comment-16920037 ] 

t oo commented on AIRFLOW-5109:
-------------------------------

[~kaxilnaik] [~hblanks] resolution is fixed but no PR linked?

> reap_process_group() can still result in uncaught ProcessLookupErrors / OSErrors
> --------------------------------------------------------------------------------
>
>                 Key: AIRFLOW-5109
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5109
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: utils
>    Affects Versions: 1.10.5
>         Environment: Found in production using current python3.6-slim-stretch Docker image and airflow 1.10.3, also present on current master branch.
>            Reporter: Hunter Blanks
>            Priority: Major
>             Fix For: 1.10.5
>
>
> {{airflow.utils.helpers.reap_process_group()}} can result in uncaught ProcessLookupError's due to race conditions in calls to Python's {{os}} module.
> {noformat}
> ProcessLookupError: [Errno 3] No such process
>   File "airflow", line 32, in <module>
>     args.func(args)
>   File "airflow/utils/cli.py", line 74, in wrapper
>     return f(*args, **kwargs)
>   File "airflow/bin/cli.py", line 523, in run
>     _run(args, dag, ti)
>   File "airflow/bin/cli.py", line 437, in _run
>     run_job.run()
>   File "airflow/jobs.py", line 209, in run
>     self._execute()
>   File "airflow/jobs.py", line 2568, in _execute
>     self.heartbeat()
>   File "airflow/jobs.py", line 192, in heartbeat
>     self.heartbeat_callback(session=session)
>   File "airflow/utils/db.py", line 69, in wrapper
>     return func(*args, **kwargs)
>   File "airflow/jobs.py", line 2632, in heartbeat_callback
>     self.task_runner.terminate()
>   File "airflow/task/task_runner/standard_task_runner.py", line 41, in terminate
>     reap_process_group(self.process.pid, self.log)
>   File "airflow/utils/helpers.py", line 290, in reap_process_group
>     os.killpg(os.getpgid(pid), signal.SIGKILL)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)