[GitHub] [airflow] ephraimbuddy commented on a change in pull request #16301: Fix task retries when they receive sigkill and have retries. Properly handle sigterm too

[GitBox <gi...@apache.org>]

ephraimbuddy commented on a change in pull request #16301:
URL: https://github.com/apache/airflow/pull/16301#discussion_r675172756



##########
File path: airflow/jobs/local_task_job.py
##########
@@ -78,16 +79,11 @@ def _execute(self):
         def signal_handler(signum, frame):
             """Setting kill signal handler"""
             self.log.error("Received SIGTERM. Terminating subprocesses")
-            self.on_kill()
             self.task_instance.refresh_from_db()
-            if self.task_instance.state not in State.finished:
-                self.task_instance.set_state(State.FAILED)
-            self.task_instance._run_finished_callback(  # pylint: disable=protected-access
-                error="task received sigterm"
-            )
-            raise AirflowException("LocalTaskJob received SIGTERM signal")
+            os.kill(self.task_instance.pid, signum)

Review comment:
       I have updated this. Actually I was misunderstanding what triggers this area of the code. 
   This is triggered by sigterm on task runner and not sigterm on task. 
   When we get the task runner sigterm, we exit the task and handle failure with callback.  This ensures tasks are retried. 




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org