[GitHub] [airflow] lucafuji commented on a change in pull request #6870: [AIRFLOW-0578] Check return code

[GitBox <gi...@apache.org>]

lucafuji commented on a change in pull request #6870: [AIRFLOW-0578] Check return code
URL: https://github.com/apache/airflow/pull/6870#discussion_r381445504
 
 

 ##########
 File path: airflow/jobs/local_task_job.py
 ##########
 @@ -95,6 +95,14 @@ def signal_handler(signum, frame):
                 # Monitor the task to see if it's done
                 return_code = self.task_runner.return_code()
                 if return_code is not None:
+                    if return_code != 0:
+                        self.task_instance.refresh_from_db()
+                        # there is one case we should not treat non zero return
+                        # code as failed: the job has been killed externally.
+                        if (not self.terminating) or self.task_instance.state == State.FAILED:
 
 Review comment:
   1. "self.terminating" means job been killed externally but not "state="failed". It's set in heartbeat_callback:L137. Basically it means whenever "ti.state != State.RUNNING", it's terminating.
   
   There are two cases this will happen
   a. explicitly calling terminate of a StandardTaskRunner, then the return code is -9 and the task_instance.state is not failed. In such case, we should not treat non zero exit code as failure.
   b. the task instance is explicitly set as failed, in this case, we should treat non zero exit code as failure.
   
   2. As mentioned above, "job failure" is handled in base_job.py:run:L230. If exception is not thrown here, job state will not be marked as failure

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services