You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/10/02 09:57:40 UTC
[GitHub] [airflow] SergioShz opened a new issue #11234: task marked as failed when sys.exit(0)
SergioShz opened a new issue #11234:
URL: https://github.com/apache/airflow/issues/11234
**Apache Airflow version**:
`apache-airflow (1.10.10)`
**Environment**:
`Docker version 19.03.12`
- **OS** (e.g. from /etc/os-release):
```
NAME="Ubuntu"
VERSION="18.04.5 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.5 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic
```
- **Kernel** (e.g. `uname -a`):
`4.19.76-linuxkit`
**What happened**:
I'm testing this small DAG (see below) consisting of a simple task using PythonOperator. I'd like to exit the call with a exit code 0 when everything went fine, however airflow seems to be marking the task as failed when this happen.
The log file:
`[2020-10-02 09:44:13,081] {taskinstance.py:669} INFO - Dependencies all met for <TaskInstance: TEST_0.test_task 2020-10-02T09:43:00+00:00 [queued]>
[2020-10-02 09:44:13,735] {taskinstance.py:669} INFO - Dependencies all met for <TaskInstance: TEST_0.test_task 2020-10-02T09:43:00+00:00 [queued]>
[2020-10-02 09:44:13,736] {taskinstance.py:879} INFO -
--------------------------------------------------------------------------------
[2020-10-02 09:44:13,737] {taskinstance.py:880} INFO - Starting attempt 1 of 1
[2020-10-02 09:44:13,737] {taskinstance.py:881} INFO -
--------------------------------------------------------------------------------
[2020-10-02 09:44:14,288] {taskinstance.py:900} INFO - Executing <Task(PythonOperator): test_task> on 2020-10-02T09:43:00+00:00
[2020-10-02 09:44:14,291] {standard_task_runner.py:53} INFO - Started process 10800 to run task
[2020-10-02 09:44:16,389] {logging_mixin.py:112} INFO - Running %s on host %s <TaskInstance: TEST_0.test_task 2020-10-02T09:43:00+00:00 [running]> 3a160363119b
[2020-10-02 09:44:17,433] {logging_mixin.py:112} INFO - doing stuff inside task
[2020-10-02 09:44:22,195] {logging_mixin.py:112} INFO - [2020-10-02 09:44:22,195] {local_task_job.py:103} INFO - Task exited with return code 0
'
**What you expected to happen**:
I'm expecting the task marked as succed, or at least understand why airflow marks this task as failed despite of the exit code 0.
**How to reproduce it**:
Here is the code
`import os
from airflow import DAG
from airflow.operators.python_operator import PythonOperator
from airflow.utils.dates import days_ago
default_args = {
'depends_on_past': False,
'start_date': days_ago(1),
'retries': 0,
}
DAG_NAME = 'TEST_0'
SCH_INTERVAL = '0/1 * * * *'
def task():
print('doing stuff inside task')
sys.exit(0)
with DAG(
DAG_NAME,
default_args=default_args,
schedule_interval=SCH_INTERVAL,
) as dag:
task_instance = PythonOperator( task_id='test_task', python_callable=task)
`
And the status in the UI:
thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SergioShz closed issue #11234: task marked as failed when sys.exit(0)
Posted by GitBox <gi...@apache.org>.
SergioShz closed issue #11234:
URL: https://github.com/apache/airflow/issues/11234
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SergioShz commented on issue #11234: task marked as failed when sys.exit(0)
Posted by GitBox <gi...@apache.org>.
SergioShz commented on issue #11234:
URL: https://github.com/apache/airflow/issues/11234#issuecomment-709216988
Hi, sorry for the late reply. I finally just avoided to use sys.exit(0). The answer of @mik-laj is what I needed just to understand a little bit more on why avoid to use this exit code.
I'm closing this, thanks all for your help!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] SergioShz commented on issue #11234: task marked as failed when sys.exit(0)
Posted by GitBox <gi...@apache.org>.
SergioShz commented on issue #11234:
URL: https://github.com/apache/airflow/issues/11234#issuecomment-702724839
Hi, thanks for the quick answer!
Yes, that's what I'm currently doing as fix
but what I'm trying to point out is that if the command succeeds is the same as having a return code of 0 and that it's exactly what sys.exit(0) does explicitly, and in this case I don't see why the task should be marked as failed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #11234: task marked as failed when sys.exit(0)
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #11234:
URL: https://github.com/apache/airflow/issues/11234#issuecomment-702705160
You don't need to explicitly call sys.exit(0) -- as Airflow will do it under the hood if the commands succeded.
You can do the other way around, i.e. do a sys.exit(1) when one of the criteria you are checking it against fails
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #11234: task marked as failed when sys.exit(0)
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #11234:
URL: https://github.com/apache/airflow/issues/11234#issuecomment-702974069
@SergioShz Airflow performs a few more operations after the execution of the operator's `execute` method, e.g. marks a task in the database as completed or executes or executes callbacks. If this code is not executed the task will always be marked as abnormal completion = failed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] JeffryMAC commented on issue #11234: task marked as failed when sys.exit(0)
Posted by GitBox <gi...@apache.org>.
JeffryMAC commented on issue #11234:
URL: https://github.com/apache/airflow/issues/11234#issuecomment-703300022
If you change
```
def task():
print('doing stuff inside task')
sys.exit(0)
```
to:
```
def task():
print('doing stuff inside task')
return
```
wouldn't that solve the issue?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #11234: task marked as failed when sys.exit(0)
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #11234:
URL: https://github.com/apache/airflow/issues/11234#issuecomment-702638542
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org