You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "pedro-cf (via GitHub)" <gi...@apache.org> on 2024/10/28 15:26:13 UTC
[D] Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful? [airflow]
GitHub user pedro-cf created a discussion: Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful?
Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful?
Without adding an Empty Operator at the end with trigger rule "all_done"?
Some kind of soft_fail but for the standard @task ?
GitHub link: https://github.com/apache/airflow/discussions/43443
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful? [airflow]
Posted by "pedro-cf (via GitHub)" <gi...@apache.org>.
GitHub user pedro-cf added a comment to the discussion: Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful?
> You can always extend the notifcation to always succeed, regardless if it fails.
I'm sorry but I don't know what you mean by this?
GitHub link: https://github.com/apache/airflow/discussions/43443#discussioncomment-11078659
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful? [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
GitHub user potiuk added a comment to the discussion: Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful?
```
try:
your code for notification
except Exception():
pass
```
You can write your own operator, (see custom operators in the docs) or write task flow tasks that will NEVER fail, regardless if the code to execute it failed or not. That's easy. you want the task to succeed, when your notification failed. So make it succeed.
GitHub link: https://github.com/apache/airflow/discussions/43443#discussioncomment-11078737
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful? [airflow]
Posted by "pedro-cf (via GitHub)" <gi...@apache.org>.
GitHub user pedro-cf edited a discussion: Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful?
Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful?
Without adding an Empty Operator at the end with trigger rule "all_done"?
Or without adding a try/except in the task code?
Some kind of soft_fail but for the standard @task ?
GitHub link: https://github.com/apache/airflow/discussions/43443
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful? [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
GitHub user potiuk added a comment to the discussion: Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful?
You can always extend the notifcation to always succeed, regardless if it fails.
GitHub link: https://github.com/apache/airflow/discussions/43443#discussioncomment-11078119
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful? [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
GitHub user potiuk edited a comment on the discussion: Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful?
```
try:
your code for notification
except Exception:
pass
```
You can write your own operator, (see custom operators in the docs) or write task flow tasks that will NEVER fail, regardless if the code to execute it failed or not. That's easy. you want the task to succeed, when your notification failed. So make it succeed.
GitHub link: https://github.com/apache/airflow/discussions/43443#discussioncomment-11078737
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful? [airflow]
Posted by "pedro-cf (via GitHub)" <gi...@apache.org>.
GitHub user pedro-cf edited a comment on the discussion: Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful?
I'll explain my scenario.
I have a specific notification task that runs at the end of the DAG.
This notifcation task must only run if the other tasks succeed (with teardown it would always run?).
On the odd chance this notifaction task fails I don't want the DAG Run to fail.
GitHub link: https://github.com/apache/airflow/discussions/43443#discussioncomment-11076701
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful? [airflow]
Posted by "pedro-cf (via GitHub)" <gi...@apache.org>.
GitHub user pedro-cf edited a comment on the discussion: Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful?
I'll explain my scenario.
I have a specific notification task that runs at the end of the DAG.
This notification task must only run if the other tasks succeed (with teardown it would always run?).
On the odd chance this notification task fails I don't want the DAG Run to fail.
GitHub link: https://github.com/apache/airflow/discussions/43443#discussioncomment-11076701
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful? [airflow]
Posted by "pedro-cf (via GitHub)" <gi...@apache.org>.
GitHub user pedro-cf edited a comment on the discussion: Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful?
I'll explain my situation.
I have a specific notification task that runs at the end of the DAG.
This notifcation task must only run if the other tasks succeed (with teardown it would always run?).
On the odd chance this notifaction task fails I don't want the DAG Run to fail.
GitHub link: https://github.com/apache/airflow/discussions/43443#discussioncomment-11076701
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful? [airflow]
Posted by "pedro-cf (via GitHub)" <gi...@apache.org>.
GitHub user pedro-cf added a comment to the discussion: Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful?
I'll explain the situation better.
I have a specific notification task that runs at the end of the DAG.
This notifcation task must only run if the other tasks succeed (with teardown it would always run?).
On the odd chance this notifaction task fails I don't want the DAG Run to fail.
GitHub link: https://github.com/apache/airflow/discussions/43443#discussioncomment-11076701
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful? [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
GitHub user potiuk added a comment to the discussion: Is it possible to allow the last task in a DAG to fail, but for the DAG Run to still be successful?
You should look at setup/teardown functionality (added in 2.8 or 2.9) https://airflow.apache.org/docs/apache-airflow/stable/howto/setup-and-teardown.html - the teardown task is what you are searching for.
GitHub link: https://github.com/apache/airflow/discussions/43443#discussioncomment-11076648
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org