You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "CEardley (via GitHub)" <gi...@apache.org> on 2025/02/28 15:54:55 UTC
[I] on_failure_callback functions specified in task aren't executed [airflow]
CEardley opened a new issue, #47220:
URL: https://github.com/apache/airflow/issues/47220
### Apache Airflow version
2.10.5
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
on_failure_callback is not called when defined in a @task.
### What you think should happen instead?
Simple example below included for clarity. On running this DAG, I would expect the `on_callback_failure` function to be called twice. However based on the logs, only the function call specified within the @dag decorator is called.
```
"""dags/some_dag.py"""
from functools import partial
import logging
from airflow.decorators import dag, task
logger = logging.getLogger(__name__)
def failure_callback(context, message):
logger.warning(f"+++ {message} +++")
@dag(on_failure_callback=partial(failure_callback, message="Called from DAG"))
def some_dag() -> None:
@task(on_failure_callback=partial(failure_callback, message="Called from within task"))
def task_to_fail():
raise ValueError
task_to_fail()
some_dag()
```
Checking the logs at `airflow/logs/scheduler/latest/dags/some_dag.py.log` shows
```
[2025-02-28T15:44:46.563+0000] {logging_mixin.py:190} INFO - [2025-02-28T15:44:46.563+0000] {some_dag.py:10} WARNING - +++ Called from DAG +++
```
But does not show the message +++ Called from within task +++.
### How to reproduce
Create DAG file as specified above. Run Dag and inspect logs.
### Operating System
Debian GNU/Linux 12 (Bookworm)
### Versions of Apache Airflow Providers
apache-airflow-providers-amazon==8.16.0
apache-airflow-providers-apache-spark==4.11.0
apache-airflow-providers-celery==3.8.5
apache-airflow-providers-common-compat==1.3.0
apache-airflow-providers-common-io==1.5.0
apache-airflow-providers-common-sql==1.21.0
apache-airflow-providers-fab==1.5.2
apache-airflow-providers-ftp==3.12.0
apache-airflow-providers-http==5.0.0
apache-airflow-providers-imap==3.8.0
apache-airflow-providers-postgres==5.13.0
apache-airflow-providers-slack==7.3.1
apache-airflow-providers-smtp==1.9.0
apache-airflow-providers-sqlite==4.0.0
### Deployment
Docker-Compose
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] on_failure_callback functions specified in task aren't executed [airflow]
Posted by "CEardley (via GitHub)" <gi...@apache.org>.
CEardley closed issue #47220: on_failure_callback functions specified in task aren't executed
URL: https://github.com/apache/airflow/issues/47220
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] on_failure_callback functions specified in task aren't executed [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #47220:
URL: https://github.com/apache/airflow/issues/47220#issuecomment-2691001011
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] on_failure_callback functions specified in task aren't executed [airflow]
Posted by "CEardley (via GitHub)" <gi...@apache.org>.
CEardley commented on issue #47220:
URL: https://github.com/apache/airflow/issues/47220#issuecomment-2691031390
On further inspection, it appears this is working. However, logs from the callback defined in the `@task` don't appear to be included in that log file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org