You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Asquator (via GitHub)" <gi...@apache.org> on 2026/01/27 12:32:45 UTC
[I] Add a way to differentiate between a task/DAG-level callback by context [airflow]
Asquator opened a new issue, #61119:
URL: https://github.com/apache/airflow/issues/61119
### Description
Currently, the `context` passed to a callback defined on `DAG` level looks pretty much the same as a `context` passed to a task callback. Thus, we the callback's code doesn't have a way to differentiate between the two cases (from experimentation I noted that the task-level callback has an `exception` field while the DAG callback doesn't, but it doesn't look reliable enough).
I expected the DAG-level callback to not have a `task_instance` field, but apparently it points to the last task executed.
I would propose injecting a new field into the context that points to the parent object (TI or DAG) that caused the callback to be called.
### Use case/motivation
In my callback code I want to know which kind of callback was triggered and handle a failure accordingly.
### Related issues
_No response_
### Are you willing to submit a PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add a way to differentiate between a task/DAG-level callback by context [airflow]
Posted by "jroachgolf84 (via GitHub)" <gi...@apache.org>.
jroachgolf84 commented on issue #61119:
URL: https://github.com/apache/airflow/issues/61119#issuecomment-3805208152
@Asquator, please feel free to open up a PR for this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add a way to differentiate between a task/DAG-level callback by context [airflow]
Posted by "sanchalitorpe-source (via GitHub)" <gi...@apache.org>.
sanchalitorpe-source commented on issue #61119:
URL: https://github.com/apache/airflow/issues/61119#issuecomment-3811161470
Hi @jroachgolf84, thanks for the confirmation!
I’d like to work on this issue. My current plan is:
- Introduce a small, explicit field in the callback context (e.g. callback_source or similar) to clearly distinguish between DAG-level and task-level callbacks
- Ensure existing context keys remain unchanged for backward compatibility
- Add unit tests covering both DAG and task callbacks
- Update documentation to describe the new context field
Before starting implementation, does this approach sound acceptable to you, or is there a preferred direction/naming for the context change?
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add a way to differentiate between a task/DAG-level callback by context [airflow]
Posted by "Asquator (via GitHub)" <gi...@apache.org>.
Asquator commented on issue #61119:
URL: https://github.com/apache/airflow/issues/61119#issuecomment-3808139153
Found out that the context you see in DAG callbacks consists of task details of the last task in the DAG by lexicographical order - https://github.com/apache/airflow/issues/26760#issuecomment-3808123412
I think it's worth starting from changing this behavior and then adding some differentiation criteria (documenting it too)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add a way to differentiate between a task/DAG-level callback by context [airflow]
Posted by "Asquator (via GitHub)" <gi...@apache.org>.
Asquator commented on issue #61119:
URL: https://github.com/apache/airflow/issues/61119#issuecomment-3807386880
I also noted that in 3.1.6 the DAG level callbacks write logs to a folder like `$AIRFLOW_HOME/logs/**dag_processor**/latest/DAG_FILE.py.log` and not the ** scheduler ** as it appears in the docs:
<img width="1613" height="128" alt="Image" src="https://github.com/user-attachments/assets/762d6f1b-d853-477a-9914-53378acefaf6" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add a way to differentiate between a task/DAG-level callback by context [airflow]
Posted by "tirkarthi (via GitHub)" <gi...@apache.org>.
tirkarthi commented on issue #61119:
URL: https://github.com/apache/airflow/issues/61119#issuecomment-3806144154
Related https://github.com/apache/airflow/issues/26760
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add a way to differentiate between a task/DAG-level callback by context [airflow]
Posted by "Asquator (via GitHub)" <gi...@apache.org>.
Asquator commented on issue #61119:
URL: https://github.com/apache/airflow/issues/61119#issuecomment-3804986158
I'd like to work on this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org