You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "deepak4babu (via GitHub)" <gi...@apache.org> on 2025/10/17 07:50:34 UTC
[I] Task context environment variables are not set in callbacks running in dag processor [airflow]
deepak4babu opened a new issue, #56756:
URL: https://github.com/apache/airflow/issues/56756
### Apache Airflow version
Other Airflow 2/3 version (please specify below)
### If "Other Airflow 2/3 version" selected, which one?
2.10.3
### What happened?
Task context environment variables are not set in callbacks running in dag processor.
We use the dag_id from AIRFLOW_CTX_DAG_ID environment variable to create some connections in our internal packages.
### What you think should happen instead?
The task context environment variables should be set in callbacks run in dag processor especially on_failure_callbacks
### How to reproduce
Create a dag with on failure callback set for a task. In the callback function, get the environment variables and save it to a file (as logging is not available for callbacks). The env list will not have AIRFLOW_CTX_* keys.
### Operating System
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)" NAME="Debian GNU/Linux" VERSION_ID="12" VERSION="12 (bookworm)" VERSION_CODENAME=bookworm ID=debian HOME_URL="https://www.debian.org/" SUPPORT_URL="https://www.debian.org/support" BUG_REPORT_URL="https://bugs.debian.org/"
### Versions of Apache Airflow Providers
apache-airflow-providers-celery==3.8.3
apache-airflow-providers-cncf-kubernetes==10.1.0
apache-airflow-providers-common-compat==1.2.1
apache-airflow-providers-common-io==1.4.2
apache-airflow-providers-common-sql==1.19.0
apache-airflow-providers-fab==1.5.2
apache-airflow-providers-ftp==3.11.1
apache-airflow-providers-http==4.13.2
apache-airflow-providers-imap==3.7.0
apache-airflow-providers-smtp==1.8.0
apache-airflow-providers-sqlite==3.9.0
### Deployment
Other
### Deployment details
We use a slightly modified version of offical helm chart
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Task context environment variables are not set in callbacks running in dag processor [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #56756:
URL: https://github.com/apache/airflow/issues/56756#issuecomment-3415240822
The `AIRFLOW_CTX_DAG_ID` is only set during parsing, not during execution of callback. It is intended purely for optimisation of dynamic Dag parsing in worker. That's the design.
I am not sure why you would need it also in the callback - can't you get it from the context passed to the callback?
converting to a discussion if more discussion is needed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Task context environment variables are not set in callbacks running in dag processor [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #56756: Task context environment variables are not set in callbacks running in dag processor
URL: https://github.com/apache/airflow/issues/56756
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org