You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/02/02 17:11:33 UTC
[GitHub] [airflow] makrushin-evgenii opened a new issue #21274: Is @DAG/@task decorators compatible with dynamically generation?
makrushin-evgenii opened a new issue #21274:
URL: https://github.com/apache/airflow/issues/21274
### Apache Airflow version
2.2.3 (latest released)
### What happened
```
from datetime import datetime
from airflow.decorators import dag, task
for n in range(1, 4):
dag_id = "test_dag_{}".format(str(n))
@dag(dag_id=dag_id, schedule_interval=None, start_date=datetime(2022, 2, 1), catchup=False)
def test_dag():
@task
def test_task(x):
print("dag_id={}, x={}".format(dag_id, x))
test_task(dag_id)
globals()[dag_id] = test_dag()
```
Running the DAG above, i retrieve the logs:
```
[2022-02-02, 16:44:26 UTC] {taskinstance.py:1426} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_OWNER=***
AIRFLOW_CTX_DAG_ID=test_dag_1
AIRFLOW_CTX_TASK_ID=test_task
AIRFLOW_CTX_EXECUTION_DATE=2022-02-02T16:32:55.747008+00:00
AIRFLOW_CTX_DAG_RUN_ID=manual__2022-02-02T16:32:55.747008+00:00
[2022-02-02, 16:44:26 UTC] {logging_mixin.py:109} INFO - dag_id=test_dag_3, x=test_dag_1
[2022-02-02, 16:51:49 UTC] {taskinstance.py:1426} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_OWNER=***
AIRFLOW_CTX_DAG_ID=test_dag_2
AIRFLOW_CTX_TASK_ID=test_task
AIRFLOW_CTX_EXECUTION_DATE=2022-02-02T16:33:08.308699+00:00
AIRFLOW_CTX_DAG_RUN_ID=manual__2022-02-02T16:33:08.308699+00:00
[2022-02-02, 16:51:49 UTC] {logging_mixin.py:109} INFO - dag_id=test_dag_3, x=test_dag_2
```
1. During the DAG generation, `dag_id` variable tooks different values
2. During the execution `test_task` method, `dag_id` variable tooks the single value "test_dag_3"
3. During the execution `test_task` method, `dag_id` variable tooks different values when passed through `x` param
### What you expected to happen
The variable takes the same value during DAG generation and methods execution:
```
[2022-02-02, 16:44:26 UTC] {taskinstance.py:1426} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_ID=test_dag_1
[2022-02-02, 16:44:26 UTC] {logging_mixin.py:109} INFO - dag_id=test_dag_1, x=test_dag_1
[2022-02-02, 16:51:49 UTC] {taskinstance.py:1426} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_ID=test_dag_2
[2022-02-02, 16:51:49 UTC] {logging_mixin.py:109} INFO - dag_id=test_dag_2, x=test_dag_2
(logs edited to show expected result)
```
### How to reproduce
Run the DAG above
### Operating System
macOS Monterey 12.1
### Versions of Apache Airflow Providers
_No response_
### Deployment
Docker-Compose
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #21274: Is @DAG/@task decorators compatible with dynamically generation?
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #21274:
URL: https://github.com/apache/airflow/issues/21274#issuecomment-1028161412
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org