You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/11/24 13:54:33 UTC
[GitHub] [airflow] marclamberti opened a new issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
marclamberti opened a new issue #12587:
URL: https://github.com/apache/airflow/issues/12587
**Apache Airflow version**: 2.0.0b3
**Kubernetes version (if you are using kubernetes)** (use `kubectl version`):
**Environment**: MacOS with Docker
- **Cloud provider or hardware configuration**:
- **OS** (e.g. from /etc/os-release):
- **Kernel** (e.g. `uname -a`):
- **Install tools**:
- **Others**:
**What happened**:
```
Traceback (most recent call last):
File "/usr/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 984, in _run_raw_task
result = task_copy.execute(context=context)
File "/usr/local/airflow/plugins/custom_trigger_operator.py", line 138, in execute
dag_run.execution_date,
UnboundLocalError: local variable 'dag_run' referenced before assignment
```
**What you expected to happen**:
The dag_run object doesn't exist in https://github.com/apache/airflow/blob/94ba200d42403d1067a63e2e6b158d893c2e8b5a/airflow/operators/dagrun_operator.py#L161
as the DagRunAlreadyExists exception is raised
**How to reproduce it**:
```
first_task = DummyOperator(...)
trigger_task = TriggerDagRunOperator(...
reset_dag_run=True,
wait_for_completion=True,
trigger_dag_id='my_dag_to_trigger'
...)
```
Run the DAG above once. It success. Clear the task first_task to run a second time the DAG and so trigger again my_dag_to_trigger, you will get the error.
**Anything else we need to know**:
Nope
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #12587:
URL: https://github.com/apache/airflow/issues/12587#issuecomment-733126058
> So I think the problem is in your custom plugin, not Airflow.
I'm afraid that we have a bug. In case of this error, this gets executed:
https://github.com/apache/airflow/blob/94ba200d42403d1067a63e2e6b158d893c2e8b5a/airflow/operators/dagrun_operator.py#L132-L167
The `dag_run` will not exist, unless I miss something. What worries me most is that this usually gets picked up by flake8 / pylint
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dimberman closed issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
Posted by GitBox <gi...@apache.org>.
dimberman closed issue #12587:
URL: https://github.com/apache/airflow/issues/12587
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #12587:
URL: https://github.com/apache/airflow/issues/12587#issuecomment-737801974
> > @turbaszek I think that's why flake8 was "okay" with it -- it the `try` block on L122 `dag_run` is created.
>
> flake8 yes, but I thought that pylint was catching such things - but maybe that's just an impression
@dimberman exactly ^^
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #12587:
URL: https://github.com/apache/airflow/issues/12587#issuecomment-733205368
Is there a chance that the dagrun already exists in this case? That might be the bug. - -in the `if self.reset_dag_run` path `dag_run` is never set.
@turbaszek I think that's why flake8 was "okay" with it -- it the `try` block on L122 `dag_run` is created.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek edited a comment on issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
Posted by GitBox <gi...@apache.org>.
turbaszek edited a comment on issue #12587:
URL: https://github.com/apache/airflow/issues/12587#issuecomment-738002984
@potiuk the problem is in `airflow/operators/dagrun_operator.py` where the `TriggerDagRunOperator` is 😉
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #12587:
URL: https://github.com/apache/airflow/issues/12587#issuecomment-733804022
@marclamberti would you like to fix this issue? We should use `DagRun.get_run` method to get proper dag_run when we catch the exception
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12587:
URL: https://github.com/apache/airflow/issues/12587#issuecomment-738010315
> @potiuk the problem is in `airflow/operators/dagrun_operator.py` where the `TriggerDagRunOperator` is
I know but the dag_run is DagRun object, isn't it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #12587:
URL: https://github.com/apache/airflow/issues/12587#issuecomment-733207706
> @turbaszek I think that's why flake8 was "okay" with it -- it the `try` block on L122 `dag_run` is created.
flake8 yes, but I thought that pylint was catching such things - but maybe that's just an impression
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #12587:
URL: https://github.com/apache/airflow/issues/12587#issuecomment-733204211
@marclamberti what's the stack trace for 2.0.0b3 please?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #12587:
URL: https://github.com/apache/airflow/issues/12587#issuecomment-737997145
> <img alt="Screen Shot 2020-12-02 at 7 17 36 PM" width="437" src="https://user-images.githubusercontent.com/2644098/100959466-2a753400-34d3-11eb-958a-feeb5bae4b9f.png">
>
> crazy that it shows up in my IDE but not in our static checks.
It is likely, because we have "dagrun.py" still excluded from pylint in pylint_tdo.txt
```
./airflow/models/dag.py
./airflow/models/dagrun.py
./airflow/www/utils.py
./airflow/configuration.py
./airflow/models/taskinstance.py
```
Until we remove all the unneeded 'cyclic dependencies' (note! cyclic dependencies, not cyclic imports!) whcih I am boasting about for quite some time, we cannot remove those from the list.
The root cause of it is the way how inter-twined those "core" classes are and how they depend on each other in all directions. I hope I will give it a shot after 2.0 again.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #12587:
URL: https://github.com/apache/airflow/issues/12587#issuecomment-737997145
> <img alt="Screen Shot 2020-12-02 at 7 17 36 PM" width="437" src="https://user-images.githubusercontent.com/2644098/100959466-2a753400-34d3-11eb-958a-feeb5bae4b9f.png">
>
> crazy that it shows up in my IDE but not in our static checks.
It is likely, because we have "dagrun.py" still excluded from pylint in pylint_todo.txt
```
./airflow/models/dag.py
./airflow/models/dagrun.py
./airflow/www/utils.py
./airflow/configuration.py
./airflow/models/taskinstance.py
```
Until we remove all the unneeded 'cyclic dependencies' (note! cyclic dependencies, not cyclic imports!) which I am boasting about for quite some time, we cannot remove those from the list.
The root cause of it is the way how inter-twined those "core" classes are and how they depend on each other in all directions. I hope I will give it a shot after 2.0 again.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #12587:
URL: https://github.com/apache/airflow/issues/12587#issuecomment-738367940
@potiuk I think the problem is somewhere else, see the above comment in Ash PR
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] marclamberti commented on issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
Posted by GitBox <gi...@apache.org>.
marclamberti commented on issue #12587:
URL: https://github.com/apache/airflow/issues/12587#issuecomment-733682227
If you want to reproduce the issue:
https://gist.github.com/marclamberti/92cb097d71d03988895861574365abeb
1. 1. Schedule triggered_dag
2. 2. Schedule master_dag
3. Once master_dag is done, click on the task start -> Clear
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #12587:
URL: https://github.com/apache/airflow/issues/12587#issuecomment-733099938
@marclamberti
THe stack trace shows this as the call site
```
File "/usr/local/airflow/plugins/custom_trigger_operator.py", line 138, in execute
dag_run.execution_date,
```
So I think the problem is in your custom plugin, not Airflow.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] marclamberti commented on issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
Posted by GitBox <gi...@apache.org>.
marclamberti commented on issue #12587:
URL: https://github.com/apache/airflow/issues/12587#issuecomment-733677192
@ashb Here it is:
```
*** Reading local file: /usr/local/airflow/logs/master_dag/trigger_dag/2020-11-24T00:00:00+00:00/2.log
[2020-11-25 12:27:03,944] {taskinstance.py:827} INFO - Dependencies all met for <TaskInstance: master_dag.trigger_dag 2020-11-24T00:00:00+00:00 [queued]>
[2020-11-25 12:27:03,966] {taskinstance.py:827} INFO - Dependencies all met for <TaskInstance: master_dag.trigger_dag 2020-11-24T00:00:00+00:00 [queued]>
[2020-11-25 12:27:03,966] {taskinstance.py:1018} INFO -
--------------------------------------------------------------------------------
[2020-11-25 12:27:03,966] {taskinstance.py:1019} INFO - Starting attempt 2 of 2
[2020-11-25 12:27:03,966] {taskinstance.py:1020} INFO -
--------------------------------------------------------------------------------
[2020-11-25 12:27:03,976] {taskinstance.py:1039} INFO - Executing <Task(TriggerDagRunOperator): trigger_dag> on 2020-11-24T00:00:00+00:00
[2020-11-25 12:27:03,983] {standard_task_runner.py:50} INFO - Started process 12030 to run task
[2020-11-25 12:27:03,990] {standard_task_runner.py:74} INFO - Running: ['airflow', 'tasks', 'run', 'master_dag', 'trigger_dag', '2020-11-24T00:00:00+00:00', '--job-id', '4', '--pool', 'default_pool', '--raw', '--subdir', 'DAGS_FOLDER/master_dag.py', '--cfg-path', '/tmp/tmp7j08b3yu']
[2020-11-25 12:27:03,991] {standard_task_runner.py:75} INFO - Job 4: Subtask trigger_dag
[2020-11-25 12:27:04,057] {logging_mixin.py:103} INFO - Running <TaskInstance: master_dag.trigger_dag 2020-11-24T00:00:00+00:00 [running]> on host c6e5bbbd7410
[2020-11-25 12:27:04,127] {taskinstance.py:1232} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_OWNER=airflow
AIRFLOW_CTX_DAG_ID=master_dag
AIRFLOW_CTX_TASK_ID=trigger_dag
AIRFLOW_CTX_EXECUTION_DATE=2020-11-24T00:00:00+00:00
AIRFLOW_CTX_DAG_RUN_ID=scheduled__2020-11-24T00:00:00+00:00
[2020-11-25 12:27:04,166] {dagrun_operator.py:134} INFO - Clearing triggered_dag on 2020-11-24T00:00:00+00:00
[2020-11-25 12:27:04,213] {taskinstance.py:1402} ERROR - local variable 'dag_run' referenced before assignment
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1087, in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File "/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1260, in _prepare_and_execute_task_with_callbacks
result = self._execute_task(context, task_copy)
File "/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1306, in _execute_task
result = task_copy.execute(context=context)
File "/usr/local/lib/python3.7/site-packages/airflow/operators/dagrun_operator.py", line 156, in execute
dag_run.execution_date,
UnboundLocalError: local variable 'dag_run' referenced before assignment
[2020-11-25 12:27:04,215] {taskinstance.py:1446} INFO - Marking task as FAILED. dag_id=master_dag, task_id=trigger_dag, execution_date=20201124T000000, start_date=20201125T122703, end_date=20201125T122704
[2020-11-25 12:27:04,282] {local_task_job.py:118} INFO - Task exited with return code 1
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] marclamberti commented on issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
Posted by GitBox <gi...@apache.org>.
marclamberti commented on issue #12587:
URL: https://github.com/apache/airflow/issues/12587#issuecomment-733180081
@ashb, I didn't mention it but I copied the exact same code in a custom plugin to get it in 1.10.12, but still the error does exist in Airflow
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #12587:
URL: https://github.com/apache/airflow/issues/12587#issuecomment-732987087
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
Posted by GitBox <gi...@apache.org>.
turbaszek commented on issue #12587:
URL: https://github.com/apache/airflow/issues/12587#issuecomment-738002984
@potiuk the problem is in `airflow/operators/dagrun.py` where the `TriggerDagRunOperator` is 😉
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dimberman commented on issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
Posted by GitBox <gi...@apache.org>.
dimberman commented on issue #12587:
URL: https://github.com/apache/airflow/issues/12587#issuecomment-737638949
<img width="437" alt="Screen Shot 2020-12-02 at 7 17 36 PM" src="https://user-images.githubusercontent.com/2644098/100959466-2a753400-34d3-11eb-958a-feeb5bae4b9f.png">
crazy that it shows up in my IDE but not in our static checks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #12587: dagrun object doesn't exist in the TriggerDagRunOperator
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #12587:
URL: https://github.com/apache/airflow/issues/12587#issuecomment-737997145
> <img alt="Screen Shot 2020-12-02 at 7 17 36 PM" width="437" src="https://user-images.githubusercontent.com/2644098/100959466-2a753400-34d3-11eb-958a-feeb5bae4b9f.png">
>
> crazy that it shows up in my IDE but not in our static checks.
It is likely, because we have "dagrun.py" still excluded from pylint in pylint_todo.txt
```
./airflow/models/dag.py
./airflow/models/dagrun.py
./airflow/www/utils.py
./airflow/configuration.py
./airflow/models/taskinstance.py
```
Until we remove all the unneeded 'cyclic dependencies' (note! cyclic dependencies, not cyclic imports!) whcih I am boasting about for quite some time, we cannot remove those from the list.
The root cause of it is the way how inter-twined those "core" classes are and how they depend on each other in all directions. I hope I will give it a shot after 2.0 again.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org