You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/10/11 07:30:52 UTC
[GitHub] [airflow] blinkseb opened a new issue #18880: Unpaused DAGs not triggered
blinkseb opened a new issue #18880:
URL: https://github.com/apache/airflow/issues/18880
### Apache Airflow version
2.1.4 (latest released)
### Operating System
Debian 9
### Versions of Apache Airflow Providers
```
apache-airflow-providers-amazon==2.2.0
apache-airflow-providers-apache-hdfs==2.1.0
apache-airflow-providers-celery==2.0.0
apache-airflow-providers-ftp==2.0.1
apache-airflow-providers-http==2.0.1
apache-airflow-providers-imap==2.0.1
apache-airflow-providers-mysql==2.1.1
apache-airflow-providers-redis==2.0.1
apache-airflow-providers-sqlite==2.0.1
```
### Deployment
Virtualenv installation
### Deployment details
_No response_
### What happened
Hello,
We have a lot of DAGs that we ran daily, but only at specific time of the year. For that, we programatically pause and unpause the DAGs when they should run. However, starting from Airflow 2.10.3, the unpaused DAG is no longer triggered (it was working correctly in previous version of Airflow).
The only thing I see in the log file is the following:
```
[2021-10-10 22:02:35,058] {dag.py:2287} INFO - Setting next_dagrun for mydag to 2021-10-09 22:21:00+00:00
[2021-10-10 22:02:35,080] {dagrun.py:477} INFO - Marking run <DagRun mydag @ 2021-10-08 22:21:00+00:00: scheduled__2021-10-08T22:21:00+00:00, externally triggered: False> successful
[...]
[2021-10-10 22:21:08,439] {dag.py:2287} INFO - Setting next_dagrun for mydag to 2021-10-10 22:21:00+00:00
[2021-10-10 22:21:08,653] {dagrun.py:477} INFO - Marking run <DagRun mydag @ 2021-10-09 22:21:00+00:00: scheduled__2021-10-09T22:21:00+00:00, externally triggered: False> successful
```
To force the DAG to run, I have to restart the scheduler, force run the first task of the DAG, and then clear it.
I also tried to wait a day or two, but the DAG is never scheduled.
Please tell me if you need more information
Thanks a lot
### What you expected to happen
The unpaused DAG should be triggered when unpaused
### How to reproduce
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] blinkseb edited a comment on issue #18880: Unpaused DAGs not triggered
Posted by GitBox <gi...@apache.org>.
blinkseb edited a comment on issue #18880:
URL: https://github.com/apache/airflow/issues/18880#issuecomment-1042686485
Hello everyone, sorry for not answering later, the upgrade to 2.2.0 was a bit harder than expected with the timetable changes!
Unfortunately, we still experience the same issue. The DAG would trigger as expected, but none of the task would run. I've been able to capture a debug log when the DAG is triggered:
```
[2022-02-13 23:02:50,016] {dag.py:2935} INFO - Setting next_dagrun for mydag to 2022-02-12T23:21:00+00:00
[2022-02-13 23:02:50,040] {scheduler_job.py:1058} DEBUG - DAG mydag not changed structure, skipping dagrun.verify_integrity
[2022-02-13 23:02:50,048] {dagrun.py:622} DEBUG - number of tis tasks for <DagRun mydag @ 2022-02-11 23:21:00+00:00: scheduled__2022-02-11T23:21:00+00:00, externally triggered: False>: 0 task(s)
[2022-02-13 23:02:50,048] {dagrun.py:545} INFO - Marking run <DagRun mydag @ 2022-02-11 23:21:00+00:00: scheduled__2022-02-11T23:21:00+00:00, externally triggered: False> successful
[2022-02-13 23:02:50,048] {dagrun.py:590} INFO - DagRun Finished: dag_id=mydag, execution_date=2022-02-11 23:21:00+00:00, run_id=scheduled__2022-02-11T23:21:00+00:00, run_start_date=2022-02-13 23:02:50.029131+00:00, run_end_date=2022-02-13 23:02:50.048701+00:00, run_duration=0.01957, state=success, external_trigger=False, run_type=scheduled, data_interval_start=2022-02-11 23:21:00+00:00, data_interval_end=2022-02-12 23:21:00+00:00, dag_hash=d2e4af2c86c72402eebb911e5269818b
```
one intriguing line is `number of tis tasks for <DAG>: 0 task(s)`, except there are tasks on the DAG as you can see on this screenshot

I've not been able to figure out (yet) where the tasks instances are created in the code. Maybe you can point me to the right direction so I can investigate further?
Thanks a lot!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #18880: Unpaused DAGs not triggered
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #18880:
URL: https://github.com/apache/airflow/issues/18880#issuecomment-1052509993
I think you need to show your dag definition (code). We have no way to reproduce it and information on how you create your dags is necessary to figure out what's going on. I believe it might have something to do with `start_date` but it is a wild gueess.
I convert it into discussion until we have more information.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] blinkseb edited a comment on issue #18880: Unpaused DAGs not triggered
Posted by GitBox <gi...@apache.org>.
blinkseb edited a comment on issue #18880:
URL: https://github.com/apache/airflow/issues/18880#issuecomment-1042686485
Hello everyone, sorry for not answering later, the upgrade to 2.2.0 was a bit harder than expected with the timetable changes!
Unfortunately, we still experience the same issue. The DAG would trigger as expected, but none of the task would run. I've been able to capture a debug log when the DAG is triggered:
```
[2022-02-13 23:02:50,016] {dag.py:2935} INFO - Setting next_dagrun for mydag to 2022-02-12T23:21:00+00:00
[2022-02-13 23:02:50,040] {scheduler_job.py:1058} DEBUG - DAG mydag not changed structure, skipping dagrun.verify_integrity
[2022-02-13 23:02:50,048] {dagrun.py:622} DEBUG - number of tis tasks for <DagRun mydag @ 2022-02-11 23:21:00+00:00: scheduled__2022-02-11T23:21:00+00:00, externally triggered: False>: 0 task(s)
[2022-02-13 23:02:50,048] {dagrun.py:545} INFO - Marking run <DagRun mydag @ 2022-02-11 23:21:00+00:00: scheduled__2022-02-11T23:21:00+00:00, externally triggered: False> successful
[2022-02-13 23:02:50,048] {dagrun.py:590} INFO - DagRun Finished: dag_id=mydag, execution_date=2022-02-11 23:21:00+00:00, run_id=scheduled__2022-02-11T23:21:00+00:00, run_start_date=2022-02-13 23:02:50.029131+00:00, run_end_date=2022-02-13 23:02:50.048701+00:00, run_duration=0.01957, state=success, external_trigger=False, run_type=scheduled, data_interval_start=2022-02-11 23:21:00+00:00, data_interval_end=2022-02-12 23:21:00+00:00, dag_hash=d2e4af2c86c72402eebb911e5269818b
```
one intriguing line is `number of tis tasks for <DAG>: 0 task(s)`, except there are tasks on the DAG as you can see on this screenshot

I've not been able to figure out (yet) where the tasks instances are created in the code. Maybe you can point me to the right direction so I can investigate further?
Thanks a lot!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on issue #18880: Unpaused DAGs not triggered
Posted by GitBox <gi...@apache.org>.
jedcunningham commented on issue #18880:
URL: https://github.com/apache/airflow/issues/18880#issuecomment-946999365
@blinkseb, I wasn't able to reproduce this in 2.2.0. Scheduling has undergone some changes with AIP-39. Any chance you can try with 2.2.0 and see if it is still an issue?
If it is, can you provide a simple DAG to reproduce it (most importantly an example `schedule_interval`, but it's always easier to have a complete DAG)? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] blinkseb commented on issue #18880: Unpaused DAGs not triggered
Posted by GitBox <gi...@apache.org>.
blinkseb commented on issue #18880:
URL: https://github.com/apache/airflow/issues/18880#issuecomment-1042686485
Hello everyone, sorry for not answering later, the upgrade to 2.2.0 was a bit harder than expected with the timetable changes!
Unfortunately, we still experience the same issue. The DAG would trigger as expected, but none of the task would run. I've been able to capture a debug log when the DAG is triggered:
```
[2022-02-13 23:02:50,016] {dag.py:2935} INFO - Setting next_dagrun for optimhome_fr_404 to 2022-02-12T23:21:00+00:00
[2022-02-13 23:02:50,040] {scheduler_job.py:1058} DEBUG - DAG optimhome_fr_404 not changed structure, skipping dagrun.verify_integrity
[2022-02-13 23:02:50,048] {dagrun.py:622} DEBUG - number of tis tasks for <DagRun optimhome_fr_404 @ 2022-02-11 23:21:00+00:00: scheduled__2022-02-11T23:21:00+00:00, externally triggered: False>: 0 task(s)
[2022-02-13 23:02:50,048] {dagrun.py:545} INFO - Marking run <DagRun optimhome_fr_404 @ 2022-02-11 23:21:00+00:00: scheduled__2022-02-11T23:21:00+00:00, externally triggered: False> successful
[2022-02-13 23:02:50,048] {dagrun.py:590} INFO - DagRun Finished: dag_id=optimhome_fr_404, execution_date=2022-02-11 23:21:00+00:00, run_id=scheduled__2022-02-11T23:21:00+00:00, run_start_date=2022-02-13 23:02:50.029131+00:00, run_end_date=2022-02-13 23:02:50.048701+00:00, run_duration=0.01957, state=success, external_trigger=False, run_type=scheduled, data_interval_start=2022-02-11 23:21:00+00:00, data_interval_end=2022-02-12 23:21:00+00:00, dag_hash=d2e4af2c86c72402eebb911e5269818b
```
one intriguing line is `number of tis tasks for <DAG>: 0 task(s)`, except there are tasks on the DAG as you can see on this screenshot

I've not been able to figure out (yet) where the tasks instances are created in the code. Maybe you can point me to the right direction so I can investigate further?
Thanks a lot!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #18880: Unpaused DAGs not triggered
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #18880:
URL: https://github.com/apache/airflow/issues/18880#issuecomment-939764486
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org