You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/10/01 18:56:09 UTC
[GitHub] [airflow] raehirsch opened a new issue #18678: Unclear when scheduled runtime actually executes DAGs
raehirsch opened a new issue #18678:
URL: https://github.com/apache/airflow/issues/18678
### Describe the issue with documentation
There is no obvious documentation about when DAGs actually run compared to their scheduled cron string. Please indicate where this is documented in the official Airflow documentation.
All I was able to find was a blog post (https://eugeneyan.com/writing/why-airflow-jobs-one-day-late/).
### How to solve the problem
At the top of the Variables page, there should be a note saying something like "if you schedule a job for '0 2 * * *' and you want it to execute on 6/19/21 at 2am, it will be actually run at 2am on 6/20/21 with the { ds } of 6/19/21 at 02:00:00 (i.e. there is always a 24 hour delay after the scheduled time)".
### Anything else
We have experience using Glue, Control-M, Kubernetes, etc and have never encountered this type of inherent delay.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #18678: Unclear when scheduled runtime actually executes DAGs
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #18678:
URL: https://github.com/apache/airflow/issues/18678#issuecomment-932531128
> There is no obvious documentation about when DAGs actually run compared to their scheduled cron string. Please indicate where this is documented in the official Airflow documentation.
There is:
https://airflow.apache.org/docs/apache-airflow/stable/concepts/scheduler.html#scheduler
It's even highlighted :
```
If you run a DAG on a schedule_interval of one day, the run with execution_date 2019-11-21 triggers soon after 2019-11-21T23:59.
Let’s Repeat That, the scheduler runs your job one schedule_interval AFTER the start date, at the END of the period.
```
In any case AIP 39 soon to be released will change things in this area dramatically
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal edited a comment on issue #18678: Unclear when scheduled runtime actually executes DAGs
Posted by GitBox <gi...@apache.org>.
eladkal edited a comment on issue #18678:
URL: https://github.com/apache/airflow/issues/18678#issuecomment-932531128
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal edited a comment on issue #18678: Unclear when scheduled runtime actually executes DAGs
Posted by GitBox <gi...@apache.org>.
eladkal edited a comment on issue #18678:
URL: https://github.com/apache/airflow/issues/18678#issuecomment-932531128
> There is no obvious documentation about when DAGs actually run compared to their scheduled cron string. Please indicate where this is documented in the official Airflow documentation.
There is:
https://airflow.apache.org/docs/apache-airflow/stable/concepts/scheduler.html#scheduler
It's even highlighted :
```
If you run a DAG on a schedule_interval of one day, the run with execution_date 2019-11-21 triggers soon after 2019-11-21T23:59.
Let’s Repeat That, the scheduler runs your job one schedule_interval AFTER the start date, at the END of the period.
```
In any case [AIP 39 - Richer scheduler interval](https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-39+Richer+scheduler_interval) soon to be released with 2.2.0 will change things in this area dramatically
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #18678: Unclear when scheduled runtime actually executes DAGs
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #18678:
URL: https://github.com/apache/airflow/issues/18678#issuecomment-932476040
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal closed issue #18678: Unclear when scheduled runtime actually executes DAGs
Posted by GitBox <gi...@apache.org>.
eladkal closed issue #18678:
URL: https://github.com/apache/airflow/issues/18678
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org