You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/08/26 05:14:52 UTC
[GitHub] [airflow] anuj-condenast opened a new issue #17842: Add a config or variable which will schedule the DAG at the same time as given instead of end of that schedule.
anuj-condenast opened a new issue #17842:
URL: https://github.com/apache/airflow/issues/17842
**Description**
We all have faced issues with Airflow scheduling, where it schedules a DAG at the end of the given schedule. We want a variable or config that could turn that end of schedule off so that if someone wishes, they could schedule at the exact time as mentioned.
**Use case / motivation**
So generally, if I schedule a task to start daily at 12:00 pm, the problem is it will start at the end of the schedule at 11:59 the next day. For daily, it's not that big of a deal, but when we are trying to run it monthly, we face issues in scheduling it exactly on the 1st of every month at a given time instead of the end of the schedule.
We understand Airflow's intention of having the end of the schedule thing, but it would also be great if there is a variable that can be turned off or something in config that can be used to schedule at the exact time and date as mentioned in the schedule. By default, we can have the same thing that airflow still follows, but one extra variable could help many people.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] anuj-condenast commented on issue #17842: Add a config or variable which will schedule the DAG at the same time as given instead of end of that schedule.
Posted by GitBox <gi...@apache.org>.
anuj-condenast commented on issue #17842:
URL: https://github.com/apache/airflow/issues/17842#issuecomment-906253450
Hi @eladkal, I am talking about something similar to this https://github.com/apache/airflow/pull/5787. I see this PR has been closed but can't see the variable schedule_at_interval_end in airflow config.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #17842: Add a config or variable which will schedule the DAG at the same time as given instead of end of that schedule.
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #17842:
URL: https://github.com/apache/airflow/issues/17842
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #17842: Add a config or variable which will schedule the DAG at the same time as given instead of end of that schedule.
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #17842:
URL: https://github.com/apache/airflow/issues/17842#issuecomment-906239734
i don't really understand the variable suggestion. if you want to trigger a run regardless of the schedule use the Trigger Run button in the UI.
Scheduling is indeed isn't simple to understand which is why we have [AIP-39 Richer scheduler_interval ](https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-39+Richer+scheduler_interval) in progress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #17842: Add a config or variable which will schedule the DAG at the same time as given instead of end of that schedule.
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #17842:
URL: https://github.com/apache/airflow/issues/17842#issuecomment-906348935
Yep. AIP-39 addresses this very problem as part of a much more comprehensive and flexible solution. Closing this one as won't fix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #17842: Add a config or variable which will schedule the DAG at the same time as given instead of end of that schedule.
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #17842:
URL: https://github.com/apache/airflow/issues/17842#issuecomment-906265075
As explained in the last comment on the PR by Ash - [AIP-39 Richer scheduler_interval](https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-39+Richer+scheduler_interval) should make it simple for you to schedule your DAGs. It currently in development
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #17842: Add a config or variable which will schedule the DAG at the same time as given instead of end of that schedule.
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #17842:
URL: https://github.com/apache/airflow/issues/17842#issuecomment-906101685
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org