[I] Pause a Dag for a set time and resume it automatically [airflow]

["joshmarkovic (via GitHub)" <gi...@apache.org>]

joshmarkovic opened a new issue, #74092:
URL: https://github.com/apache/airflow/issues/74092

   ### Description
   
   Let users pause a Dag for a set time, such as 1 hour, 4 hours, or until a chosen date and time. When the time is up, Airflow unpauses the Dag on its own.
   
   In the UI, the pause toggle could offer a few choices: pause until I turn it back on (today's behavior), pause for 1 hour, pause for 4 hours, pause for 1 day, or pause until a date and time. While a timed pause is active, the Dag list and the Dag page would show when the Dag comes back on. The same option would help in the REST API and the CLI.
   
   I'm happy to build this and open a PR. Before I start, I'd welcome a steer on whether this fits Airflow and how you'd like it shaped.
   
   ### Use case/motivation
   
   We often need to stop a Dag for a short, known period: database maintenance, a vendor outage, or a holiday. As of Airflow 3.3.2, a pause has no end time, so someone has to remember to unpause the Dag. When they forget, the Dag stays off and data quietly goes stale. Others looking at the UI also can't tell whether a paused Dag is meant to stay off or should already be back on.
   
   The workarounds live outside the pause button: a cron job, or a separate Dag that unpauses through the CLI or the API. People forget they exist, and nothing in the UI shows that a resume is planned.
   
   What I'd like to happen:
   
   - When I pause a Dag, I can choose how long it stays paused, or until when.
   - At that time, Airflow unpauses the Dag, just as if someone had clicked the toggle.
   - The UI shows that the Dag is paused until a given time.
   - I can still unpause early or change the end time.
   
   ### Related issues
   
   - #15675 asks for an optional reason when pausing a Dag. A comment there also asks for an unpause time, but the issue itself is about the reason.
   - #16349 asked for this same timed unpause in 2021. It was closed as a duplicate of #15675.
   - #69699 proposes a global pause for maintenance. It freezes the whole deployment and has no end time, so it solves a different problem.
   
   ### Are you willing to submit a PR?
   
   - [x] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's Code of Conduct
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Pause a Dag for a set time and resume it automatically [airflow]

["joshmarkovic (via GitHub)" <gi...@apache.org>]

joshmarkovic commented on issue #74092:
URL: https://github.com/apache/airflow/issues/74092#issuecomment-5962032379

   I'd like to start on this. Since it builds on the draining state added in #72407, I want to check the direction first.
   
   Proposal:
   
   - Add a nullable paused_until timestamp to the Dag. It can only be set when pausing or draining a Dag.
   - A pause or drain request that includes paused_until sets or replaces it, and one that sends it as null removes it. A request that leaves it out keeps it, so bulk actions and other state changes don't silently drop a timer. Unpausing always removes it, and so does the auto-pause after repeated failed runs, so a person decides when to resume.
   - A scheduler step, modeled on the one that finishes a drain, returns Dags to active once paused_until has passed. It adds a pause_expired entry to the audit log. From then on, the Dag behaves as if someone had unpaused it.
   - If the time passes while the Dag is still draining, it goes straight back to active.
   - The REST API accepts paused_until alongside is_paused or scheduling_state, for single and bulk updates, and returns it in Dag responses. airflowctl gets --until on dags pause and dags drain.
   - In the UI, the toggle stays a single click. A separate "Pause for" action offers 1 hour, 4 hours, 1 day, or a chosen date and time. A paused Dag shows "Paused until ..." where the next run would normally be.
   
   I plan two PRs. The first covers the model, migration, scheduler, API, airflowctl and docs. The second covers the UI, after #73763 lands, since both touch the same components.
   
   Questions:
   
   - Does paused_until work as the name, or would you prefer something else?
   - Is a separate "Pause for" action the right place, or should the choice live in the toggle's pause flow?
   
   ---
   Drafted-by: Claude Code (Opus 5.5); reviewed by @joshmarkovic before posting
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Pause a Dag for a set time and resume it automatically [airflow]

["boring-cyborg[bot] (via GitHub)" <gi...@apache.org>]

boring-cyborg[bot] commented on issue #74092:
URL: https://github.com/apache/airflow/issues/74092#issuecomment-5954157877

   Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org