You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "TSKVenkat (via GitHub)" <gi...@apache.org> on 2026/08/29 10:03:13 UTC
[PR] Fix scheduler stalling when a Dag switches to a coarser schedule [airflow]
TSKVenkat opened a new pull request, #72250:
URL: https://github.com/apache/airflow/pull/72250
Changing a Dag from an hourly to a daily cron stops it being scheduled again. The scheduler keeps logging `run already exists; skipping dagrun creation` for a `logical_date` that is already taken and never advances past it, so the Dag silently stops producing runs until rows are deleted by hand.
`_align_to_prev` maps the previous run's interval end back onto the new schedule. When the new cron is coarser than the one that produced that run, the result is the boundary the run already occupies, so `next_dagrun_info` proposes a `logical_date` identical to the existing run's.
Reproduced on `main` (3.4.0) with the reporter's scenario — an hourly run at `2026-05-04 00:00` with interval `[00:00, 01:00)`, then a switch to `0 0 * * *`:
```
previous run logical_date : 2026-05-04 00:00:00+00:00
proposed next logical_date: 2026-05-04 00:00:00+00:00 <- collides, scheduler stalls
```
and after the change:
```
proposed next logical_date: 2026-05-05 00:00:00+00:00 <- advances
```
A guard for this already existed, added in #66132, but it only covered the zero-length intervals `CronTriggerTimetable` leaves behind (`start == end`). The collision is not specific to those: it happens whenever the previous run came from any finer schedule. Comparing the proposed `logical_date` against the previous run's expresses the invariant the scheduler actually depends on — a proposed run must be strictly later than the last one — and subsumes the zero-length case, so the existing test for it still passes unchanged.
Tests: `test_coarser_schedule_does_not_re_emit_logical_date` covers the hourly-to-daily switch for both catchup settings. It fails on `main` (2 failed) and passes with this change, while `test_zero_length_last_interval_does_not_re_emit_logical_date` passes on both sides. Full `airflow-core/tests/unit/timetables/` run: 253 passed, 1 pre-existing failure in `test_workday_timetable.py` caused by `pandas` missing from my environment (it fails identically on unmodified `main`).
closes: #66754
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Code following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix scheduler stalling when a Dag switches to a coarser schedule [airflow]
Posted by "Miretpl (via GitHub)" <gi...@apache.org>.
Miretpl commented on PR #72250:
URL: https://github.com/apache/airflow/pull/72250#issuecomment-5645743757
The issue was fixed by #72498 PR, so closing this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix scheduler stalling when a Dag switches to a coarser schedule [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on PR #72250:
URL: https://github.com/apache/airflow/pull/72250#issuecomment-5461692599
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our [Contributors' Guide](https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:
- Pay attention to the quality of your code (ruff, mypy and type annotations). Our [prek-hooks]( https://github.com/apache/airflow/blob/main/contributing-docs/08_static_code_checks.rst#prerequisites-for-prek-hooks) will help you with that.
- In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/main/airflow-core/docs/howto/custom-operator.rst) Consider adding an example Dag that shows how users should use it.
- Consider using [Breeze environment](https://github.com/apache/airflow/blob/main/dev/breeze/doc/README.rst) for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
- Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
- Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
- Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#coding-style-and-best-practices).
- Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix scheduler stalling when a Dag switches to a coarser schedule [airflow]
Posted by "TSKVenkat (via GitHub)" <gi...@apache.org>.
TSKVenkat commented on PR #72250:
URL: https://github.com/apache/airflow/pull/72250#issuecomment-5505095793
Gentle ping, per the 72-hour guidance in the contributors' guide.
This is waiting on a maintainer to approve the CI workflow run — only 2 checks have executed so far, so the full suite has not had a chance to report. Happy to rebase or adjust anything if that would help.
Quick recap for whoever picks it up: changing a Dag from an hourly to a daily cron stops it being scheduled again, because the timetable proposes a `logical_date` the previous run already occupies and the scheduler loops on `run already exists; skipping dagrun creation`. #66132 added a guard for this but only for the zero-length intervals `CronTriggerTimetable` leaves behind; the same collision happens after any switch to a coarser cron. The change generalises that existing guard, so the test added with #66132 still passes untouched.
No rush at all — just flagging it since it had gone quiet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix scheduler stalling when a Dag switches to a coarser schedule [airflow]
Posted by "Miretpl (via GitHub)" <gi...@apache.org>.
Miretpl closed pull request #72250: Fix scheduler stalling when a Dag switches to a coarser schedule
URL: https://github.com/apache/airflow/pull/72250
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org