You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "deepujain (via GitHub)" <gi...@apache.org> on 2026/09/07 16:51:30 UTC
[PR] Keep scheduled tasks when serialized DAG is missing (#62050) [airflow]
deepujain opened a new pull request, #72652:
URL: https://github.com/apache/airflow/pull/72652
## Summary
Keep scheduled task instances pending when their serialized DAG is temporarily unavailable, instead of bulk-failing every scheduled task for that DAG.
## Changes
- Preserve scheduled task instances during a transient serialized-DAG lookup failure.
- Exclude the affected DAG from the current scheduling batch so healthy DAGs can continue to use available capacity.
- Add regression coverage for the retry-on-a-later-tick behavior.
## Test plan
- [ ] `uv run --project airflow-core pytest airflow-core/tests/unit/jobs/test_scheduler_job.py::TestSchedulerJob::test_missing_serialized_dag_keeps_scheduled_tasks_for_a_later_tick -xvs` (not available in this environment)
- [x] `git diff --check`
- [ ] CI passes
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (OpenAI Codex)
Generated-by: OpenAI Codex following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
Fixes #62050
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Keep scheduled tasks when serialized Dag is missing [airflow]
Posted by "deepujain (via GitHub)" <gi...@apache.org>.
deepujain commented on PR #72652:
URL: https://github.com/apache/airflow/pull/72652#issuecomment-6104939602
@ashb @XD-DENG Checks are green on the latest commit. Could you review this when you have a chance?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Keep scheduled tasks when serialized DAG is missing (#62050) [airflow]
Posted by "deepujain (via GitHub)" <gi...@apache.org>.
deepujain commented on PR #72652:
URL: https://github.com/apache/airflow/pull/72652#issuecomment-5748220496
The lowest-deps CI failure (`test_form_parser_error_keeps_its_own_detail`, 422 vs 400) traces to the starlette floor on this branch. With `starlette>=1.0.1`, the urlencoded form parser does not bound the field count, so the 2000-field test body parses cleanly and then fails pydantic validation (422) instead of raising starlette's own 400. Upstream addressed this in #73302 by raising the floor to `starlette>=1.3.1`, whose `FormParser` enforces `max_fields` and reports a 400.
I have cherry-picked #73302 onto this branch so the lowest-deps job resolves the bounded parser. No changes to the PR's own code; CI should go green on the rerun.
---
Drafted-by: Muse Spark (Meta) (no human review before posting)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Keep scheduled tasks when serialized DAG is missing (#62050) [airflow]
Posted by "deepujain (via GitHub)" <gi...@apache.org>.
deepujain commented on PR #72652:
URL: https://github.com/apache/airflow/pull/72652#issuecomment-5780006028
Rebased the missing-serialized-Dag fix onto current main. The scheduled-task regression passes in Breeze, with Ruff lint and formatting checks passing. CI is running on the updated head.
---
Drafted-by: OpenAI Codex (no human review before posting)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Keep scheduled tasks when serialized Dag is missing [airflow]
Posted by "deepujain (via GitHub)" <gi...@apache.org>.
deepujain closed pull request #72652: Keep scheduled tasks when serialized Dag is missing
URL: https://github.com/apache/airflow/pull/72652
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Keep scheduled tasks when serialized Dag is missing [airflow]
Posted by "deepujain (via GitHub)" <gi...@apache.org>.
deepujain commented on PR #72652:
URL: https://github.com/apache/airflow/pull/72652#issuecomment-6113686362
Thanks for maintaining this project. I've kept this PR rebased and up to date, but it hasn't found a reviewer yet, so I'm closing it to keep your queue tidy. Happy to reopen and rebase anytime if it's still useful.
Effort: open since Sep 7, 2026, rebased 2 times, 4 updates posted, 2 commits, +9 −11 across 2 files, checks green on the latest commit, fixes #62050.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Keep scheduled tasks when serialized DAG is missing (#62050) [airflow]
Posted by "deepujain (via GitHub)" <gi...@apache.org>.
deepujain commented on PR #72652:
URL: https://github.com/apache/airflow/pull/72652#issuecomment-5789231324
Rebased onto main at b6538c32. The scheduled-task regression for a missing serialized Dag passes in Breeze, with clean Ruff lint and formatting checks. Fresh hosted CI is starting on the updated head.
---
Drafted-by: OpenAI Codex (no human review before posting)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org