You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "seanmuth (via GitHub)" <gi...@apache.org> on 2026/09/23 14:49:19 UTC
[PR] [v3-3-test] Fix backfill DagRuns being permanently starved from scheduling (#72730) [airflow]
seanmuth opened a new pull request, #73620:
URL: https://github.com/apache/airflow/pull/73620
`DagRun.get_running_dag_runs_to_examine` ordered RUNNING DagRuns by `BackfillDagRun.sort_ordinal` (nulls first) ahead of everything else, so every non-backfill RUNNING DagRun (which has no matching `BackfillDagRun` row, and therefore a NULL `sort_ordinal`) sorted strictly ahead of every backfill DagRun regardless of how long that backfill run had been waiting. Combined with the per-loop `LIMIT max_dagruns_per_loop_to_schedule` (default 20), this meant a backfill DagRun's task instances could never be created or scheduled at all once the number of concurrently RUNNING non-backfill DagRuns met or exceeded that limit.
This removes `sort_ordinal` from that query entirely, so a RUNNING backfill DagRun competes purely on `last_scheduling_decision` (nulls first) like every other RUNNING DagRun. Deprioritizing backfills while `QUEUED` is unaffected, that's handled separately by `get_queued_dag_runs_to_set_running` via the backfill's own `max_active_runs`.
(cherry picked from commit a7b60ec14ef6d98169d84ddce37c933c3016fb3d)
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (Claude, Sonnet 5)
<!-- Generated-by: Claude Sonnet 5 following https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [v3-3-test] Fix backfill DagRuns being permanently starved from scheduling (#72730) [airflow]
Posted by "seanmuth (via GitHub)" <gi...@apache.org>.
seanmuth closed pull request #73620: [v3-3-test] Fix backfill DagRuns being permanently starved from scheduling (#72730)
URL: https://github.com/apache/airflow/pull/73620
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [v3-3-test] Fix backfill DagRuns being permanently starved from scheduling (#72730) [airflow]
Posted by "seanmuth (via GitHub)" <gi...@apache.org>.
seanmuth commented on PR #73620:
URL: https://github.com/apache/airflow/pull/73620#issuecomment-5797248124
Closing — opened this without checking first. Will follow up separately if/when a backport is actually wanted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org