You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "YoannAbriel (via GitHub)" <gi...@apache.org> on 2026/03/10 10:12:35 UTC
[PR] fix(scheduler): set end_date on tasks skipped by dagrun timeout [airflow]
YoannAbriel opened a new pull request, #63250:
URL: https://github.com/apache/airflow/pull/63250
## Problem
When a DAG run times out via `dagrun_timeout`, unfinished task instances are marked `SKIPPED` but their `end_date` is never set. The UI computes duration as `now - start_date`, so skipped tasks show a continuously increasing duration — confusing and incorrect.
## Root Cause
In `SchedulerJobRunner._schedule_dag_run`, the timeout handler sets `task_instance.state = TaskInstanceState.SKIPPED` but doesn't touch `end_date`. Without an `end_date`, the UI falls back to computing live duration.
## Fix
Set `end_date = timezone.utcnow()` alongside the state change for all tasks skipped by dagrun timeout. Added a test that creates a timed-out DAG run with running tasks and verifies `end_date` is populated after the scheduler processes it.
Closes: #58536
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Code
Generated-by: Claude Code following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
---
* Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(scheduler): set end_date on tasks skipped by dagrun timeout [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #63250:
URL: https://github.com/apache/airflow/pull/63250#issuecomment-4034429821
Did you actually check if it works? Or is that change just PR-ed without actually running some real tests? Can you show screenshots and logs from the execution?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(scheduler): set end_date on tasks skipped by dagrun timeout [airflow]
Posted by "YoannAbriel (via GitHub)" <gi...@apache.org>.
YoannAbriel commented on PR #63250:
URL: https://github.com/apache/airflow/pull/63250#issuecomment-4042253948
Reproduced and verified on Airflow 3.1.8 (standalone, SequentialExecutor). Test DAG with `dagrun_timeout=timedelta(seconds=30)` and 3 tasks chained: `quick_task` (5s) -> `slow_task` (120s) -> `final_task` (5s).
**Before fix (unpatched 3.1.8):**
| Task | State | end_date |
|------|-------|----------|
| quick_task | success | 2026-03-11 21:05:05 |
| slow_task | skipped | None |
| final_task | None | None |
**After fix:**
| Task | State | end_date |
|------|-------|----------|
| quick_task | success | 2026-03-11 21:09:21 |
| slow_task | skipped | 2026-03-11 21:09:45 |
| final_task | None | None (never started, expected) |

Two lines in `_schedule_dag_run` — `now = timezone.utcnow()` and `task_instance.end_date = now` before merging skipped tasks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(scheduler): set end_date on tasks skipped by dagrun timeout [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #63250:
URL: https://github.com/apache/airflow/pull/63250#issuecomment-4034433398
explanation: This is a **vital** part of airflow in order to modify scheduler you need to know exactly what you are doing and actually test it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(scheduler): set end_date on tasks skipped by dagrun timeout [airflow]
Posted by "YoannAbriel (via GitHub)" <gi...@apache.org>.
YoannAbriel commented on PR #63250:
URL: https://github.com/apache/airflow/pull/63250#issuecomment-4034739694
Fair — unit tests only so far. Will set up a local environment with a DAG that hits the timeout path and share logs/screenshots.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(scheduler): set end_date on tasks skipped by dagrun timeout [airflow]
Posted by "Vamsi-klu (via GitHub)" <gi...@apache.org>.
Vamsi-klu commented on PR #63250:
URL: https://github.com/apache/airflow/pull/63250#issuecomment-4764226387
This should not set only `end_date`. The timeout path creates terminal skipped TIs but bypasses normal duration bookkeeping, so task-instance APIs and table views reading `TI.duration` can still show blank/stale duration. Please use the same state transition helper or explicitly set duration from `start_date` and `now`, and extend the test to assert `duration`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(scheduler): set end_date on tasks skipped by dagrun timeout [airflow]
Posted by "YoannAbriel (via GitHub)" <gi...@apache.org>.
YoannAbriel commented on PR #63250:
URL: https://github.com/apache/airflow/pull/63250#issuecomment-4042257078
Reproduced and verified on Airflow 3.1.8 (standalone, SequentialExecutor). Test DAG with `dagrun_timeout=timedelta(seconds=30)` and 3 tasks chained: `quick_task` (5s) -> `slow_task` (120s) -> `final_task` (5s).
**Before fix (unpatched 3.1.8):**
| Task | State | end_date |
|------|-------|----------|
| quick_task | success | 2026-03-11 21:05:05 |
| slow_task | skipped | None |
| final_task | None | None |
**After fix:**
| Task | State | end_date |
|------|-------|----------|
| quick_task | success | 2026-03-11 21:09:21 |
| slow_task | skipped | 2026-03-11 21:09:45 |
| final_task | None | None (never started, expected) |
Two lines in `_schedule_dag_run` — `now = timezone.utcnow()` and `task_instance.end_date = now` before merging skipped tasks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org