You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/07/13 11:27:02 UTC
[PR] [v3-3-test] Fix scheduler firing on_failure_callback for heartbeat-timed-out retries (#66767) [airflow]
github-actions[bot] opened a new pull request, #69824:
URL: https://github.com/apache/airflow/pull/69824
When a worker stops heartbeating (OOMKill, node eviction), the scheduler's
``_purge_task_instances_without_heartbeats`` built a ``TaskCallbackRequest``
without ``task_callback_type``. The Dag processor's task-callback dispatch
branches on that field: ``UP_FOR_RETRY`` runs ``on_retry_callback``, anything
else (including ``None``) runs ``on_failure_callback``. So heartbeat-timeout
cleanup always fired ``on_failure_callback`` even when the task still had
retries remaining, producing spurious failure alerts for tasks that
ultimately succeeded on retry.
Set ``task_callback_type`` from ``ti.is_eligible_to_retry()``, the canonical
retry-eligibility predicate, guarded by ``max_tries > 0``. The guard covers
the one gap the predicate has here: this path doesn't load ``ti.task``, so the
predicate falls back to ``try_number <= max_tries`` and drops the
retries-configured check its task-loaded branch applies. Deferring to the
predicate also keeps a ``RESTARTING`` task (cleared while running) retry-
eligible past ``max_tries``, where a hand-rolled ``try_number <= max_tries``
check would have fired ``on_failure_callback``.
closes: #65400
(cherry picked from commit f2403ccb58b32ae9a073b70d6648c11447d6c9b0)
Co-authored-by: Stefan Wang <1f...@gmail.com>
Signed-off-by: 1fanwang <1f...@gmail.com>
Co-authored-by: kimhaggie <ki...@gmail.com>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [v3-3-test] Fix scheduler firing on_failure_callback for heartbeat-timed-out retries (#66767) [airflow]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #69824:
URL: https://github.com/apache/airflow/pull/69824#issuecomment-4958069165
Hi maintainer, this PR was merged without a milestone set.
We've automatically set the milestone to **[Airflow 3.3.1](https://github.com/apache/airflow/milestone/146)** based on: bug fix merged to version branch
If this milestone is not correct, please update it to the appropriate milestone.
> This comment was generated by [Milestone Tag Assistant](https://github.com/apache/airflow/blob/main/.github/workflows/milestone-tag-assistant.yml).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [v3-3-test] Fix scheduler firing on_failure_callback for heartbeat-timed-out retries (#66767) [airflow]
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal merged PR #69824:
URL: https://github.com/apache/airflow/pull/69824
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org