You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2026/07/13 11:27:02 UTC

[PR] [v3-3-test] Fix scheduler firing on_failure_callback for heartbeat-timed-out retries (#66767) [airflow]

github-actions[bot] opened a new pull request, #69824:
URL: https://github.com/apache/airflow/pull/69824

   When a worker stops heartbeating (OOMKill, node eviction), the scheduler's
   ``_purge_task_instances_without_heartbeats`` built a ``TaskCallbackRequest``
   without ``task_callback_type``. The Dag processor's task-callback dispatch
   branches on that field: ``UP_FOR_RETRY`` runs ``on_retry_callback``, anything
   else (including ``None``) runs ``on_failure_callback``. So heartbeat-timeout
   cleanup always fired ``on_failure_callback`` even when the task still had
   retries remaining, producing spurious failure alerts for tasks that
   ultimately succeeded on retry.
   
   Set ``task_callback_type`` from ``ti.is_eligible_to_retry()``, the canonical
   retry-eligibility predicate, guarded by ``max_tries > 0``. The guard covers
   the one gap the predicate has here: this path doesn't load ``ti.task``, so the
   predicate falls back to ``try_number <= max_tries`` and drops the
   retries-configured check its task-loaded branch applies. Deferring to the
   predicate also keeps a ``RESTARTING`` task (cleared while running) retry-
   eligible past ``max_tries``, where a hand-rolled ``try_number <= max_tries``
   check would have fired ``on_failure_callback``.
   
   closes: #65400
   (cherry picked from commit f2403ccb58b32ae9a073b70d6648c11447d6c9b0)
   
   Co-authored-by: Stefan Wang <1f...@gmail.com>
   Signed-off-by: 1fanwang <1f...@gmail.com>
   Co-authored-by: kimhaggie <ki...@gmail.com>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [v3-3-test] Fix scheduler firing on_failure_callback for heartbeat-timed-out retries (#66767) [airflow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #69824:
URL: https://github.com/apache/airflow/pull/69824#issuecomment-4958069165

   Hi maintainer, this PR was merged without a milestone set.
   We've automatically set the milestone to **[Airflow 3.3.1](https://github.com/apache/airflow/milestone/146)** based on: bug fix merged to version branch
   If this milestone is not correct, please update it to the appropriate milestone.
   
   > This comment was generated by [Milestone Tag Assistant](https://github.com/apache/airflow/blob/main/.github/workflows/milestone-tag-assistant.yml).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [v3-3-test] Fix scheduler firing on_failure_callback for heartbeat-timed-out retries (#66767) [airflow]

Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal merged PR #69824:
URL: https://github.com/apache/airflow/pull/69824


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org