You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "GayathriSrividya (via GitHub)" <gi...@apache.org> on 2026/06/01 09:52:45 UTC
[PR] Fix missing task instance history for non-running retries [airflow]
GayathriSrividya opened a new pull request, #67831:
URL: https://github.com/apache/airflow/pull/67831
Cherry-picks the proposed fix from #67371 for priority issue #67238.
This updates failure handling to preserve task instance history for retry paths when the task is not in RUNNING state (except RESTARTING, which is already handled).
Also includes the companion test adjustment to avoid detached-session behavior in .
Related: #67238
Related: #65366
## Local validation
- 2 files left unchanged
- All checks passed!
- [94mRunning command in Breeze: pytest [0m
[94mairflow-core/tests/unit/models/test_taskinstance.py::TestTaskInstance::test_hand[0m
[94mle_failure_updates_queued_task_updates_state [0m
[94mairflow-core/tests/unit/models/test_taskinstance.py::TestTaskInstance::test_task[0m
[94m_instance_history_is_created_when_ti_goes_for_retry -xs[0m
[94mUsing project name: breeze-breeze-run-d00b8e7e[0m
[93mCI image for Python [0m[1;93m3.10[0m[93m was never built locally or was deleted. Forcing build.[0m
[94mRunning command: [0m[1;94m<[0m[1;94mdocker[0m[94m inspect ghcr.io/apache/airflow/main/ci/python3.[0m[1;94m10[0m[1;94m>[0m
[94mWorking directory [0m[94m/Users/gayathrir/Documents/workspace/AI/[0m[94mfork[0m
[94mdocker inspect ghcr.io/apache/airflow/main/ci/python3.[0m[1;94m10[0m (failed locally: Docker not installed)
- Refreshed [1;36m102[0m providers with [1;36m2207[0m Python files.
============================= test session starts ==============================
platform darwin -- Python 3.13.13, pytest-9.0.3, pluggy-1.6.0 -- /Users/gayathrir/Documents/workspace/AI/fork/.venv/bin/python
cachedir: .pytest_cache
rootdir: /Users/gayathrir/Documents/workspace/AI/fork
configfile: pyproject.toml
plugins: unordered-0.7.0, mock-3.15.1, instafail-0.5.0, cov-7.1.0, timeouts-1.2.1, xdist-3.8.0, asyncio-1.4.0, custom-exit-code-0.3.0, time-machine-3.2.0, rerunfailures-16.3, icdiff-0.9, kgb-7.3, requests-mock-1.12.1, anyio-4.13.0
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=function, asyncio_default_test_loop_scope=function
setup timeout: 0.0s, execution timeout: 0.0s, teardown timeout: 0.0s
collecting ... collected 0 items / 1 error
==================================== ERRORS ====================================
_____ ERROR collecting airflow-core/tests/unit/models/test_taskinstance.py _____
ImportError while importing test module '/Users/gayathrir/Documents/workspace/AI/fork/airflow-core/tests/unit/models/test_taskinstance.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
.venv/lib/python3.13/site-packages/_pytest/python.py:507: in importtestmodule
mod = import_path(
.venv/lib/python3.13/site-packages/_pytest/pathlib.py:587: in import_path
importlib.import_module(module_name)
../../../../.local/share/uv/python/cpython-3.13.13-macos-aarch64-none/lib/python3.13/importlib/__init__.py:88: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<frozen importlib._bootstrap>:1395: in _gcd_import
???
<frozen importlib._bootstrap>:1360: in _find_and_load
???
<frozen importlib._bootstrap>:1310: in _find_and_load_unlocked
???
<frozen importlib._bootstrap>:488: in _call_with_frames_removed
???
<frozen importlib._bootstrap>:1395: in _gcd_import
???
<frozen importlib._bootstrap>:1360: in _find_and_load
???
<frozen importlib._bootstrap>:1331: in _find_and_load_unlocked
???
<frozen importlib._bootstrap>:935: in _load_unlocked
???
<frozen importlib._bootstrap_external>:1023: in exec_module
???
<frozen importlib._bootstrap>:488: in _call_with_frames_removed
???
airflow-core/tests/unit/models/__init__.py:22: in <module>
from airflow._shared.timezones import timezone
airflow-core/src/airflow/__init__.py:64: in <module>
from airflow import configuration, settings
airflow-core/src/airflow/configuration.py:821: in <module>
conf: AirflowConfigParser = initialize_config()
^^^^^^^^^^^^^^^^^^^
airflow-core/src/airflow/configuration.py:693: in initialize_config
if airflow_config_parser.getboolean("core", "unit_test_mode"):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
airflow-core/src/airflow/_shared/configuration/parser.py:1368: in getboolean
val = str(self.get(section, key, _extra_stacklevel=1, **kwargs)).lower().strip()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
airflow-core/src/airflow/_shared/configuration/parser.py:1334: in get
section, key, deprecated_section, deprecated_key, warning_emitted = self._resolve_deprecated_lookup(
airflow-core/src/airflow/_shared/configuration/parser.py:1230: in _resolve_deprecated_lookup
option_description = self.configuration_description.get(section, {}).get("options", {}).get(key, {})
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../../../../.local/share/uv/python/cpython-3.13.13-macos-aarch64-none/lib/python3.13/functools.py:1025: in __get__
val = self.func(instance)
^^^^^^^^^^^^^^^^^^^
airflow-core/src/airflow/_shared/configuration/parser.py:322: in configuration_description
for section, section_content in self._provider_metadata_configuration_description.items():
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../../../../.local/share/uv/python/cpython-3.13.13-macos-aarch64-none/lib/python3.13/functools.py:1025: in __get__
val = self.func(instance)
^^^^^^^^^^^^^^^^^^^
airflow-core/src/airflow/_shared/configuration/parser.py:411: in _provider_metadata_configuration_description
for _, config in self._provider_manager_type().provider_configs:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
airflow-core/src/airflow/providers_manager.py:478: in __getattribute__
return object.__getattribute__(self, name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
airflow-core/src/airflow/providers_manager.py:1504: in provider_configs
self.initialize_providers_configuration()
airflow-core/src/airflow/providers_manager.py:364: in wrapped_function
func(*args, **kwargs)
airflow-core/src/airflow/providers_manager.py:625: in initialize_providers_configuration
self.initialize_providers_list()
airflow-core/src/airflow/providers_manager.py:364: in wrapped_function
func(*args, **kwargs)
airflow-core/src/airflow/providers_manager.py:514: in initialize_providers_list
discover_all_providers_from_packages(self._provider_dict, self._provider_schema_validator)
airflow-core/src/airflow/_shared/providers_discovery/providers_discovery.py:324: in discover_all_providers_from_packages
provider_info = entry_point.load()()
^^^^^^^^^^^^^^^^^^
../../../../.local/share/uv/python/cpython-3.13.13-macos-aarch64-none/lib/python3.13/importlib/metadata/__init__.py:179: in load
module = import_module(match.group('module'))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
../../../../.local/share/uv/python/cpython-3.13.13-macos-aarch64-none/lib/python3.13/importlib/__init__.py:88: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E ModuleNotFoundError: No module named 'airflow.providers.ibm'
=========================== short test summary info ============================
ERROR airflow-core/tests/unit/models/test_taskinstance.py
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
========================= 1 warning, 1 error in 0.39s ========================== (failed locally due missing provider module in environment)
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — GitHub Copilot (GPT-5.3-Codex)
Generated-by: GitHub Copilot (GPT-5.3-Codex) following the project guidelines.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix missing task instance history for non-running retries [airflow]
Posted by "GayathriSrividya (via GitHub)" <gi...@apache.org>.
GayathriSrividya commented on PR #67831:
URL: https://github.com/apache/airflow/pull/67831#issuecomment-4966548421
> The guard flip from `ti.state == RUNNING` to `ti.state != RESTARTING` in `fetch_handle_failure_context` looks right. A retry-eligible failure from QUEUED or DEFERRED never reached `prepare_db_for_next_try` before, so its history row got silently dropped, and RESTARTING is correctly the one state left out since the clear already recorded it. That lines up with `is_eligible_to_retry`, where RESTARTING is the always eligible cleared-while-running path. The `ti = session.merge(ti)` change in `test_handle_failure_updates_queued_task_updates_state` is right too, since the old line discarded the instance the session actually tracks. I checked that RESTARTING and RUNNING both behave the same under either guard, so this only closes the non-running gap.
Thanks for validating the guard change and the test fix. I’ll wait for formal review and follow up if anything else comes up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix missing task instance history for non-running retries [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #67831:
URL: https://github.com/apache/airflow/pull/67831#issuecomment-5839360247
Hello @GayathriSrividya - thank you for your contributions to Apache Airflow!
The Airflow community has introduced a limit of **5 open pull requests at a time** for contributors without write access to the repository. You currently have 7 open pull requests, so - as a one-time step of introducing the limit - we closed the ones where maintainers have not engaged yet:
* #67831 - Fix missing task instance history for non-running retries
* #67862 - Fix AttributeError when notifier renders templates with SerializedDAG
* #67998 - Execution API: Resolve retry callback team via TI history
* #68125 - Fix backfill completion race during creation
* #69878 - Show clear error for invalid sql_alchemy_conn at startup
* #69972 - Fix dag parsing persistence when bundle cache is missing after processor restart
These pull requests stay open because maintainers are already engaged in them - they count towards your limit:
* #68499 - Fix JWT token not refreshed when token expires mid-request
This is **not** a judgement of you or of your changes. We never told contributors before that opening many pull requests at once was a problem, so there is nothing to feel bad about - and nothing is lost: your branches, commits and the review history stay where they are.
What we ask you to do is to make your **first prioritization decision**: choose which of the pull requests above matter most to you, and reopen them (up to 5 open at a time, including the ones still open) with the "Reopen pull request" button or `gh pr reopen <PR_NUMBER> --repo apache/airflow`. Reopen the ones you are ready to follow through - keep them rebased, respond to review comments and fix failing checks.
While your pull requests are waiting for review, the most valuable thing you can do is help in other ways - reviewing other contributors' pull requests, helping with issues, and taking part in the discussions on the devlist and Slack.
Why we introduced the limit, what it means for you and how to reopen or restore a pull request is explained in https://github.com/apache/airflow/blob/main/contributing-docs/32_open_pull_request_limit.rst.
---
Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix missing task instance history for non-running retries [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed pull request #67831: Fix missing task instance history for non-running retries
URL: https://github.com/apache/airflow/pull/67831
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix missing task instance history for non-running retries [airflow]
Posted by "Vamsi-klu (via GitHub)" <gi...@apache.org>.
Vamsi-klu commented on PR #67831:
URL: https://github.com/apache/airflow/pull/67831#issuecomment-4966077883
The guard flip from `ti.state == RUNNING` to `ti.state != RESTARTING` in `fetch_handle_failure_context` looks right. A retry-eligible failure from QUEUED or DEFERRED never reached `prepare_db_for_next_try` before, so its history row got silently dropped, and RESTARTING is correctly the one state left out since the clear already recorded it. That lines up with `is_eligible_to_retry`, where RESTARTING is the always eligible cleared-while-running path. The `ti = session.merge(ti)` change in `test_handle_failure_updates_queued_task_updates_state` is right too, since the old line discarded the instance the session actually tracks. I checked that RESTARTING and RUNNING both behave the same under either guard, so this only closes the non-running gap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org