nathadfield opened a new pull request, #61132:
URL: https://github.com/apache/airflow/pull/61132
## Description
Fixes a regression in Airflow 3.x where deferrable sensors with `soft_fail=True` fail with `AirflowSensorTimeout` instead of being marked as SKIPPED when they timeout. This breaks compatibility with Airflow 2.x behavior and user expectations.
## Root Cause
In `BaseSensorOperator.resume_execution()`, the exception handling order was incorrect:
1. `TaskDeferralTimeout` was caught and immediately converted to `AirflowSensorTimeout`
2. The `soft_fail`/`never_fail` checks only caught `AirflowException` and `TaskDeferralError`
3. By the time these checks ran, the timeout was already converted, causing the flags to be ignored
## Solution
- Use nested try-except blocks to convert `TaskDeferralTimeout` → `AirflowSensorTimeout` while ensuring both exceptions are caught by the outer handler
- Check `soft_fail` and `never_fail` flags before raising `AirflowSensorTimeout`
- Align error messages with the `execute()` method for consistency
## Changes
**`task-sdk/src/airflow/sdk/bases/sensor.py`:**
- Modified `resume_execution()` to use nested exception handling
- Changed to separate `soft_fail` and `never_fail` checks with explicit error messages
**`task-sdk/tests/task_sdk/bases/test_sensor.py`:**
- Added tests for timeout scenarios with `soft_fail=True/False`
- Added tests for timeout scenarios with `never_fail=True`
- Added tests for trigger failure scenarios with `soft_fail=True/False`
- Added tests for trigger failure scenarios with `never_fail=True`
- Fixed existing tests to use `TriggerFailureReason` enum values
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
<!--
Generated-by: Claude Code following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
kaxil merged PR #61132: URL: https://github.com/apache/airflow/pull/61132 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org
nathadfield commented on code in PR #61132:
URL: https://github.com/apache/airflow/pull/61132#discussion_r2735567747
##########
task-sdk/src/airflow/sdk/bases/sensor.py:
##########
@@ -250,12 +250,15 @@ def run_duration() -> float:
def resume_execution(self, next_method: str, next_kwargs: dict[str, Any] | None, context: Context):
try:
Review Comment:
Thanks! Comment added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
github-actions[bot] commented on PR #61132:
URL: https://github.com/apache/airflow/pull/61132#issuecomment-3844494660
### Backport successfully created: v3-1-test
<table>
<tr>
<th>Status</th>
<th>Branch</th>
<th>Result</th>
</tr>
<tr>
<td>✅</td>
<td>v3-1-test</td>
<td><a href="https://github.com/apache/airflow/pull/61421"><img src="https://img.shields.io/badge/PR-61421-blue" alt="PR Link"></a></td>
</tr>
</table>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
SameerMesiah97 commented on code in PR #61132:
URL: https://github.com/apache/airflow/pull/61132#discussion_r2733932165
##########
task-sdk/src/airflow/sdk/bases/sensor.py:
##########
@@ -250,12 +250,15 @@ def run_duration() -> float:
def resume_execution(self, next_method: str, next_kwargs: dict[str, Any] | None, context: Context):
try:
Review Comment:
I would add a comment here explaining why there are nested try/except blocks. Perhaps something like:
`Nested try/except ensures TaskDeferralTimeout conversion does not bypass soft_fail / never_fail handling.`
Makes it easier to reason about.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
nathadfield commented on PR #61132: URL: https://github.com/apache/airflow/pull/61132#issuecomment-3818598860 Hey folks! If any of you get the time to review this PR soon I'd be most appreciative. Many thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org For queries about this service, please contact Infrastructure at: users@infra.apache.org