[PR] Include server error detail in Task SDK API error tracebacks [airflow]

["pierrejeambrun (via GitHub)" <gi...@apache.org>]

pierrejeambrun opened a new pull request, #71440:
URL: https://github.com/apache/airflow/pull/71440

   Internal SDK API client errors raise `ServerResponseError`, which deliberately keeps the server's error payload as structured `.detail` behind a generic message so handlers can log it as a structured field (`log.info(..., detail=e.detail)`). On paths where the error propagates **uncaught** to a generic logger — e.g. the executor / celery `trace_task` — only `str(exc)` (`"Server returned error"`) is shown and the detail is lost. That's the case in #57961 (the `task_instances.start()` path).
   
   This attaches the structured detail as an **exception note**, following the same `add_note` pattern already used for the correlation-id in `raise_on_4xx_5xx_with_note`. The detail then shows up in tracebacks that propagate uncaught, while `.detail` (structured, used by handlers) and the generic message are left unchanged. Exception notes require Python 3.11+, consistent with the existing correlation-id note (the reporter is on 3.12).
   
   closes: #57961
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   - [X] Yes — Claude Code (Opus 4.8)
   
   Generated-by: Claude Code (Opus 4.8) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Include server error detail in Task SDK API error tracebacks [airflow]

["amoghrajesh (via GitHub)" <gi...@apache.org>]

amoghrajesh commented on code in PR #71440:
URL: https://github.com/apache/airflow/pull/71440#discussion_r3759091599


##########
task-sdk/src/airflow/sdk/api/client.py:
##########
@@ -211,6 +211,13 @@ def raise_on_4xx_5xx_with_note(response: httpx.Response):
         e.add_note(
             f"Correlation-id={response.headers.get('correlation-id', None) or response.request.headers.get('correlation-id', 'no-correlation-id')}"
         )
+        # ServerResponseError keeps the server's error payload as structured ``detail`` behind a
+        # generic message; attach it as a note too so the details survive in tracebacks that
+        # propagate uncaught to a generic logger (e.g. the executor), not only where a handler
+        # logs ``e.detail`` explicitly.

Review Comment:
   ```suggestion
           # .detail sits behind the generic message and only reaches logs where a handler
           # logs it by hand. Add it as a note too, so uncaught paths (e.g. the executor) keep it.
   
   ```
   
   nit.



##########
task-sdk/tests/task_sdk/api/test_client.py:
##########
@@ -197,6 +198,24 @@ def test_server_response_error_pickling(self):
         assert unpickled.response.status_code == 404
         assert unpickled.request.url == "http://error"
 
+    @pytest.mark.skipif(sys.version_info < (3, 11), reason="Exception notes (PEP 678) require Python 3.11")
+    def test_server_error_detail_added_as_note(self):
+        """The structured ``detail`` is attached as an exception note so it survives in tracebacks
+        that propagate uncaught to a generic logger (e.g. the executor), where only the generic
+        ``str(exc)`` message would otherwise be shown."""

Review Comment:
   ```suggestion
            """Notes survive uncaught propagation, handled sites still log detail directly."""
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Include server error detail in Task SDK API error tracebacks [airflow]

["github-actions[bot] (via GitHub)" <gi...@apache.org>]

github-actions[bot] commented on PR #71440:
URL: https://github.com/apache/airflow/pull/71440#issuecomment-5265829393

   ### Backport successfully created: v3-3-test
   
   Note: As of [Merging PRs targeted for Airflow 3.X](https://github.com/apache/airflow/blob/main/dev/README_AIRFLOW3_DEV.md#merging-prs-targeted-for-airflow-3x)
   the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.
   
   In matter of doubt please ask in [#release-management](https://apache-airflow.slack.com/archives/C03G9H97MM2) Slack channel.
   
   <table>
           <tr>
           <th>Status</th>
           <th>Branch</th>
           <th>Result</th>
       </tr>
       <tr>
           <td>✅</td>
           <td>v3-3-test</td>
           <td><a href="https://github.com/apache/airflow/pull/71491"><img src="https://img.shields.io/badge/PR-71491-blue" alt="PR Link"></a></td>
       </tr>
   </table>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] Include server error detail in Task SDK API error tracebacks [airflow]

["pierrejeambrun (via GitHub)" <gi...@apache.org>]

pierrejeambrun merged PR #71440:
URL: https://github.com/apache/airflow/pull/71440


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org