You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "henry3260 (via GitHub)" <gi...@apache.org> on 2025/11/15 13:28:31 UTC

[PR] Fix XCom key handling when keys contain special characters like slash [airflow]

henry3260 opened a new pull request, #58344:
URL: https://github.com/apache/airflow/pull/58344

   This PR picks up the work from @Brunda10 in PR #56042, which had become stale.
   
   * Fixes 404 errors for XCom keys containing special characters (e.g., /).
   * Keys are now URL-encoded (quoted) before being sent to the API.
   * Keys are decoded (unquoted) when received from the API.
   * Adds required unit tests to verify the fix, as requested in the original PR.
   
   Closes: #55410
   
   <!-- Please keep an empty line above the dashes. -->
   ---
   **^ Add meaningful description above**
   Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information.
   In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
   In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
   In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on PR #58344:
URL: https://github.com/apache/airflow/pull/58344#issuecomment-3641402095

   Thanks for the PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "henry3260 (via GitHub)" <gi...@apache.org>.
henry3260 commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2532667465


##########
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_xcom.py:
##########
@@ -161,6 +162,18 @@ def test_should_raise_404_for_non_existent_xcom(self, test_client):
         assert response.status_code == 404
         assert response.json()["detail"] == f"XCom entry with key: `{TEST_XCOM_KEY_2}` not found"
 
+    def test_should_respond_200_native_with_slash_key(self, test_client):
+        slash_key = "folder/sub/value"
+        self._create_xcom(slash_key, TEST_XCOM_VALUE)
+        encoded_key = quote(slash_key, safe="")
+        response = test_client.get(
+            f"/dags/{TEST_DAG_ID}/dagRuns/{run_id}/taskInstances/{TEST_TASK_ID}/xcomEntries/{encoded_key}"

Review Comment:
   Yes, you are right! Yes, because I use [{xcom_key:path}](vscode-file://vscode-app/c:/Users/Henry%20Chen/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) in the route definition, we can directly use `/dags/my_Dag/dagRuns/run_1/taskInstances/task_1/xcomEntries/folder/sub/value `without encoding.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "henry3260 (via GitHub)" <gi...@apache.org>.
henry3260 commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2610156556


##########
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_xcom.py:
##########
@@ -673,7 +702,8 @@ def test_patch_xcom_entry(self, key, patch_body, expected_status, expected_detai
         # Ensure the XCom entry exists before updating
         if expected_status != 404:
             self._create_xcom(TEST_XCOM_KEY, TEST_XCOM_VALUE)
-            new_value = XComModel.serialize_value(patch_body["value"])
+            # The value is double-serialized: first json.dumps(patch_body["value"]), then json.dumps() again
+            new_value = json.dumps(json.dumps(patch_body["value"]))

Review Comment:
   It’s because we call json.dump twice.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2557065624


##########
task-sdk/src/airflow/sdk/api/client.py:
##########
@@ -460,6 +462,7 @@ def get(
             params.update({"map_index": map_index})
         if include_prior_dates:
             params.update({"include_prior_dates": include_prior_dates})
+        key = quote(key, safe="")

Review Comment:
   not sure if this is needed on the client side. I don't think so 



##########
task-sdk/src/airflow/sdk/api/client.py:
##########
@@ -517,6 +521,7 @@ def delete(
         params = {}
         if map_index is not None and map_index >= 0:
             params = {"map_index": map_index}
+        key = quote(key, safe="")

Review Comment:
   same for others



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "jason810496 (via GitHub)" <gi...@apache.org>.
jason810496 commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2553711712


##########
airflow-core/src/airflow/api_fastapi/execution_api/routes/xcoms.py:
##########
@@ -88,111 +88,15 @@ async def xcom_query(
     return query
 
 
-@router.head(

Review Comment:
   It would be better to not change the definition order to make the line difference as less as possible.
   Having `key: path` and `key: Annotated[str, Path(min_length=1)],` for each router would be enough. In case we accidentally change some detail during the copy paste. Thanks!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2532035775


##########
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_xcom.py:
##########
@@ -161,6 +162,18 @@ def test_should_raise_404_for_non_existent_xcom(self, test_client):
         assert response.status_code == 404
         assert response.json()["detail"] == f"XCom entry with key: `{TEST_XCOM_KEY_2}` not found"
 
+    def test_should_respond_200_native_with_slash_key(self, test_client):
+        slash_key = "folder/sub/value"
+        self._create_xcom(slash_key, TEST_XCOM_VALUE)
+        encoded_key = quote(slash_key, safe="")
+        response = test_client.get(
+            f"/dags/{TEST_DAG_ID}/dagRuns/{run_id}/taskInstances/{TEST_TASK_ID}/xcomEntries/{encoded_key}"

Review Comment:
   I would have expected this path to "just" be this:
   `/dags/my_Dag/dagRuns/run_1/taskInstances/task_1/xcomEntries/folder/sub/value`
   
   Is there a reason you didn't.
   
   When _actually_ running starlette+FastAPI, does it treat these cases any differently?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2610124820


##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/xcom.py:
##########
@@ -257,7 +257,7 @@ def create_xcom_entry(
         )
 
     try:
-        value = json.dumps(request_body.value)
+        value = XComModel.serialize_value(request_body.value)

Review Comment:
   I think there is a specific PR for that (https://github.com/apache/airflow/pull/59129) to handle double serialization problem. We shouldn't update any of that in this PR, it is out of scope. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #58344:
URL: https://github.com/apache/airflow/pull/58344#issuecomment-3641408825

   ### Backport successfully created: v3-1-test
   
   <table>
                   <tr>
                       <th>Status</th>
                       <th>Branch</th>
                       <th>Result</th>
                   </tr>
                   <tr>
                       <td>✅</td>
                       <td>v3-1-test</td>
                       <td><a href="https://github.com/apache/airflow/pull/59311"><img src="https://img.shields.io/badge/PR-59311-blue" alt="PR Link"></a></td>
                   </tr>
               </table>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "henry3260 (via GitHub)" <gi...@apache.org>.
henry3260 commented on PR #58344:
URL: https://github.com/apache/airflow/pull/58344#issuecomment-3566461934

   > Nice!
   > 
   > It would be nice to add `pytest.mark.parametrize` with slash value key in `task-sdk-integration-tests/tests/task_sdk_tests/test_xcom_operations.py` tests.
   > 
   > Thanks!
   
   Thanks for reviewing!
   
   Slash keys aren’t supported today: the route pattern /{dag_id}/{run_id}/{task_id}/{key} treats {key} as a single path segment, so "folder/sub/value" splits into multiple segments and won’t match. Adding support means restructuring the routes (e.g. {key:path} plus reworking /item/{offset} and /slice), or switching to a query param (?key=). That’s a broader change, so I propose we keep the simple key test in this PR, optionally mark a slash-key test as xfail, and follow up with a dedicated issue/PR for proper support. If you prefer to fold the refactor into this PR, let me know—just want to confirm scope first.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "henry3260 (via GitHub)" <gi...@apache.org>.
henry3260 commented on PR #58344:
URL: https://github.com/apache/airflow/pull/58344#issuecomment-3612621759

   > @henry3260 Do you mind rebasing to fix the conflicts. We can probably go ahead and merge, the PR is looking good.
   
   Sure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2610146129


##########
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_xcom.py:
##########
@@ -673,7 +702,8 @@ def test_patch_xcom_entry(self, key, patch_body, expected_status, expected_detai
         # Ensure the XCom entry exists before updating
         if expected_status != 404:
             self._create_xcom(TEST_XCOM_KEY, TEST_XCOM_VALUE)
-            new_value = XComModel.serialize_value(patch_body["value"])
+            # The value is double-serialized: first json.dumps(patch_body["value"]), then json.dumps() again
+            new_value = json.dumps(json.dumps(patch_body["value"]))

Review Comment:
   Why do we need to change that ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "henry3260 (via GitHub)" <gi...@apache.org>.
henry3260 commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2532667465


##########
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_xcom.py:
##########
@@ -161,6 +162,18 @@ def test_should_raise_404_for_non_existent_xcom(self, test_client):
         assert response.status_code == 404
         assert response.json()["detail"] == f"XCom entry with key: `{TEST_XCOM_KEY_2}` not found"
 
+    def test_should_respond_200_native_with_slash_key(self, test_client):
+        slash_key = "folder/sub/value"
+        self._create_xcom(slash_key, TEST_XCOM_VALUE)
+        encoded_key = quote(slash_key, safe="")
+        response = test_client.get(
+            f"/dags/{TEST_DAG_ID}/dagRuns/{run_id}/taskInstances/{TEST_TASK_ID}/xcomEntries/{encoded_key}"

Review Comment:
   Yes, you are right! Yes, because I use [`{xcom_key:path}`](vscode-file://vscode-app/c:/Users/Henry%20Chen/AppData/Local/Programs/Microsoft%20VS%20Code/resources/app/out/vs/code/electron-browser/workbench/workbench.html) in the route definition, we can directly use `/dags/my_Dag/dagRuns/run_1/taskInstances/task_1/xcomEntries/folder/sub/value `without encoding.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "henry3260 (via GitHub)" <gi...@apache.org>.
henry3260 commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2607748348


##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/xcom.py:
##########
@@ -257,7 +257,7 @@ def create_xcom_entry(
         )
 
     try:
-        value = json.dumps(request_body.value)
+        value = XComModel.serialize_value(request_body.value)

Review Comment:
   > This change shouldn't be. It was purposely done in #58900
   
   https://github.com/apache/airflow/blob/925376664f80a2d8ca316b5579c8fe1954458265/airflow-core/src/airflow/api_fastapi/core_api/routes/public/xcom.py#L318-L339
   It seems` xcom_new_value` was already serialized on line 318. Calling `json.dumps()` again here causes double serialization. Is this intended?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "jason810496 (via GitHub)" <gi...@apache.org>.
jason810496 commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2608875015


##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/xcom.py:
##########
@@ -257,7 +257,7 @@ def create_xcom_entry(
         )
 
     try:
-        value = json.dumps(request_body.value)
+        value = XComModel.serialize_value(request_body.value)

Review Comment:
   I think we are referring to #59129 instead of #58900.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2610149302


##########
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_xcom.py:
##########
@@ -683,7 +713,7 @@ def test_patch_xcom_entry(self, key, patch_body, expected_status, expected_detai
         assert response.status_code == expected_status
 
         if expected_status == 200:
-            assert response.json()["value"] == XComModel.serialize_value(new_value)
+            assert response.json()["value"] == new_value

Review Comment:
   Same here?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2532030953


##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/xcom.py:
##########
@@ -80,6 +81,8 @@ def get_xcom_entry(
     stringify: Annotated[bool, Query()] = False,
 ) -> XComResponseNative | XComResponseString:
     """Get an XCom entry."""
+    if xcom_key:

Review Comment:
   This `if` is wrong -- xcom key is required. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "jason810496 (via GitHub)" <gi...@apache.org>.
jason810496 commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2553711712


##########
airflow-core/src/airflow/api_fastapi/execution_api/routes/xcoms.py:
##########
@@ -88,111 +88,15 @@ async def xcom_query(
     return query
 
 
-@router.head(

Review Comment:
   It would be better to not change the definition order in `airflow-core/src/airflow/api_fastapi/execution_api/routes/xcoms.py` to make the line difference as less as possible.
   
   Having `key: path` and `key: Annotated[str, Path(min_length=1)],` for each router would be enough. In case we accidentally change some detail during the copy paste.
   
   Thanks!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "henry3260 (via GitHub)" <gi...@apache.org>.
henry3260 commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2568930317


##########
task-sdk/src/airflow/sdk/api/client.py:
##########
@@ -460,6 +462,7 @@ def get(
             params.update({"map_index": map_index})
         if include_prior_dates:
             params.update({"include_prior_dates": include_prior_dates})
+        key = quote(key, safe="")

Review Comment:
   > not sure if this is needed on the client side. I don't think so
   
   Thanks for pointing this out!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2557086297


##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/xcom.py:
##########
@@ -80,6 +80,7 @@ def get_xcom_entry(
     stringify: Annotated[bool, Query()] = False,
 ) -> XComResponseNative | XComResponseString:
     """Get an XCom entry."""
+    # FastAPI decodes path params, use the value directly.

Review Comment:
   ```suggestion
   ```



##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/xcom.py:
##########
@@ -314,7 +315,7 @@ def update_xcom_entry(
 ) -> XComResponseNative:
     """Update an existing XCom entry."""
     # Check if XCom entry exists
-    xcom_new_value = XComModel.serialize_value(patch_body.value)
+    # FastAPI decodes path params, use the value directly.

Review Comment:
   Same etc.. to remove



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "henry3260 (via GitHub)" <gi...@apache.org>.
henry3260 commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2553949986


##########
airflow-core/src/airflow/api_fastapi/execution_api/routes/xcoms.py:
##########
@@ -88,111 +88,15 @@ async def xcom_query(
     return query
 
 
-@router.head(

Review Comment:
   I modified the key:path in the router definition to address unintended greedy matching issues in some routes. Previously, these issues caused conflicts that affected routes associated with XComs, as broader path definitions were overriding more specific ones.
    To resolve this, I reordered the router definitions to prioritize XComs and other specific routes, ensuring they are correctly preserved and not inadvertently overridden. Let me know your thoughts or if there are additional areas we should consider reviewing!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2607457530


##########
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_xcom.py:
##########
@@ -646,6 +659,24 @@ def test_should_respond_403(self, unauthorized_test_client):
         )
         assert response.status_code == 403
 
+    def test_create_xcom_entry_with_slash_key(self, test_client):
+        slash_key = "a/b/c"
+        body = XComCreateBody(key=slash_key, value=TEST_XCOM_VALUE)
+        response = test_client.post(
+            f"/dags/{TEST_DAG_ID}/dagRuns/{run_id}/taskInstances/{TEST_TASK_ID}/xcomEntries",
+            json=body.dict(),
+        )
+        assert response.status_code == 201
+        assert response.json()["key"] == slash_key
+        # Verify retrieval via encoded path
+        encoded_key = quote(slash_key, safe="")

Review Comment:
   You don't need those. The test client does the encoding.



##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/xcom.py:
##########
@@ -257,7 +257,7 @@ def create_xcom_entry(
         )
 
     try:
-        value = json.dumps(request_body.value)
+        value = XComModel.serialize_value(request_body.value)

Review Comment:
   This change shouldn't be. It was purposely done in #58900 



##########
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_xcom.py:
##########
@@ -673,7 +704,7 @@ def test_patch_xcom_entry(self, key, patch_body, expected_status, expected_detai
         # Ensure the XCom entry exists before updating
         if expected_status != 404:
             self._create_xcom(TEST_XCOM_KEY, TEST_XCOM_VALUE)
-            new_value = XComModel.serialize_value(patch_body["value"])
+            expected_value = XComModel.serialize_value(patch_body["value"])

Review Comment:
   Why change that?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2610159829


##########
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_xcom.py:
##########
@@ -673,7 +702,8 @@ def test_patch_xcom_entry(self, key, patch_body, expected_status, expected_detai
         # Ensure the XCom entry exists before updating
         if expected_status != 404:
             self._create_xcom(TEST_XCOM_KEY, TEST_XCOM_VALUE)
-            new_value = XComModel.serialize_value(patch_body["value"])
+            # The value is double-serialized: first json.dumps(patch_body["value"]), then json.dumps() again
+            new_value = json.dumps(json.dumps(patch_body["value"]))

Review Comment:
   I mean that code change wasn't needed. It makes it looks like something changed in the test. But it fine



##########
airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_xcom.py:
##########
@@ -673,7 +702,8 @@ def test_patch_xcom_entry(self, key, patch_body, expected_status, expected_detai
         # Ensure the XCom entry exists before updating
         if expected_status != 404:
             self._create_xcom(TEST_XCOM_KEY, TEST_XCOM_VALUE)
-            new_value = XComModel.serialize_value(patch_body["value"])
+            # The value is double-serialized: first json.dumps(patch_body["value"]), then json.dumps() again
+            new_value = json.dumps(json.dumps(patch_body["value"]))

Review Comment:
   I mean that code change wasn't needed. It makes it looks like something changed in the test. But its fine



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun merged PR #58344:
URL: https://github.com/apache/airflow/pull/58344


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2532030669


##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/xcom.py:
##########
@@ -241,6 +246,7 @@ def create_xcom_entry(
             )
 
     # Check existing XCom
+    request_body.key = unquote(request_body.key)

Review Comment:
   Setting this back to request.body feels a little bit out of place. Since this is a JSON body with the key in it, we shouldn't have to do any translation or alteration of the key.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2532034570


##########
airflow-core/src/airflow/api_fastapi/execution_api/routes/xcoms.py:
##########
@@ -240,6 +247,8 @@ def get_mapped_xcom_by_slice(
     params: Annotated[GetXComSliceFilterParams, Query()],
     session: SessionDep,
 ) -> XComSequenceSliceResponse:
+    if key:
+        key = unquote(key)

Review Comment:
   Do we really need the unqoute? I would have expected FastAPI to unquote it before it gives it to us.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "jason810496 (via GitHub)" <gi...@apache.org>.
jason810496 commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2553978553


##########
airflow-core/src/airflow/api_fastapi/execution_api/routes/xcoms.py:
##########
@@ -88,111 +88,15 @@ async def xcom_query(
     return query
 
 
-@router.head(

Review Comment:
   Sorry for the mis-leading in previous review, you are correct!
   
   Based on the CI result, it seems the previous version was correct. It's necessary to reorder the router to avoid the greedy matching problem.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "henry3260 (via GitHub)" <gi...@apache.org>.
henry3260 commented on PR #58344:
URL: https://github.com/apache/airflow/pull/58344#issuecomment-3615392067

   > @henry3260 Do you mind rebasing to fix the conflicts. We can probably go ahead and merge, the PR is looking good.
   
   All CI tests have passed. Ready for merge!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix XCom key handling when keys contain special characters like slash [airflow]

Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #58344:
URL: https://github.com/apache/airflow/pull/58344#discussion_r2610124820


##########
airflow-core/src/airflow/api_fastapi/core_api/routes/public/xcom.py:
##########
@@ -257,7 +257,7 @@ def create_xcom_entry(
         )
 
     try:
-        value = json.dumps(request_body.value)
+        value = XComModel.serialize_value(request_body.value)

Review Comment:
   I think there is a specific PR for that (https://github.com/apache/airflow/pull/59129) to handle double serialization problem. I recommend to not mix things up and not update anything not related to 'xcom key handling with slash' to not make the scope bigger.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org