You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "jroachgolf84 (via GitHub)" <gi...@apache.org> on 2026/05/08 12:34:29 UTC

[PR] `fix/issue-66592`: Respecting `unwrap_single` for non-deferrable execution [airflow]

jroachgolf84 opened a new pull request, #66596:
URL: https://github.com/apache/airflow/pull/66596

   ## Description
   
   According to #66592:
   
   > When using `KubernetesJobOperator` with `do_xcom_push=True`, the task returns the content of JSON file found at `/airflow/xcom/return.json` in k8s pod as a return_value XCom, but the data type of the returned XCom value varies depending on whether the deferrable paramter was set to `True` or `False`:
   > 
   > - When `deferrable=True` parameter is supplied, `KubernetesJobOperator` returns XCom value with data type of `dict`
   > - When `deferrable=False` parameter is supplied, `KubernetesJobOperator` returns XCom value with data type of `list[dict]`
   
   This PR remedies this issue by "unwrapping" the `xcom_result` if the following condition is met: `if self.unwrap_single and len(xcom_result) == 1`. This ensures that the behavior for non-deferrable execution matches that of deferrable execution, providing parity for users.
   
   ## Testing
   
   These changes were testing using the newly-created `test_execute_xcom_respects_unwrap_single` unit test. This test can be run using the command below.
   
   ```bash
   breeze testing providers-tests providers/cncf/kubernetes/tests/unit/cncf/kubernetes/operators/test_job.py
   ```
   
   * closes: #66592 
   
   ##### Was generative AI tooling used to co-author this PR?
   
   No, generative AI was not used to co-author this PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] `fix/issue-66592`: Respecting `unwrap_single` for non-deferrable execution [airflow]

Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl merged PR #66596:
URL: https://github.com/apache/airflow/pull/66596


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] `fix/issue-66592`: Respecting `unwrap_single` for non-deferrable execution [airflow]

Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #66596:
URL: https://github.com/apache/airflow/pull/66596#discussion_r3210917768


##########
providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/operators/job.py:
##########
@@ -255,7 +255,7 @@ def execute(self, context: Context):
                     f"Kubernetes job '{self.job.metadata.name}' is failed with error '{error_message}'"
                 )
             if self.do_xcom_push:
-                return xcom_result
+                return xcom_result[0] if self.unwrap_single and len(xcom_result) == 1 else xcom_result

Review Comment:
   Danger! if `xcom_result` is a dict with exactly one entry this will fail. So you would need to check type.
   
   Devils advocate: What is the container (by intend) returns a list with one entry as XCom? Then you would un-wrap.
   
   I assume the error is generated if multiple containers launch, is there a possibility to get the count of containers to prevent such side effects that specific content is by accident un-wrapped?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] `fix/issue-66592`: Respecting `unwrap_single` for non-deferrable execution [airflow]

Posted by "jroachgolf84 (via GitHub)" <gi...@apache.org>.
jroachgolf84 commented on code in PR #66596:
URL: https://github.com/apache/airflow/pull/66596#discussion_r3211198363


##########
providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/operators/job.py:
##########
@@ -255,7 +255,7 @@ def execute(self, context: Context):
                     f"Kubernetes job '{self.job.metadata.name}' is failed with error '{error_message}'"
                 )
             if self.do_xcom_push:
-                return xcom_result
+                return xcom_result[0] if self.unwrap_single and len(xcom_result) == 1 else xcom_result

Review Comment:
   Thanks for the catch! I think that's a good question. Can you provide a more concrete example? For example, say that 8 containers launch. Then what? 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] `fix/issue-66592`: Respecting `unwrap_single` for non-deferrable execution [airflow]

Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #66596:
URL: https://github.com/apache/airflow/pull/66596#discussion_r3213441940


##########
providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/operators/job.py:
##########
@@ -255,7 +255,7 @@ def execute(self, context: Context):
                     f"Kubernetes job '{self.job.metadata.name}' is failed with error '{error_message}'"
                 )
             if self.do_xcom_push:
-                return xcom_result
+                return xcom_result[0] if self.unwrap_single and len(xcom_result) == 1 else xcom_result

Review Comment:
   Okay, was reading more context of the code above the +/- lines shown in Github diff and now realize that the `xcom_result` is initialized as a list so then it is safe and consistent to apply.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org