[jira] [Work started] (AIRFLOW-6166) KubernetesPodOperator resolve default xcom read behavior or improve associated documentation

["Daniel Imberman (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on AIRFLOW-6166 started by Daniel Imberman.
------------------------------------------------
> KubernetesPodOperator resolve default xcom read behavior or improve associated documentation
> --------------------------------------------------------------------------------------------
>
>                 Key: AIRFLOW-6166
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-6166
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: documentation, executor-kubernetes, xcom
>    Affects Versions: 1.10.7
>         Environment: Ubuntu 16.04, minikube version: v1.5.2
>            Reporter: Cabell Maddux
>            Assignee: Daniel Imberman
>            Priority: Minor
>              Labels: kubernetes, xcom
>
> Hi,
> I'm relatively new to airflow and am testing out Airflow running with the Kubernetes executor locally on Minikube based off of the CI configs in the repo ([https://github.com/apache/airflow/tree/master/scripts/ci/kubernetes/app]).
> Important to note that airflow on minikube using image: apache/airflow:master-python3.7-ci
> Was able to get everything up and running smoothly, however thought default behavior for xcom when no `do_xcom_push` argument is provided to a `KubernetesPodOperator` was a bit confusing.
> If a user doesn't explicitly create a result.json in the pod default `XCOM_MOUNT_PATH` as part of the task pod, obviously the task will fail ([https://github.com/apache/airflow/blob/a36cfe049a2c5948b24fde7a878fe19cabede5f7/airflow/kubernetes/pod_launcher.py#L214)]
> ```
> cat: can't open '/airflow/xcom/return.json': No such file or directory
> ```
> Rather than the user having to explicitly create result.json as part of their task, maybe more aligned with other operators to me to create `\{XCOM_MOUNT_PATH}/result.json` when creating the xcom sidecar - [https://github.com/apache/airflow/blob/a36cfe049a2c5948b24fde7a878fe19cabede5f7/airflow/kubernetes/pod_generator.py#L36].
> Otherwise, maybe would be nice to document that user of KubernetesPodOperator should be careful to explicitly deal with `do_xcom_push` to avoid a bit of confusion.
> Happy to open a PR if this is something that you think needs to be dealt with either in code or via docs. Also found some older tests associated with this ([https://github.com/apache/airflow/blob/36f3bfb0619cc78698280f6ec3bc985f84e58343/tests/contrib/minikube/test_kubernetes_pod_operator.py#L321)|https://github.com/apache/airflow/blob/36f3bfb0619cc78698280f6ec3bc985f84e58343/tests/contrib/minikube/test_kubernetes_pod_operator.py#L321-]
> If already in the works, no worries.
> Thanks!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)