You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Dobromir Montauk (Jira)" <ji...@apache.org> on 2020/05/18 05:45:00 UTC
[jira] [Assigned] (AIRFLOW-6782) airflow-xcom-sidecar container
prevents pods from completion
[ https://issues.apache.org/jira/browse/AIRFLOW-6782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dobromir Montauk reassigned AIRFLOW-6782:
-----------------------------------------
Assignee: Dobromir Montauk
> airflow-xcom-sidecar container prevents pods from completion
> ------------------------------------------------------------
>
> Key: AIRFLOW-6782
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6782
> Project: Apache Airflow
> Issue Type: Bug
> Components: xcom
> Affects Versions: 1.10.7
> Reporter: Maksym Shalak
> Assignee: Dobromir Montauk
> Priority: Major
> Attachments: Screenshot from 2020-02-12 16-19-49.png
>
>
> {color:#172b4d}I launch tasks using KubernetesPodOperator with do_xcom_push=True.{color}
> {color:#172b4d}This creates my container with an airflow-xcom-sidecar container inside a pod. {color}
> {color:#172b4d}In 90% of cases that works fine, but sometimes my main container completes, but airflow-xcom-sidecar is still running and never completes, which prevents my pod from completion. {color}
> {color:#172b4d}I see nothing in airflow-xcom-sidecar co{color}ntainer logs. The data which I want to write is inside /airflow/xcom/return.json, so no bug there.
> {color:#172b4d}Looks like this issue is common: [https://stackoverflow.com/questions/54388441/kubernetes-pod-created-through-airflow-remains-in-running-state] {color}
> {color:#172b4d}Let me know if any additional information needed.{color}
>
> In results of *kubectl describe pod* I see that base container terminated, but{color:#172b4d} airflow-xcom-sidecar{color} is still in progress:
> {code:java}
> Containers:
> base:
> Container ID: docker://e50ffc8e47ca6b75176732a4eb5dee6bd9d7decbf7c272acbd2dadc94034a807
> ...
> State: Terminated
> Reason: Completed
> Exit Code: 0
> Started: Wed, 12 Feb 2020 14:27:23 +0200
> Finished: Wed, 12 Feb 2020 14:28:19 +0200
> Ready: False
> Restart Count: 0
> Requests:
> cpu: 300m
> memory: 500Mi
> Environment: <none>
> Mounts:
> /airflow/xcom from xcom (rw)
> airflow-xcom-sidecar:
> Container ID: docker://f5a1a4d14bf1820387f5ed21541b4de3b5f89d067e4f91d18909d9a266c26696
> Image: alpine
> Image ID: docker-pullable://alpine@sha256:ddba4d27a7ffc3f86dd6c2f92041af252a1f23a8e742c90e6e1297bfa1bc0c45
> Port: <none>
> Host Port: <none>
> Command:
> sh
> -c
> trap "exit 0" INT; while true; do sleep 30; done;
> State: Running
> Started: Wed, 12 Feb 2020 14:27:24 +0200
> Ready: True
> Restart Count: 0
> Requests:
> cpu: 1m
> Environment: <none>
> Mounts:
> /airflow/xcom from xcom (rw)
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)