You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Daniel Imberman (JIRA)" <ji...@apache.org> on 2019/05/20 15:46:00 UTC
[jira] [Resolved] (AIRFLOW-4393) Add retry logic when fetching pod
status and/or logs in KubernetesPodOperator
[ https://issues.apache.org/jira/browse/AIRFLOW-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Imberman resolved AIRFLOW-4393.
--------------------------------------
Resolution: Fixed
> Add retry logic when fetching pod status and/or logs in KubernetesPodOperator
> -----------------------------------------------------------------------------
>
> Key: AIRFLOW-4393
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4393
> Project: Apache Airflow
> Issue Type: Bug
> Reporter: Olivier Van Goethem
> Priority: Minor
> Fix For: 1.10.4
>
>
> Over the last weeks we have observed 2 occasions where:
> * KubernetesPodOperator successfully launches the pod
> * KubernetesPodOperator then fails when attempting to check the pod status/logs due to:
> {code:java}
> NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f1329dc0cf8>: Failed to establish a new connection: [Errno 111] Connection refused',)
> {code}
> * KubernetesPodOperator launches another pod
> * We now have 2 pods running in parallel
> The cause for the 'Connection refused' is due to a transient network error. As such, the KubernetesPodOperator should attempt to retry checking the pod status/logs rather than failing the task.
> Airflow is being run through GCP's Cloud Composer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)