You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "Dan Davydov (JIRA)" <ji...@apache.org> on 2017/03/21 21:18:41 UTC

[jira] [Created] (AIRFLOW-1024) Handle CeleryExecutor errors gracefully

Dan Davydov created AIRFLOW-1024:
------------------------------------

             Summary: Handle CeleryExecutor errors gracefully
                 Key: AIRFLOW-1024
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1024
             Project: Apache Airflow
          Issue Type: Bug
            Reporter: Dan Davydov


If the Airflow celery executor receives a bad response from a worker (e.g. unpickled response), then it will crash the scheduler and cause it to restart.

We should code defensively around the interactions with celery so that we just log errors instead of crashing the scheduler.

It might makes sense to make the try catches one level higher (to catch errors from all executors), but this needs some investigation.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)