You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "Cyril Scetbon (JIRA)" <ji...@apache.org> on 2016/06/06 19:44:21 UTC

[jira] [Created] (AIRFLOW-215) Airflow worker (CeleryExecutor) needs to be restarted to pick up tasks

Cyril Scetbon created AIRFLOW-215:
-------------------------------------

             Summary: Airflow worker (CeleryExecutor) needs to be restarted to pick up tasks
                 Key: AIRFLOW-215
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-215
             Project: Apache Airflow
          Issue Type: Bug
    Affects Versions: Airflow 1.7.1.2
            Reporter: Cyril Scetbon


We have a main dag that dynamically creates subdags containing tasks using BashOperator. Using CeleryExecutor we see Celery tasks been created with *STARTED* status but they are not picked up by our worker. However, if we restart our worker, then tasks are picked up. 

Here you can find code if you want to try to reproduce it https://www.dropbox.com/s/8u7xf8jt55v8zio/dags.zip.

We also tested using LocalExecutor and everything worked fine.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)