You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Massimo Menichinelli (JIRA)" <ji...@apache.org> on 2017/07/06 10:44:00 UTC
[jira] [Comment Edited] (AIRFLOW-1355) Unable to launch jobs : DAGs
not being executed.
[ https://issues.apache.org/jira/browse/AIRFLOW-1355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076336#comment-16076336 ]
Massimo Menichinelli edited comment on AIRFLOW-1355 at 7/6/17 10:43 AM:
------------------------------------------------------------------------
If I use Airflow (1.8.1) on my machine (macOS Sierra 10.12.5) by launching first the scheduler and then the webserver with SequentialExecutor (i.e. basic default configuration), it works fine. But with docker compose (https://github.com/puckel/docker-airflow), DAGs are launched but not picked by scheduler / worker.
So not sure if the problem is with Airflow or its docker compose. It used to work well until 1-2 weeks ago (and the docker compose hasn't changed), not sure what has changed and caused this.
I have:
Docker: Version 17.06.0-ce-mac18 (18433) Channel: stable
docker-compose: version 1.14.0, build c7bdf9e
I tried with previous versions of Docker: 1.12.6 (14937) and 17.03.1 (16048), the problem is still the same.
DAGs are found and launched in the webserver, but scheduler and worker actually don't run them and the process keeps running:
{{webserver_1 | [2017-07-06 09:42:20,577] [123] {models.py:168} INFO - Filling up the DagBag from /usr/local/airflow/dags
worker_1 | [2017-07-06 09:42:21 +0000] [43] [INFO] Handling signal: ttou
worker_1 | [2017-07-06 09:42:21 +0000] [116] [INFO] Worker exiting (pid: 116)
webserver_1 | [2017-07-06 09:42:21 +0000] [42] [INFO] Handling signal: ttou
webserver_1 | [2017-07-06 09:42:21 +0000] [115] [INFO] Worker exiting (pid: 115)
scheduler_1 | [2017-07-06 09:42:22 +0000] [43] [INFO] Handling signal: ttin}}[link title|http://example.com]
was (Author: openp2pdesign):
If I use Airflow (1.8.1) on my machine (macOS Sierra 10.12.5) by launching first the scheduler and then the webserver with SequentialExecutor (i.e. basic default configuration), it works fine. But with [docker compose](https://github.com/puckel/docker-airflow), DAGs are launched but not picked by scheduler / worker.
So not sure if the problem is with Airflow or its docker compose. It used to work well until 1-2 weeks ago (and the docker compose hasn't changed), not sure what has changed and caused this.
I have:
Docker: Version 17.06.0-ce-mac18 (18433) Channel: stable
docker-compose: version 1.14.0, build c7bdf9e
I tried with [previous versions](https://gist.github.com/franklinyu/5e0bb9d6c0d873f33c78415dd2ea4138) of Docker: 1.12.6 (14937) and 17.03.1 (16048), the problem is still the same.
DAGs are found and launched in the webserver, but scheduler and worker actually don't run them and the process keeps running:
```
webserver_1 | [2017-07-06 09:42:20,577] [123] {models.py:168} INFO - Filling up the DagBag from /usr/local/airflow/dags
worker_1 | [2017-07-06 09:42:21 +0000] [43] [INFO] Handling signal: ttou
worker_1 | [2017-07-06 09:42:21 +0000] [116] [INFO] Worker exiting (pid: 116)
webserver_1 | [2017-07-06 09:42:21 +0000] [42] [INFO] Handling signal: ttou
webserver_1 | [2017-07-06 09:42:21 +0000] [115] [INFO] Worker exiting (pid: 115)
scheduler_1 | [2017-07-06 09:42:22 +0000] [43] [INFO] Handling signal: ttin
```
> Unable to launch jobs : DAGs not being executed.
> ------------------------------------------------
>
> Key: AIRFLOW-1355
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1355
> Project: Apache Airflow
> Issue Type: Bug
> Components: DAG
> Affects Versions: Airflow 2.0
> Environment: Mac OS and Ubuntu
> Reporter: Pavan KN
>
> Steps to re-produce:
> 1. Create new installation
> 2. Launch Airflow
> 3. Enable a DAG and trigger it manually
> DAG/Job won't get executed. Will stay in Running status, but no execution starts and continues to stay at same status.
> Same issues is there with Sequential, Local and Celeri executors.
> Happening in 2.0 version. Tried on multiple Mac machines and on Ubuntu.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)