[jira] [Created] (AIRFLOW-627) Tasks getting Queued when Pool is full sometimes never get run

["Ben Tallman (JIRA)" <ji...@apache.org>]

Ben Tallman created AIRFLOW-627:
-----------------------------------

             Summary: Tasks getting Queued when Pool is full sometimes never get run
                 Key: AIRFLOW-627
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-627
             Project: Apache Airflow
          Issue Type: Bug
          Components: scheduler
    Affects Versions: Airflow 1.8
         Environment: Celery Executor, Master Branch, Postgres
            Reporter: Ben Tallman


[2016-11-14 10:54:04,174] {models.py:168} INFO - Filling up the DagBag from /opt/efs/airflow/dags/crawl_traffic_prod.py
[2016-11-14 10:54:07,562] {base_hook.py:67} INFO - Using connection to: db.xyz.com
[2016-11-14 10:54:07,667] {base_hook.py:67} INFO - Using connection to: db.xyz.com
[2016-11-14 10:54:27,214] {models.py:168} INFO - Filling up the DagBag from /opt/efs/airflow/dags/crawl_traffic_prod.py
[2016-11-14 10:54:30,150] {base_hook.py:67} INFO - Using connection to: db.xyz.com
[2016-11-14 10:54:30,311] {base_hook.py:67} INFO - Using connection to: db.xyz.com
[2016-11-14 10:54:32,438] {models.py:1072} INFO - Dependencies all met for <TaskInstance: crawl_traffic_prod_pod.orgs_beginning_with_350 2016-11-13 01:00:00 [queued]>
[2016-11-14 10:54:32,700] {models.py:1069} WARNING - Dependencies not met for <TaskInstance: crawl_traffic_prod_pod.orgs_beginning_with_350 2016-11-13 01:00:00 [queued]>, dependency 'DAG's Pool Has Space' FAILED: Task's pool 'prod_pod_crawler' is full.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)