You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "Dan Davydov (JIRA)" <ji...@apache.org> on 2017/03/21 00:47:41 UTC

[jira] [Created] (AIRFLOW-1019) active_dagruns shouldn't include paused DAGs

Dan Davydov created AIRFLOW-1019:
------------------------------------

             Summary: active_dagruns shouldn't include paused DAGs
                 Key: AIRFLOW-1019
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1019
             Project: Apache Airflow
          Issue Type: Bug
            Reporter: Dan Davydov
            Priority: Critical


Since 1.8.0 Airflow resets orphaned tasks (tasks that are in the DB but not in the executor's memory). The problem is that Airflow counts dagruns in paused DAGs as running as long as the dagruns state is running. Instead we should join against non-paused DAGs everywhere we calculate active dagruns (e.g. in _process_task_instances in the Scheduler class in jobs.py). If there are enough paused DAGs it brings the scheduler to a halt especially on scheduler restarts.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)