You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@airflow.apache.org by "Grant Nicholas (JIRA)" <ji...@apache.org> on 2017/12/04 19:40:00 UTC

[jira] [Created] (AIRFLOW-1884) Ensure scheduler is crash safe for externally triggered and backfilled dagruns

Grant Nicholas created AIRFLOW-1884:
---------------------------------------

             Summary: Ensure scheduler is crash safe for externally triggered and backfilled dagruns
                 Key: AIRFLOW-1884
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1884
             Project: Apache Airflow
          Issue Type: Bug
            Reporter: Grant Nicholas
            Assignee: Grant Nicholas


Orphaned task instances are only reset for dagruns that are both not externally triggered and not backfilled. This violates the crash safety property of the scheduler, ie) if the scheduler crashes in the middle of one of these dagruns then tasks can be stuck in the "Queued" state forever and never executed. 

I found the changeset this regression happened in, it is this one:
https://issues.apache.org/jira/browse/AIRFLOW-1059

This change reverts the special casing logic so that all dagruns have orphaned tasks reset on startup of the scheduler. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)