[jira] [Created] (AIRFLOW-316) Always check DB state for Backfill Job execution

["Dan Davydov (JIRA)" <ji...@apache.org>]

Dan Davydov created AIRFLOW-316:
-----------------------------------

             Summary: Always check DB state for Backfill Job execution
                 Key: AIRFLOW-316
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-316
             Project: Apache Airflow
          Issue Type: Bug
            Reporter: Dan Davydov
            Assignee: Dan Davydov


Always check DB state and not just the local state for backfill jobs for determining which task instances have not yet completed execution. This is to avoid potential race conditions with e.g. two backfill jobs running the same task instance.

At Airbnb we have noticed a race condition with the subdag operator (which uses the backfill job) where all of the tasks in the subdag complete successfully, but the parent DAG is still waiting on a task in the subdag to succeed. Resolving this JIRA should fix the subdag operator.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)