[jira] [Created] (AIRFLOW-380) Cannot cancel "catch-up" dag runs after unpausing a dag (or restarting scheduler after long)

["Vineet Goel (JIRA)" <ji...@apache.org>]

Vineet Goel created AIRFLOW-380:
-----------------------------------

             Summary: Cannot cancel "catch-up" dag runs after unpausing a dag (or restarting scheduler after long)
                 Key: AIRFLOW-380
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-380
             Project: Apache Airflow
          Issue Type: Improvement
            Reporter: Vineet Goel


Currently when we unpause a dag or bring the scheduler back up after long, the scheduler starts scheduler dag runs for each execution in between (for the time that has elapsed). There should be a way to cancel these "backfills" if they are not really required.

While this is simpler to clear using the cli if the dagruns are created simultaneously, it becomes a bit hard if the max_active_runs for the dag is 1. This way, we need to clear each dag run individually which isn't ideal. One workaround is to keep start_date as datetime.now() and deleting past dag runs from the database prior to unpausing the dag. However this isn't ideal.

Is there a way of doing that I don't know about?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)