[jira] [Assigned] (AIRFLOW-5686) Deleting DAG can leave scheduled/queued tasks consuming pool slots

["Sajid Sajid (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AIRFLOW-5686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sajid Sajid reassigned AIRFLOW-5686:
------------------------------------

    Assignee: Sajid Sajid

> Deleting DAG can leave scheduled/queued tasks consuming pool slots
> ------------------------------------------------------------------
>
>                 Key: AIRFLOW-5686
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-5686
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 1.10.5
>            Reporter: Ash Berlin-Taylor
>            Assignee: Sajid Sajid
>            Priority: Critical
>             Fix For: 2.0.0
>
>
> If you delete a dag file when it had tasks in the scheduled or queued state, those tasks instances are never touched again.
> With the slight tweak in Pool (making the default pool an explicit one) this now matters, and this ends up with the scheduler being "blocked" from running new tasks wiht this message:
> {noformat}
> Figuring out tasks to run in Pool(name=default_pool) with -9022 open slots and 45 task instances ready to be queued {noformat}
> The fix should be to set any task instance in a non-terminal state (None, queued, scheduled, running, up_for_retry etc.) to "removed" inside DAG.deactivate_stale_dags (which is already called on scheduler shutdown).



--
This message was sent by Atlassian Jira
(v8.20.1#820001)