[jira] [Commented] (AIRFLOW-6175) Executor changes queued tasks to scheduled without removing from task queue

["Kaxil Naik (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990033#comment-16990033 ] 

Kaxil Naik commented on AIRFLOW-6175:
-------------------------------------

PR: https://github.com/apache/airflow/pull/6732

> Executor changes queued tasks to scheduled without removing from task queue
> ---------------------------------------------------------------------------
>
>                 Key: AIRFLOW-6175
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-6175
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: scheduler
>    Affects Versions: 1.10.6
>            Reporter: Daniel Imberman
>            Priority: Blocker
>             Fix For: 1.10.6
>
>
> There is a bug caused by scheduler_jobs refactor which leads to task failure and scheduler locking.
> Essentially when a there is an overflow of tasks going into the scheduler, the tasks are set back to scheduled, but are not removed from the executor's queued_tasks queue.
> This means that the executor will attempt to run tasks that are in the scheduled state, but those tasks will fail dependency checks. Eventually the queue is filled with scheduled tasks, and the scheduler can no longer run.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)