[jira] [Commented] (AIRFLOW-56) Airflow's scheduler can "loose" queued tasks

["Bence Nagy (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-56?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15273954#comment-15273954 ] 

Bence Nagy commented on AIRFLOW-56:
-----------------------------------

I think we're hitting this issue as well. Let me know if there's any details that could be useful for debugging.

> Airflow's scheduler can "loose" queued tasks
> --------------------------------------------
>
>                 Key: AIRFLOW-56
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-56
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: executor, scheduler
>    Affects Versions: Airflow 1.7.0
>            Reporter: Bolke de Bruin
>            Priority: Blocker
>             Fix For: Airflow 1.7.1
>
>
> Tasks that get into queued status can get "lost" on the scheduler/executor. They remain in the DB with state "queued", but the scheduler won't pick them up as it will not look at the DB currently. The executor does not see those tasks, although the scheduler has NOT been restarted (ie. to see this issue it is not required to have "-n x" as a parameter to the scheduler).
> This happens *also* to tasks scheduled by the scheduler and not just with backfills created by the SubDagOperator.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)