[jira] [Commented] (AIRFLOW-1074) Do not count queued tasks in scheduler concurrency check

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-1074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15966400#comment-15966400 ] 

ASF subversion and git services commented on AIRFLOW-1074:
----------------------------------------------------------

Commit 8f9f5084bfdc2aa4017fee12e22d2e94672765ba in incubator-airflow's branch refs/heads/master from [~saguziel]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=8f9f508 ]

[AIRFLOW-1074] Don't count queued tasks for concurrency limits

There may be orphaned tasks queued but not in a
running dag run that
will not cleared. We should not count these as
they will interfere.

I hate to do this, but I changed my mind on
counting queued tasks.

1. Queued tasks that are actually queued generally
get set to running pretty quickly.
2. Because of the worker-side check, we won't
actually pass concurrency.

I don't think the queued thing is a big deal
because of this, I'm more worried about orphaned
tasks that are in QUEUED state but not in a
running dag_run (so they wont get reset)
interfering with concurrency.

There may be orphaned tasks queued but not in a
running dag run that
will not cleared. We should not count these as
they will interfere.

Closes #2221 from saguziel/aguziel-concurrency-2


> Do not count queued tasks in scheduler concurrency check
> --------------------------------------------------------
>
>                 Key: AIRFLOW-1074
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1074
>             Project: Apache Airflow
>          Issue Type: Bug
>            Reporter: Alex Guziel
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)