[jira] [Commented] (AIRFLOW-41) SubdagOperators can oversubscribe to pools due to race condition

["Ry Walker (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-41?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819306#comment-16819306 ] 

Ry Walker commented on AIRFLOW-41:
----------------------------------

Looks like this issue should have closed w/ 
[https://github.com/apache/airflow/pull/1872]

> SubdagOperators can oversubscribe to pools due to race condition
> ----------------------------------------------------------------
>
>                 Key: AIRFLOW-41
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-41
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: scheduler, subdag
>    Affects Versions: 1.7.1
>            Reporter: Bolke de Bruin
>            Priority: Major
>
> SubDagOperators essentially create their own mini-scheduler. Which can interfere with the main scheduler. 
> SubdagOperators check if there is slot available in a Pool. However this slot is not claimed at the same time leaving room for main scheduler to also check for the slot. Both can then obtain a slot and thus oversubscribe
> A solution could be a centralized PoolHandler that gives out slots



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)