You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Felix Uellendall (JIRA)" <ji...@apache.org> on 2018/09/01 09:12:00 UTC
[jira] [Commented] (AIRFLOW-2934) Pools not respected for internal
subdag tasks
[ https://issues.apache.org/jira/browse/AIRFLOW-2934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599585#comment-16599585 ]
Felix Uellendall commented on AIRFLOW-2934:
-------------------------------------------
This is a [known bug|https://issues.apache.org/jira/browse/AIRFLOW-41] with SubDag's and LocalExecutor.
In Airflow's 1.10 Release there is a quick fix where the SubDagOperator's executor param default value has been set to SequentialExecutor to workaround this issue.
But of course if you dont want to upgrade you can set it by [yourself|https://github.com/apache/incubator-airflow/blob/master/airflow/operators/subdag_operator.py#L50]
> Pools not respected for internal subdag tasks
> ---------------------------------------------
>
> Key: AIRFLOW-2934
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2934
> Project: Apache Airflow
> Issue Type: Bug
> Components: pools, subdag
> Affects Versions: 1.9.0
> Environment: Linux victorvianna 4.14.65-1-MANJARO #1 SMP PREEMPT Sat Aug 18 13:29:56 UTC 2018 x86_64 GNU/Linux
> Python 3.6.6
> Reporter: Victor Vianna
> Priority: Blocker
> Labels: pool, subdag
> Attachments: Screenshot from 2018-08-22 12-32-32.png, dag_pool.py
>
>
> I'm trying to have some subdags execute one task at a time. The way I found was to create a first pool for the SubdagOperators (pool1 in the attached code file) and a second one for the internal tasks (pool2). However, it appears that pools for subdag elements are not being respected. Running airflow 1.9.0 with LocalExecutor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)