You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2019/09/11 15:28:01 UTC
[jira] [Commented] (AIRFLOW-3902) Singleton operator to skip
downstream tasks
[ https://issues.apache.org/jira/browse/AIRFLOW-3902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16927687#comment-16927687 ]
ASF GitHub Bot commented on AIRFLOW-3902:
-----------------------------------------
stale[bot] commented on pull request #4727: [AIRFLOW-3902] Add new singleton_operator and unittest
URL: https://github.com/apache/airflow/pull/4727
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Singleton operator to skip downstream tasks
> -------------------------------------------
>
> Key: AIRFLOW-3902
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3902
> Project: Apache Airflow
> Issue Type: New Feature
> Components: operators
> Reporter: Kiran Pathak
> Assignee: Kiran Pathak
> Priority: Major
> Labels: operator, singleton
> Attachments: image-2019-02-18-05-03-00-290.png, image-2019-02-18-05-03-58-662.png
>
>
> This operator will:
> * check for downstream tasks from previous DagRuns in *_unfinished_* state
> * skip downstream tasks from execution if it finds a downstream task in running state from previous DagRuns
> This feature ensures downstream children of this operator only one instance is scheduled to run. It avoids unnecessary processing / scheduling if it does not need to.
> !image-2019-02-18-05-03-00-290.png|width=213,height=102!
> !image-2019-02-18-05-03-58-662.png|width=224,height=88!
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)