[jira] [Commented] (AIRFLOW-1488) Add a sensor operator to wait on DagRuns

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16927757#comment-16927757 ] 

ASF GitHub Bot commented on AIRFLOW-1488:
-----------------------------------------

stale[bot] commented on pull request #4291: [AIRFLOW-1488] Add the TriggeredDagRunSensor operator
URL: https://github.com/apache/airflow/pull/4291
 
 
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Add a sensor operator to wait on DagRuns
> ----------------------------------------
>
>                 Key: AIRFLOW-1488
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1488
>             Project: Apache Airflow
>          Issue Type: New Feature
>          Components: contrib, operators
>            Reporter: Yati
>            Assignee: Yati
>            Priority: Major
>
> The [ExternalTaskSensor|https://airflow.incubator.apache.org/code.html#airflow.operators.ExternalTaskSensor] operator already allows for encoding dependencies on tasks in external DAGs. However, when you have teams, each owning multiple small-to-medium sized DAGs, it is desirable to be able to wait on an external DagRun as a whole. This allows the owners of an upstream DAG to refactor their code freely by splitting/squashing task responsibilities, without worrying about dependent DAGs breaking.
> I'll now enumerate the easiest ways of achieving this that come to mind:
> * Make all DAGs always have a join DummyOperator in the end, with a task id that follows some convention, e.g., "{{ dag_id }}.__end__".
> * Make ExternalTaskSensor poke for a DagRun instead of TaskInstances when the external_task_id argument is None.
> * Implement a separate DagRunSensor operator.
> After considerations, we decided to implement a separate operator, which we've been using in the team for our workflows, and I think it would make a good addition to contrib.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)