[jira] [Commented] (AIRFLOW-1825) Set Multi dag dependency

["Song Yikun (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AIRFLOW-1825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17038155#comment-17038155 ] 

Song Yikun commented on AIRFLOW-1825:
-------------------------------------

Not sure this is still relevant (otherwise it should be closed?), I think this could be achieved by just two sequential ExternalTaskSensors? `wait_dag_a` and followed by `wait_dag_b`, or schedule them both as upstream relative to DAG C.  

 

i.e.

 

wait_dag_a >> wait_dag_b >> dag_c

(note the order does not matter because you need both)

 

wait_dag_a >> dag_c

wait_dag_b >> dag_c 

 

There should not be a lot of `overhead` mentioned above if you use relative big DAG, otherwise, it might be a good idea to concatenate your DAGs.

> Set Multi dag dependency 
> -------------------------
>
>                 Key: AIRFLOW-1825
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1825
>             Project: Apache Airflow
>          Issue Type: Task
>            Reporter: Snigdha Nair
>            Priority: Major
>
> I have 3 dags A, B and C. Dag C should get triggered only after tasks in dag A and B completes. Is there a way to implement this in airflow? I am able to set dependency between dag A and C using Triggerdagrun Operator. But when I try to set dependency between dag B and C, C is getting triggered when either A or B completes. Can someone please help me in solving this? I understand that explains external task sensor Operator can be used. But it continuously polls if task in dag A and B is complete which might create performance hit over a period of time.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)