[GitHub] [airflow] BasPH commented on issue #6317: [AIRFLOW-5644] Simplify TriggerDagRunOperator usage

[GitBox <gi...@apache.org>]

BasPH commented on issue #6317: [AIRFLOW-5644] Simplify TriggerDagRunOperator usage
URL: https://github.com/apache/airflow/pull/6317#issuecomment-596129144
 
 
   Hi @Sharadh,
   
   The main motivation for this change was code clarity. Back then, I found the `TriggerDagRunOperator` very awkward to read and the `context` and `dag_run_obj` seem to come out of nowhere. You are correct in that it reduces the functionality, however I still think this change makes the code much better readable and makes the expected result clear.
   
   To achieve the same, I suggest to split your task into two:
   
   (1) a PythonOperator which succeeds if `should_trigger==True`, else raises `AirflowSkipException`
   (2) the TriggerDagRunOperator
   
   ```python
   from airflow.exceptions import AirflowSkipException
   
   step1 = SomeOperator()
   step2 = AnotherOperator()
   
   def _should_trigger(dag_run, **_):
       if not dag_run.conf["should_trigger"]:
           raise AirflowSkipException("should_trigger set to False")
   
   should_trigger = PythonOperator(
       task_id="should_trigger",
       python_callable=_should_trigger,
       provide_context=True,
   )
   
   trigger_bar_dag = TriggerDagRunOperator(
       trigger_dag_id="bar",
       conf={"downstream_payload": "{{ dag_run.conf['downstream_payload'] }}"},
   )
   
   step1 >> step2 >> should_trigger >> trigger_bar_dag
   ```
   
   Since you're passing the payload to the to-be-triggered-DAG, the `conf` arguments gets a little weird. It's templated, so I think the code above should do the trick. I didn't test it, so use at your own risk :-)
   
   The `should_trigger` task now determines whether or not to continue. If not, it will raise an AirflowSkipException, which sets its state to skipped, and automatically skips downstream tasks (assuming they have the default TriggerRule).
   
   Does this clarify?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services