[GitHub] [airflow] Sharadh commented on issue #6317: [AIRFLOW-5644] Simplify TriggerDagRunOperator usage

[GitBox <gi...@apache.org>]

Sharadh commented on issue #6317: [AIRFLOW-5644] Simplify TriggerDagRunOperator usage
URL: https://github.com/apache/airflow/pull/6317#issuecomment-596043701
 
 
   Greetings! I'm curious to know if you folks knew this change reduced functionality. Specifically, we have workflows where the `python_callable` was useful with two things:
   
   1. Dynamically generate the `conf` required for the `trigger_dag` call
   2. Return a false-y value so the `trigger_dag` call does not take place
   
   I am not sure how this can be done after the change.
   
   In general, having the convenience of an arbitrary python callable to hook into and modify behavior based on incoming conf is very valuable. For a practical example, this task would trigger a dag only if a flag was set in the conf; this flag can vary between dag runs, but the same dag can model both behaviors:
   
   ```py
   
   step1 = SomeOperator()
   
   step2 = AnotherOperator()
   
   def check_and_trigger(context, dag_run_obj):
       payload = context["dag_run"].conf
   
       if not payload["should_trigger"]:
           return False
   
       dag_run_obj.payload = payload["downstream_payload"]
       return dag_run_obj
   
   maybe_trigger_bar_dag = TriggerDagRunOperator(
       trigger_dag_id="bar",
       python_callable=check_and_trigger,
   )
   
   step1 >> step2 >> maybe_trigger_bar_dag
   ```
   In our use-case, the Dag itself is static but takes in a few parameters via `conf`, which comes in via the experimental API or from another scheduled dag. It helps us reuse dags without getting into the gnarly world of sub-dags.
   
   Please let me know if I can explain things further. I was unable to find the motivation for this change apart from the Jira ticket linked, so please do point me to more reading if it exists, so I can gain context.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services