You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/02/05 15:32:03 UTC
[GitHub] [airflow] yuqian90 commented on issue #13827: ExternalTaskMarker with user_defined_macros do not work in Airflow 2.0 or with store_serialized_dags = True in Airflow 1.10.*
yuqian90 commented on issue #13827:
URL: https://github.com/apache/airflow/issues/13827#issuecomment-774104828
Tried to discuss this on slack in #airflow-2-0. One possible workaround at the moment is to use `AirflowPlugin` instead of `user_defined_macros`. `AirflowPlugin` works fine in the webserver so jinja templates can be rendered just fine. The only downside is that macros are more cumbersome to use in jinja templates. E.g. with `user_defined_macros`, we could do `{{ next_holiday(execution_date) }}`. But with `AirflowPlugin`, we have to write `{{ macros.my_plugin.next_holiday(execution_date) }}`.
@kaxil suggested the following small change to `dag.clear()` which should make `ExternalTaskMarker` that have already run play well with `dag.clear()` because `get_rendered_template_fields()` reads from the db:
```
diff --git a/airflow/models/dag.py b/airflow/models/dag.py
index 15332f334..30aae0083 100644
--- a/airflow/models/dag.py
+++ b/airflow/models/dag.py
@@ -1260,7 +1260,7 @@ class DAG(LoggingMixin):
max_recursion_depth, ExternalTaskMarker.__name__, ti.task_id
)
)
- ti.render_templates()
+ ti.get_rendered_template_fields()
external_tis = session.query(TI).filter(
TI.dag_id == task.external_dag_id,
TI.task_id == task.external_task_id,
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org